Inkling by Thinking Machines now available on Modal

Summary

Thinking Machines released Inkling, a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. The 975B-parameter mixture-of-experts transformer activates 41B parameters per token, uses a 1M-token context window, and employs local attention: five sliding window layers for every global layer. Modal provided day-zero support with a custom DFlash block-diffusion speculator, achieving 250 tokens per second per user on 8x B200s with 67% faster interactivity than the model’s built-in multi-token prediction path. The DFlash drafter uses all local attention and causal layers, keeping cost flat as block size grows. Inkling is available today as an OpenAI-compatible shared endpoint through Modal Auto Endpoints with token-based pricing, allowing users to try the model interactively.

Takeaways

1. 1

   Inkling is a 975B-parameter mixture-of-experts transformer with 41B active parameters, a 1M-token context window, and a local attention layout with five sliding window layers per global layer.

2. 2

   On 8x B200s with Modal’s DFlash speculator, Inkling reached 250 tokens per second per user and 67% faster interactivity than its built-in multi-token prediction path at matched throughput.

3. 3

   The DFlash drafter uses all local attention and causal layers, enabling flat cost scaling as block size increases, which suits models shifting toward local attention.

FAQ