---
title: "Kimi K3 by Moonshot now available on Modal"
description: "Moonshot released Kimi K3, a 2.8 trillion parameter multimodal mixture-of-experts transformer with a 1 million token context window and native vision, now on Modal. Modal runs the model at 460 tokens per second on release day through a partnership with Moonshot and vLLM, with token-based pricing on a Shared API or dedicated Auto Endpoint. K3 activates 16 of 896 experts per token and uses Kimi Delta Attention along with Attention Residuals to achieve roughly 2.5 times the scaling efficiency of its predecessor K2. Moonshot employed quantization-aware training from the SFT stage with MXFP4 weights and MXFP8 activations, and contributed a new prefix caching implementation to vLLM to address challenges from KDA. Modal deployed a custom DFlash speculative decoding drafter tuned to K3’s architecture, yielding 360 percent faster interactivity and 88 percent higher throughput on agentic workloads."
---

# Kimi K3 by Moonshot now available on Modal

[Modal](https://yomu.fyi/company/modal) · Jul 27, 2026

**Type:** Announcement

## Summary

Moonshot released Kimi K3, a 2.8 trillion parameter multimodal mixture-of-experts transformer with a 1 million token context window and native vision, now on Modal. Modal runs the model at 460 tokens per second on release day through a partnership with Moonshot and vLLM, with token-based pricing on a Shared API or dedicated Auto Endpoint. K3 activates 16 of 896 experts per token and uses Kimi Delta Attention along with Attention Residuals to achieve roughly 2.5 times the scaling efficiency of its predecessor K2. Moonshot employed quantization-aware training from the SFT stage with MXFP4 weights and MXFP8 activations, and contributed a new prefix caching implementation to vLLM to address challenges from KDA. Modal deployed a custom DFlash speculative decoding drafter tuned to K3’s architecture, yielding 360 percent faster interactivity and 88 percent higher throughput on agentic workloads.

## Context

Moonshot released Kimi K3, an open 2.8 trillion parameter multimodal mixture-of-experts model with a 1 million token context window. Modal provides day-zero inference support at 460 tokens per second, with token-based pricing and dedicated capacity, plus a custom DFlash speculative decoding drafter tuned to K3’s architecture for faster agentic workloads.

## Approach / What changed

Modal makes Kimi K3 available via an OpenAI-compatible Shared API with token-based pricing and as a dedicated Auto Endpoint. They deployed a custom DFlash speculative decoding drafter trained using 32 B300 nodes (28 running K3 at TP8 to produce hidden states, 4 training the draft model) to accelerate decoding, achieving substantial speedups on agentic tasks.

## Takeaways

- Kimi K3 is a 2.8 trillion parameter mixture-of-experts transformer with 16 of 896 experts active per token, a 1M token context window, and native vision.
- Modal achieves 460 tokens per second on K3 inference using a custom DFlash speculative decoding drafter, resulting in 360% faster interactivity and 88% higher throughput on agentic workloads.
- Moonshot used quantization-aware training with MXFP4 weights and MXFP8 activations, and contributed a new prefix caching implementation to vLLM to overcome issues with Kimi Delta Attention.

## FAQ

### How fast is Kimi K3 on Modal?

Modal runs K3 at 460 tokens per second on release day. With the custom DFlash speculator, agentic workloads see 360% faster interactivity (from 100 to 460 tokens/s) and 88% higher throughput (from 800k to 1.5 million TPM per GPU).

### What hardware was used to train the DFlash speculator?

The final training run used 32 B300 nodes: 28 running Kimi K3 at TP8 to produce hidden states, and 4 training the draft model against those hidden states.

### What architectural innovations does Kimi K3 use?

K3 uses Kimi Delta Attention (KDA) and Attention Residuals, which together provide roughly 2.5 times the scaling efficiency of K2. KDA broke conventional prefix caching, so Moonshot wrote a new implementation and contributed it to vLLM.

**Tags:** [Inference](https://yomu.fyi/topic/inference), [Kimi K3](https://yomu.fyi/topic/kimi-k3), [Mixture of Experts](https://yomu.fyi/topic/mixture-of-experts), [Modal](https://yomu.fyi/topic/modal), [Speculative Decoding](https://yomu.fyi/topic/speculative-decoding)

- Source: [Modal](https://modal.com/blog/kimi-k3-by-moonshot-now-available-on-modal)
- Source URL: https://modal.com/blog/kimi-k3-by-moonshot-now-available-on-modal
- Ingested by Yomu: 2026-09-20T07:47:15.323Z

[Read original post](https://modal.com/blog/kimi-k3-by-moonshot-now-available-on-modal)