---
title: "Mixture-of-Kittens: our open-source MoE megakernel for NVL72s"
description: "Mixture-of-Kittens (MoK) is an open-source production mixture-of-experts (MoE) training megakernel designed for GB300 NVL72 systems, where the MoE layer had become a major training bottleneck and could consume more than half of end-to-end training time. It redesigns the layer around a single fully deterministic kernel that fuses communication and computation, minimizing CPU work and CPU-GPU synchronization while overlapping operations across the 72-GPU NVLink domain. The post describes target-specific scheduling, push-versus-pull communication choices, pipelined expert computation, ring token buffers, megakernel structure, determinism, and MXFP8 support for DeepSeek-V3-style layers. Across four model shapes, MoK is reported as up to 2.37x faster for MXFP8 forward and 1.78x for backward, while production training on 512 GPUs reached 1.41x tokens per second versus a DeepEP-based implementation. MoK is fully open-sourced and intended to remain modifiable for other platforms."
---

# Mixture-of-Kittens: our open-source MoE megakernel for NVL72s

[Cursor](https://yomu.fyi/company/cursor) · Stuart Sul, Nash Brown, Henry Wildermuth, William Lin & Federico Cassano · Aug 4, 2026

**Type:** Problem & solution

## Summary

Mixture-of-Kittens (MoK) is an open-source production mixture-of-experts (MoE) training megakernel designed for GB300 NVL72 systems, where the MoE layer had become a major training bottleneck and could consume more than half of end-to-end training time. It redesigns the layer around a single fully deterministic kernel that fuses communication and computation, minimizing CPU work and CPU-GPU synchronization while overlapping operations across the 72-GPU NVLink domain. The post describes target-specific scheduling, push-versus-pull communication choices, pipelined expert computation, ring token buffers, megakernel structure, determinism, and MXFP8 support for DeepSeek-V3-style layers. Across four model shapes, MoK is reported as up to 2.37x faster for MXFP8 forward and 1.78x for backward, while production training on 512 GPUs reached 1.41x tokens per second versus a DeepEP-based implementation. MoK is fully open-sourced and intended to remain modifiable for other platforms.

## Context

As Composer training and inference scaled, the mixture-of-experts layer consistently remained a major bottleneck, consuming more than half of end-to-end training time in some workloads. On GB300 NVL72 systems, communication became the limiting factor, while GPU streams could catch up to relatively slow Grace CPU-side work and leave GPUs idle.

## Approach / What changed

MoK redesigns the full MoE layer as a single fully deterministic megakernel that fuses communication and computation. It targets NVL72's 72-GPU NVLink domain with communication-computation overlap, target-specific push or pull scheduling, ring token buffers, reduced CPU-GPU synchronization, and MXFP8 support.

## Takeaways

- MoK increased end-to-end tokens per second by 1.41x on a 512-GPU production training stack compared with its previous DeepEP-based implementation.
- Against the fastest public baselines, MoK is reported as up to 2.37x faster for MXFP8 forward, 1.78x for MXFP8 backward, 1.92x for BF16 forward, and 1.58x for BF16 backward.
- The kernel targets DeepSeek-V3-style MoE layers and combines routed experts, a shared expert, expert-parallel communication, and feed-forward computation in one deterministic execution path.

**Tags:** [Distributed Systems](https://yomu.fyi/topic/distributed-systems), [GPU](https://yomu.fyi/topic/gpu), [MXFP8](https://yomu.fyi/topic/mxfp8)

- Source: [Cursor](https://cursor.com/blog/mixture-of-kittens)
- Source URL: https://cursor.com/blog/mixture-of-kittens
- Ingested by Yomu: 2026-09-02T11:41:49.798Z

[Read original post](https://cursor.com/blog/mixture-of-kittens)