Cursor open-sourced Mixture-of-Kittens, a deterministic MoE training megakernel, under Apache 2.0

Mixture-of-Kittens (MoK) is a Mixture-of-Experts training megakernel that Cursor released as open source under Apache 2.0 on August 4, 2026. The kernel fuses MoE communication and computation into a single kernel, eliminating kernel launch boundaries between minibatches and removing CPU-GPU synchronization, and Cursor states it uses MoK to train Composer, its agentic coding model. In the published benchmarks MXFP8 forward throughput is up to 2.37x the fastest public baseline, and end-to-end training on 512 GPUs is 1.41x faster, moving from 760.9 to 1,070.2 tokens per second per GPU. ASAP works from Cursor's engineering blog and the GitHub repository to lay out what the kernel changes and how far the numbers carry.

One kernel swallowed the entire MoE layer

MoK is a training megakernel that fuses the communication and computation of an MoE layer into a single kernel, released by Cursor on August 4, 2026. The repository describes the project as a fully deterministic mixture-of-experts training megakernel built from first principles for NVL72s.

In conventional MoE training a layer breaks into dispatch, expert computation, and combine, and each stage runs as its own kernel. Every stage boundary is a point where the GPU waits for communication to finish. MoK removes those boundaries so no kernel launch sits between minibatches, and it runs scheduling entirely on the device so no CPU-GPU synchronization is required. Cursor reports the schedule kernel itself accounts for less than 3% of total MoE runtime.

The overlap strategy is explicit as well. MoK partitions streaming multiprocessors into computation SMs and communication SMs that run concurrently, and tunes minibatch size to keep the overlap window full. Cursor publishes a formula for the optimal minibatch size on Blackwell alongside a concrete setting of roughly 2,368 tokens.

Which baseline the 2.37x is measured against

The 2.37x figure is MXFP8 forward throughput measured against HybridEP + Megatron, which Cursor names as the fastest public baseline. In the same comparison MXFP8 backward is 1.78x, BF16 forward is 1.92x, and BF16 backward is 1.58x, at 2,048 tokens per GPU and an expert-parallel degree of 64.

Naming the baseline precisely matters because the number travels under other labels. Several outlets covering the release in August 2026 reported 2.37x as a margin over DeepSeek's DeepEP. DeepEP appears in Cursor's blog as the baseline for the end-to-end training comparison, not the microbenchmark, where the comparison is against HybridEP + Megatron. The two comparisons differ in both target and multiple.

The benchmarked model shapes follow real public models. Cursor measured four configurations: Kimi K2.7 Code (384 experts, hidden dimension 7,168), GLM-5.2 (256 experts, hidden dimension 6,144), Qwen3.5-397B-A17B (512 experts, top-k 10), and DeepSeek-V4-Pro (384 experts, intermediate dimension 3,072).

Where pulling beat pushing

Pull-based forward dispatch is what cuts signaling latency from 103 microseconds to 18 microseconds, a 5.8x reduction. The same change raises NVLink bandwidth utilization by 29% relative to push-based dispatch.

The direction is assigned per stage rather than globally. Forward dispatch pulls and forward combine pushes; in the backward pass reverse-combine pulls and reverse-dispatch pushes. Neither direction wins everywhere, and the deciding factor is whether the receiving side knows when it needs the data. Where the receiver knows its own timing, pulling removes the readiness signal entirely; where the sender finalizes results first, pushing avoids a round trip.

The ring buffer follows the same logic. MoK keeps a fixed-size macrobatch ring buffer of a few hundred megabytes and interleaves dispatch with combine, and the backward pass traverses the ring in reverse to minimize replay of forward activations.

Treating determinism as a first-class property

Determinism is a design goal MoK states from the first line of its repository description, and the implementation fixes the order of floating point operations so that the same input produces bitwise-identical output regardless of hardware scheduling, as of the August 2026 release.

In distributed training determinism is the property most easily traded for speed. Accumulating in arrival order shortens waits, but it also changes the order of floating point additions between runs, so results drift slightly. That drift is harmless most of the time and bills you only when something breaks, because it removes any way to tell whether a loss spike came from data, a code change, or numerical noise.

At tens of thousands of GPUs the bill grows. The more a single training run costs, the more expensive it becomes to let an unreproducible anomaly slide. Putting determinism in the repository's first sentence rather than in a footnote reflects a judgment that debuggability is an asset of the same rank as throughput.

Reading the gap between 2.37x and 1.41x

The 2.37x microbenchmark gain narrows to 1.41x in real training on 512 GPUs. Cursor's end-to-end figure moves a DeepEP-based production stack from 760.9 tokens per second per GPU to 1,070.2, measured across several GB300 NVL72 racks.

The gap between those numbers reflects structure rather than overstatement. A microbenchmark isolates a single MoE layer, while a real training step also contains attention, optimizer updates, the data pipeline, and checkpointing, none of which this kernel touches. Since MoE occupies a fixed share of step time, any speedup on that share converges to the share itself.

The number that matters operationally is therefore 1.41x. Inverted, it means the same token budget takes about 71% of the previous wall-clock time, which is a capital question at large training scale rather than a benchmarking one. It is also a figure from Cursor's specific production stack and model shapes, and a different MoE share yields a different multiple. Applying the microbenchmark multiple to a different pipeline does not hold.

What it takes to actually run this code

Running MoK requires NVIDIA Blackwell SM100 or SM103 GPUs, Python 3.12+, PyTorch 2.10+, and CUDA toolkit 13.0+. The repository names GB200 NVL72 and GB300 NVL72 as the target hardware and specifies that the installed PyTorch build must target CUDA 13.0+ with a major and minor version matching the system CUDA toolkit.

Those requirements show that an open license and practical access are separate questions. Apache 2.0 grants broad rights to use, modify, and redistribute, but the kernel targets one physical configuration: a 72-GPU rack tied together by NVLink. Few teams operate that class of hardware directly, and renting it still means securing rack-level topology to approach the published numbers.

The value of the release does not rest on benchmark reproduction, though. The overlap design, the per-stage assignment of pull and push, and the accumulation ordering that preserves determinism are all readable regardless of what hardware a team owns. The immediate return for most teams is not throughput but a concrete map of where large-scale MoE training bottlenecks form.

Open questions

Every performance figure Cursor published was measured on NVL72 rack configurations, and no results are given for smaller clusters. How the communication pattern behaves below a full rack cannot be read from this release.

The second question is training quality. The published comparison concentrates on throughput and determinism, and the post does not report how the final quality of an MXFP8-trained model compares with BF16 training.

The third is maintenance. Megakernel designs bind tightly to a hardware generation. This implementation targets Blackwell SM100 and SM103, and how much of the design carries to the next architecture depends on the cadence at which Cursor updates the repository.

Source: Cursor's engineering blog post "Mixture-of-Kittens: our open-source MoE megakernel for NVL72s" (August 4, 2026) and the cursor/mixture-of-kittens GitHub repository, compiled by ASAP.