Experiment with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding

Summary

Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture, enabling developers to experiment with a multimodal mixture-of-experts model with a 125B-parameter main model, 51B N-gram embeddings, and 6B parameters activated per token. It supports a native 262,144-token context window, extensible to 1M tokens with YaRN. To reduce long-context attention compute and KV-cache growth, its hybrid design uses Gated DeltaNet in three of every four layers and Qwen Sparse Attention in the remaining layer, where micro-block importance guides retrieval. On NVIDIA GB300 NVL72, it delivers more than 16K tokens per second per GPU and more than 200 tokens per second per user; QSA achieved up to 7.6x prefill and 4.9x decoding speedups versus full attention. Fine-tuning is available through NVIDIA NeMo AutoModel with SFT or LoRA, and reinforcement learning through NVIDIA NeMo RL.

Takeaways

1. 1

   Three out of every four layers use Gated DeltaNet to maintain a fixed-size recurrent state, preventing KV-cache growth as sequences lengthen; Qwen Sparse Attention handles precise retrieval in the remaining layer.

2. 2

   At a 1M-token context, QSA's attention kernel delivered reported speedups of up to 7.6x during prefill and 4.9x during decoding versus full attention.

3. 3

   The model can be fine-tuned with full SFT or memory-efficient LoRA through NVIDIA NeMo AutoModel, then post-trained with NeMo RL; supported inference options include SGLang, vLLM, and TokenSpeed.