---
title: "Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72"
description: "Alibaba’s Qwen3.8-2.4T-A95B (Qwen3.8-Max) is an open-weight 2.4T-parameter model with 95B activated per token, designed for reasoning and agentic workloads. Its fine-grained mixture-of-experts architecture combines full and linear attention, supporting context of up to one million tokens and output lengths up to 128K while using bounded recurrent state in linear-attention layers. NVIDIA describes multinode serving on the 72-GPU GB300 NVL72, where a 130 TB/s NVLink domain supports expert communication; in FP8, the model reportedly exceeds 4K tokens per second per GPU and 350 tokens per second per user without additional tuning. Developers can control reasoning depth with low, high, or xhigh modes, choose SGLang, vLLM, NVIDIA Dynamo, or NIM for serving, and use NeMo AutoModel for full SFT or LoRA post-training."
---

# Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Michelle Horton · Aug 12, 2026

**Type:** Explainer

## Summary

Alibaba’s Qwen3.8-2.4T-A95B (Qwen3.8-Max) is an open-weight 2.4T-parameter model with 95B activated per token, designed for reasoning and agentic workloads. Its fine-grained mixture-of-experts architecture combines full and linear attention, supporting context of up to one million tokens and output lengths up to 128K while using bounded recurrent state in linear-attention layers. NVIDIA describes multinode serving on the 72-GPU GB300 NVL72, where a 130 TB/s NVLink domain supports expert communication; in FP8, the model reportedly exceeds 4K tokens per second per GPU and 350 tokens per second per user without additional tuning. Developers can control reasoning depth with low, high, or xhigh modes, choose SGLang, vLLM, NVIDIA Dynamo, or NIM for serving, and use NeMo AutoModel for full SFT or LoRA post-training.

## Context

Serving a 2.4T-parameter open-weight model requires data-center-scale accelerated compute and co-design across chips, system architecture, and software. Agentic workloads also accumulate instructions, tool outputs, documents, code, logs, and reasoning traces, making attention, compute, and KV-cache memory constraints as context grows.

## Approach / What changed

NVIDIA supports multinode deployment on GB300 NVL72 with optimized kernels, inference runtimes, and distributed serving recipes. The model uses hybrid full and linear attention, fine-grained MoE routing, and configurable reasoning modes. Developers can serve it with SGLang, vLLM, NVIDIA Dynamo, or model-free NIM, and post-train checkpoints with NeMo AutoModel using full SFT or LoRA.

## Takeaways

- Qwen3.8-2.4T-A95B activates 95B parameters per token from its 2.4T total, using a learned router to select experts and reduce serving cost relative to a comparable dense model.
- The hybrid attention design alternates full attention with linear-attention layers that replace the growing KV cache with a bounded recurrent state for long contexts up to one million tokens.
- On GB300 NVL72, FP8 serving without additional model tuning exceeds 4K tokens per second per GPU and 350 tokens per second per user; NVFP4 optimizations are expected to improve performance further.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Deployment](https://yomu.fyi/topic/deployment), [LLMs](https://yomu.fyi/topic/llm), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

- Source: [NVIDIA](https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72)
- Source URL: https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72
- Ingested by Yomu: 2026-08-28T12:01:02.016Z

[Read original post](https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72)