Google Cloud C4 Brings a 70% TCO improvement on GPT OSS with Intel and Hugging Face

Summary

Intel and Hugging Face benchmarked OpenAI GPT OSS text generation to evaluate performance on Google Cloud C4 virtual machines powered by Intel Xeon 6 processors against previous-generation C3 instances. The benchmark evaluated the unsloth/gpt-oss-120b-BF16 Mixture of Experts model across batch sizes from 1 to 64 using bfloat16 precision, static KV cache, and SDPA attention. To eliminate redundant computation, Hugging Face merged an optimization directing each expert sub-network to process only its routed tokens rather than all tokens. Testing demonstrated that C4 instances deliver 1.4x to 1.7x higher throughput per vCPU compared to C3 instances. At a batch size of 64, C4 achieved a 1.7x Total Cost of Ownership improvement alongside lower hourly pricing.

Context

Evaluating text generation performance and Total Cost of Ownership improvements for OpenAI GPT OSS Mixture of Experts models when transitioning from Google Cloud C3 instances to C4 instances.

Approach / What changed

Intel and Hugging Face merged an expert execution optimization (PR #40304) into transformers so MoE experts process only routed tokens. They benchmarked unsloth/gpt-oss-120b-BF16 on GCP C4 VMs (Intel Xeon 6) versus C3 VMs (4th Gen Intel Xeon) across batch sizes 1 to 64 with static KV cache and SDPA attention.

Takeaways

• Hugging Face merged an MoE optimization (PR #40304) that routes only designated tokens to each expert, eliminating redundant compute and reducing wasted FLOPs.
• Google Cloud C4 instances with Intel Xeon 6 processors achieved 1.4x to 1.7x higher normalized throughput per vCPU over C3 instances across batch sizes up to 64.
• At batch size 64, C4 delivered a 1.7x Total Cost of Ownership advantage over C3 due to combined throughput gains and near price parity per vCPU.

Related reading

• Hugging Face · Sep 11, 2025

  Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers

  To support OpenAI's GPT-OSS series of models, the transformers library introduced several performance upgrades that apply across supported architectures. The release integrates zero-build custom kernels downloadable directly from the Hub, reducing external dependency bloat and compilation friction for operations like Liger RMSNorm and MegaBlocks MoE. Native support for MXFP4 quantization groups vector elements into 32-value blocks with shared scales, allowing GPT-OSS 20B to fit in roughly 16 GB of VRAM and GPT-OSS 120B in roughly 80 GB. In addition, transformers incorporates Flash Attention 3 with attention sinks, continuous batching via the generate_batch API for experimentation, and automatic memory pre-allocation to speed up model loading on GPUs.

  Aritra Roy Gosthipaty, Sergio Paniego, Vaibhav Srivastav, Pedro Cuenca, Arthur Zucker, Nathan Habib, Cyril Vallez

• Hugging Face · Jan 27, 2026

  Unlocking Agentic RL Training for GPT-OSS: A Practical Retrospective