---
title: "Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original"
description: "Structural compression and 4-bit quantization can reduce an LLM’s reasoning, mathematical, and coding capabilities, while conventional healing methods either require costly quantization-aware training or distill from an already degraded checkpoint. Quantization-Aware Healing (QAH) instead distills the original, full-size, full-precision model into a structurally compressed student running in MXFP4, using KL divergence over teacher logits. A chunked KL loss allows healing on documents up to 32k tokens within a fixed GPU memory budget. Applied to GPT-OSS 120B compressed to 60B parameters, QAH’s 4-bit model outperformed its 60B bfloat16 counterpart on 7 of 9 benchmarks, including gains of 7.4 points on AA-LCR and 5.6 on AIME 2025. In a separate comparison, QAH reached its peak about seven times faster than QAT and avoided QAT’s later 19-point collapse."
---

# Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

[Hugging Face](https://yomu.fyi/company/hugging-face) · Antonio Tiene, Iker García-Ferrero, Ali Hashemi, Bakbergen Ryskulov · Aug 25, 2026

**Type:** Benchmark

## Summary

Structural compression and 4-bit quantization can reduce an LLM’s reasoning, mathematical, and coding capabilities, while conventional healing methods either require costly quantization-aware training or distill from an already degraded checkpoint. Quantization-Aware Healing (QAH) instead distills the original, full-size, full-precision model into a structurally compressed student running in MXFP4, using KL divergence over teacher logits. A chunked KL loss allows healing on documents up to 32k tokens within a fixed GPU memory budget. Applied to GPT-OSS 120B compressed to 60B parameters, QAH’s 4-bit model outperformed its 60B bfloat16 counterpart on 7 of 9 benchmarks, including gains of 7.4 points on AA-LCR and 5.6 on AIME 2025. In a separate comparison, QAH reached its peak about seven times faster than QAT and avoided QAT’s later 19-point collapse.

## Context

Structural compression followed by 4-bit quantization systematically degrades capabilities such as reasoning, mathematical problem-solving, and code generation. Existing healing approaches have limitations: QAT is costly and can become unstable after its best point, while QAD from a recovered bfloat16 checkpoint anchors the quantized student to a degraded target.

## Approach / What changed

Quantization-Aware Healing distills a frozen, original pre-compression full-precision teacher directly into a structurally compressed MXFP4 student through KL divergence on output logits. The teacher and student may have different architectures. For long-context healing, QAH uses a memory-efficient chunked KL loss that processes sequence slices without materializing the full vocabulary-by-sequence grid.

## Takeaways

- A GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4 beat its 60B bfloat16 counterpart on 7 of 9 benchmarks.
- QAH reached an average benchmark peak of 54.9 in about 100 steps, compared with 54.6 for QAT at about 700 steps; QAT later lost nearly 19 points by step 1,200.
- The QAH model uses roughly four times less weight memory than the bfloat16 student and surpassed the full-size teacher on LiveCodeBench, scoring 66.5 versus 66.0.

**Tags:** [Deployment](https://yomu.fyi/topic/deployment), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

- Source: [Hugging Face](https://huggingface.co/blog/MultiverseComputingCAI/quantization-aware-healing)
- Source URL: https://huggingface.co/blog/MultiverseComputingCAI/quantization-aware-healing
- Ingested by Yomu: 2026-08-29T05:42:01.306Z

[Read original post](https://huggingface.co/blog/MultiverseComputingCAI/quantization-aware-healing)