---
title: "Understanding W8A8 INT8 LLM quantization: Half the size, better performance, same accuracy"
description: "Serving Llama 3.1 8B in BF16 requires roughly 14.9 GB for weights, leaving little VRAM on a 16 GB GPU for KV cache and activations, which limits practical deployment. The workflow benchmarks the base model, applies INT8 W8A8 quantization with llm-compressor, then repeats accuracy and performance tests under identical hardware and load conditions. SmoothQuant modifies weights to reduce activation outliers, while GPTQ uses calibration data and error compensation before rounding weights to INT8; the experiment used 512 WikiText-2 samples and targeted deployment with vLLM. On the Llama 3.1 8B Instruct model, compression reduced storage from 14.9 GB to 8.0 GB, a 46% reduction, and accuracy was preserved across MMLU, ARC, HellaSwag, and IFeval; generation-speed results are reserved for part 2."
---

# Understanding W8A8 INT8 LLM quantization: Half the size, better performance, same accuracy

[Red Hat](https://yomu.fyi/company/red-hat) · Sana Fayyaz · Sep 7, 2026

**Type:** Tutorial

## Summary

Serving Llama 3.1 8B in BF16 requires roughly 14.9 GB for weights, leaving little VRAM on a 16 GB GPU for KV cache and activations, which limits practical deployment. The workflow benchmarks the base model, applies INT8 W8A8 quantization with llm-compressor, then repeats accuracy and performance tests under identical hardware and load conditions. SmoothQuant modifies weights to reduce activation outliers, while GPTQ uses calibration data and error compensation before rounding weights to INT8; the experiment used 512 WikiText-2 samples and targeted deployment with vLLM. On the Llama 3.1 8B Instruct model, compression reduced storage from 14.9 GB to 8.0 GB, a 46% reduction, and accuracy was preserved across MMLU, ARC, HellaSwag, and IFeval; generation-speed results are reserved for part 2.

## Context

Llama 3.1 8B in BF16 occupies roughly 15 GB of GPU memory for weights, leaving limited space for the KV cache and activations on a 16 GB GPU. These memory demands constrain request concurrency, response speed, and practical deployment cost, motivating lower-precision representations that do not degrade model capabilities.

## Approach / What changed

The workflow benchmarks the BF16 model, applies INT8 W8A8 quantization with llm-compressor, and benchmarks the compressed model under the same conditions. SmoothQuant modifies weights to reduce activation outliers, while GPTQ uses calibration data and error compensation to limit rounding damage. The experiment uses 512 WikiText-2 samples and prepares the model for vLLM deployment.

## Takeaways

- W8A8 represents both weights and activations as INT8 during matrix multiplication, allowing use of INT8 tensor cores instead of the BF16 tensor cores used by the uncompressed model.
- SmoothQuant and GPTQ are applied sequentially: SmoothQuant addresses activation outliers, while GPTQ calibrates weights and compensates for quantization error before rounding.
- The model shrank from 14.9 GB to 8.0 GB. The compressed size exceeds the theoretical 7.5 GB INT8 weight size because some components, including lm\_head, remain in BF16 and quantization scales are stored alongside the weights.

**Tags:** [GPU](https://yomu.fyi/topic/gpu), [Quantization](https://yomu.fyi/topic/quantization), [vLLM](https://yomu.fyi/topic/vllm)

- Source: [Red Hat](https://developers.redhat.com/articles/2026/09/07/understanding-w8a8-int8-llm-quantization)
- Source URL: https://developers.redhat.com/articles/2026/09/07/understanding-w8a8-int8-llm-quantization
- Ingested by Yomu: 2026-09-07T08:00:55.360Z

[Read original post](https://developers.redhat.com/articles/2026/09/07/understanding-w8a8-int8-llm-quantization)