# Making LLMs faster without sacrificing accuracy

[Amazon](https://yomu.fyi/company/amazon) · Tao Yu, Youngsuk Park · May 15, 2026

**Type:** Problem & solution

## Summary

Standard scaling laws optimize parameter count and training data budgets to minimize loss but ignore internal Transformer architecture configurations. To address throughput disparities among equally sized models, researchers developed a scaling framework incorporating hidden size, the multilayer perceptron to attention parameter ratio, and grouped-query attention. The method calibrates a correction surface over standard Chinchilla scaling laws to independently optimize separable architectural parameters for target accuracy. Evaluated across more than 200 models up to three billion parameters, the framework produced the Panda model family for maximum accuracy and the Surefire family for Pareto efficiency. Surefire models matched or exceeded LLaMA-3.2 accuracy while increasing inference throughput by 12% to 47% across vLLM and SGLang serving systems on modern GPUs.

## Context

Existing scaling laws like Chinchilla optimize parameter count and training token budgets for a target loss level but do not account for internal architectural parameters. Consequently, models with identical parameter counts and training data can exhibit inference throughput differences of up to 40%.

## Approach / What changed

Researchers augmented the Chinchilla scaling framework by modeling three architectural factors: hidden size, the ratio of MLP parameters to attention parameters, and grouped-query attention. The framework fits standard Chinchilla coefficients to determine a reference loss and calibrates a correction surface across architectural configurations, allowing separable parameters to be optimized independently to find Pareto-optimal designs.

## Takeaways

- Two models with identical parameter counts, training datasets, and accuracy can differ by up to 40% in inference throughput depending on internal architectural choices.
- Optimal MLP-to-attention parameter ratios for LLaMA-3.2-style architectures sit near 1.0, indicating that open-weight models with higher ratios such as 4.8 over-allocate parameters to MLP layers.
- Architectural scaling laws calibrated on small models between 80 million and 297 million parameters reliably predict optimal configurations at one-billion and three-billion parameter scales.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://www.amazon.science/blog/making-llms-faster-without-sacrificing-accuracy)