Hugging FaceAntonio Tiene, Ali Hashemi, David Jansen, Roman RauschProblem & solution
Removing entire transformer blocks is a cheap way to speed up large language models, yielding predictable inference gains and stacking with quantization and other compression methods. The challenge is that block choices interact, making it a combinatorial problem rather than a simple ranking task. The authors reformulate block selection as a constrained binary optimization (CBO) problem, mapping it onto an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of removed blocks. They note that mean-field methods, which treat each block independently, are inadequate for this interacting system. Block removal composes with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing, fitting into a larger pipeline rather than competing with it. The paper is titled ‘LLM Compression by Block Removal with Constrained Binary Optimization.’
The post introduces a method for compressing large language models by removing entire transformer blocks, framing the selection of which blocks to remove as a constrained binary optimization problem that maps onto an Ising glass spin system.
The approach reformulates block selection as a constrained binary optimization (CBO) problem, attaching a binary variable to each transformer block (0 for keep, 1 for remove), analogous to spins in an Ising glass with all-to-all interactions and a fixed number of removed blocks (conserved magnetization).
Block removal provides predictable inference speedups and memory savings, stacking with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing.
The selection of which blocks to remove is a combinatorial problem because the effect of removing one block depends on which others are removed, making it unsuitable for simple ranking approaches.
The problem is mapped to an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of ‘up’ spins, enabling the use of physics-based optimization techniques.
Because the effect of removing any one block depends on which other blocks are removed alongside it, making the choices interact. This interaction turns the problem into a combinatorial optimization with binary variables, not a simple ranking task.
The Ising glass formulation models block selection as a spin system with all-to-all interactions and a fixed number of removed blocks, representing conserved magnetization. This allows the use of physics-based optimization methods, though mean-field approaches that treat blocks independently are noted as inadequate.
Yes, block removal composes with quantization, low-rank/SVD compression, width pruning, and knowledge-distillation-based healing. It slots into a larger pipeline rather than competing with these other methods.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
A five-checkpoint workflow for assignment problems: write the spec, name the objective, test whether choices interact, pick a solver class by scale, then validate against live state.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
