---
title: "Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem"
description: "Removing entire transformer blocks is a cheap way to speed up large language models, yielding predictable inference gains and stacking with quantization and other compression methods. The challenge is that block choices interact, making it a combinatorial problem rather than a simple ranking task. The authors reformulate block selection as a constrained binary optimization (CBO) problem, mapping it onto an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of removed blocks. They note that mean-field methods, which treat each block independently, are inadequate for this interacting system. Block removal composes with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing, fitting into a larger pipeline rather than competing with it. The paper is titled ‘LLM Compression by Block Removal with Constrained Binary Optimization.’"
---

# Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

[Hugging Face](https://yomu.fyi/company/hugging-face) · Antonio Tiene, Ali Hashemi, David Jansen, Roman Rausch · Sep 21, 2026

**Type:** Problem & solution

## Summary

Removing entire transformer blocks is a cheap way to speed up large language models, yielding predictable inference gains and stacking with quantization and other compression methods. The challenge is that block choices interact, making it a combinatorial problem rather than a simple ranking task. The authors reformulate block selection as a constrained binary optimization (CBO) problem, mapping it onto an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of removed blocks. They note that mean-field methods, which treat each block independently, are inadequate for this interacting system. Block removal composes with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing, fitting into a larger pipeline rather than competing with it. The paper is titled ‘LLM Compression by Block Removal with Constrained Binary Optimization.’

## Context

The post introduces a method for compressing large language models by removing entire transformer blocks, framing the selection of which blocks to remove as a constrained binary optimization problem that maps onto an Ising glass spin system.

## Approach / What changed

The approach reformulates block selection as a constrained binary optimization (CBO) problem, attaching a binary variable to each transformer block (0 for keep, 1 for remove), analogous to spins in an Ising glass with all-to-all interactions and a fixed number of removed blocks (conserved magnetization).

## Takeaways

- Block removal provides predictable inference speedups and memory savings, stacking with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing.
- The selection of which blocks to remove is a combinatorial problem because the effect of removing one block depends on which others are removed, making it unsuitable for simple ranking approaches.
- The problem is mapped to an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of ‘up’ spins, enabling the use of physics-based optimization techniques.

## FAQ

### Why is block selection a combinatorial problem rather than a ranking problem?

Because the effect of removing any one block depends on which other blocks are removed alongside it, making the choices interact. This interaction turns the problem into a combinatorial optimization with binary variables, not a simple ranking task.

### How does the Ising glass formulation help with block removal?

The Ising glass formulation models block selection as a spin system with all-to-all interactions and a fixed number of removed blocks, representing conserved magnetization. This allows the use of physics-based optimization methods, though mean-field approaches that treat blocks independently are noted as inadequate.

### Can block removal be combined with other compression techniques?

Yes, block removal composes with quantization, low-rank/SVD compression, width pruning, and knowledge-distillation-based healing. It slots into a larger pipeline rather than competing with these other methods.

**Tags:** [Optimization](https://yomu.fyi/topic/optimization), [Transformer](https://yomu.fyi/topic/transformer)

- Source: [Hugging Face](https://huggingface.co/blog/MultiverseComputingCAI/pruning-llms-like-a-physicist-block-removal-as-an)
- Source URL: https://huggingface.co/blog/MultiverseComputingCAI/pruning-llms-like-a-physicist-block-removal-as-an
- Ingested by Yomu: 2026-09-21T15:00:10.076Z

[Read original post](https://huggingface.co/blog/MultiverseComputingCAI/pruning-llms-like-a-physicist-block-removal-as-an)