---
title: "llama.cpp"
description: "1 posts about llama.cpp, summarised, each linking to the original."
---

# llama.cpp
> 1 posts about llama.cpp, summarised, each linking to the original.

## Articles

### [Transformers now runs llama.cpp quants](https://yomu.fyi/post/transformers-now-runs-llama-cpp-quants.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Marc Sun, Arthur Zucker, Lysandre
- Published: Sep 22, 2026
- Type: Announcement

Transformers now supports running GGUF quantized models locally, reusing llama.cpp’s ggml kernels for efficient inference on Apple Silicon. The GGUF format packages weights, tokenizer, and chat template in one file, with quantization levels like Q4\_K\_M that trade precision for memory. Users can load a checkpoint from the Hub with from\_pretrained and generate text using the standard Transformers API, or serve it via an OpenAI-compatible endpoint. The initial focus is on the Qwen3.5 architecture, with benchmarks on a MacBook Pro M2 Max showing token-generation rates close to llama.cpp, including prefill time. The integration uses packed quantization kernels and generation-loop optimizations to reduce overhead, though it is currently limited to single conversations on MPS without batching support.