# New in llama.cpp: Model Management

[Hugging Face](https://yomu.fyi/company/hugging-face) · Xuan-Son Nguyen, Victor Mustar · Dec 11, 2025

**Type:** Announcement

## Summary

llama.cpp server now includes a router mode that enables dynamic loading, unloading, and switching between multiple LLMs without restarting the server. The architecture runs each model in an isolated process to ensure a single model crash does not affect other active instances. Running llama-server without specifying a model activates auto-discovery across the cache directory or a designated folder of GGUF files. In addition to on-demand loading and least-recently-used eviction capped by default at four models, the server provides endpoints for manual loading, unloading, and listing model statuses. Models inherit global configuration options or use dedicated configuration presets while also integrating directly into the built-in web UI.

## Context

Users requested Ollama-style dynamic model management in llama.cpp server to switch between models, conduct A/B testing, or support multi-tenant deployments without restarting the server.

## Approach / What changed

llama.cpp introduced router mode with a multi-process architecture where each model runs in an isolated process. It incorporates GGUF file auto-discovery, on-demand loading, request routing via the model field, LRU eviction, and configuration presets.

## Takeaways

- Router mode uses a multi-process architecture where each model operates in its own process, isolating crashes between instances.
- Models load on demand upon request and are evicted using an LRU strategy once the maximum capacity, configured by --models-max with a default of 4, is reached.
- Models inherit global server flags like context size and GPU offloading by default, but can be customized with per-model settings using INI preset files.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Open Source](https://yomu.fyi/topic/open-source)

[Read original post](https://huggingface.co/blog/ggml-org/model-management-in-llamacpp)