Reachy Mini goes fully local

Summary

Reachy Mini’s conversation app can now run its speech stack locally instead of sending audio to a server, avoiding cloud, API keys, and data leaving the machine. The tutorial uses speech-to-speech as a cascaded VAD → STT → LLM → TTS pipeline exposed through a Realtime API-compatible /v1/realtime WebSocket, with Silero VAD v5, Parakeet-TDT 0.6B v3, Qwen3-TTS, and a selectable LLM backend. It shows llama.cpp serving Gemma 4 E4B with parallel slots, 64k context, flash attention, and full sliding-window attention, then covers vLLM, MLX, Transformers, hosted Responses API endpoints, and network connection to a wireless robot. Once the backend and app are connected, conversations run through the local loop, while individual stages can be exchanged to trade latency, quality, language coverage, and hardware choices.

Takeaways

1. 1

   The local pipeline keeps audio on hardware controlled by the user and avoids per-minute or per-token API costs.

2. 2

   The speech-to-speech server exposes /v1/realtime over WebSocket, allowing Reachy Mini to use the same protocol with local or external Responses API-compatible LLM servers.

3. 3

   For vLLM, automatic tool choice, a model-specific tool-call parser, and disabled thinking are required or recommended for reliable, natural-feeling conversations; speculative decoding is optional and can reduce latency.