Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

Summary

Granite Switch 4.1 models combine chat, general and specialized decisions, and speech-to-text from a single hosted endpoint. The 8B and 30B models include a new Decision aLoRA for general-purpose decision-making and a Guardian aLoRA that outperforms Jev 1.13 and other closed models on safety tasks. They also embed Granite-Speech-5.0-Turbo-CTC, the fastest ASR model on the OpenASR Leaderboard, enabling efficient transcription. All components are composed into one model using cache-efficient Activated LoRAs, taking up about the same GPU VRAM as the base model alone, and the KV cache from base generation can be reused for any subsequent adapter decisions. Granite Switch 4.1 can switch between chat, decision, and spoken requests quickly, and a 3B variant achieved 60-100 decisions per second playing Doom, compared to Jev’s 10, while also narrating and answering questions.

Takeaways

1. 1

   Granite Switch 4.1 combines chat, decision-making, and speech-to-text in one model using cache-efficient activated LoRAs.

2. 2

   The Guardian aLoRA outperforms Jev 1.13, Microsoft-Decision-1, and GPT-Luna-Decisions on safety-related decision tasks.

3. 3

   Specialized adapters from Granite Libraries can be added alongside generalist decision adapters with negligible cost due to KV cache sharing.

FAQ