Up to 3.2x Faster Inference with LFM2.5-DSpark

Summary

Liquid AI released DSpark draft checkpoints for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B, adding speculative decoding with a minimal memory increase. DSpark uses a DFlash-style parallel backbone conditioned on target context features, a lightweight sequential Markov-chain head, and a confidence-scheduled verifier that prunes low-confidence suffixes. The draft models are simplified attention-only networks with five layers and a block of nine, trained for 15 epochs on SFT, chat, code, and function-calling data; each contains roughly 300M parameters. Across five datasets, LFM2.5-2.6B averaged 2.67x speedup on an H100 and 2.27x on an M4 Max, while reducing multi-tool function-calling latency by 57% on average. Greedy decoding remains identical to baseline by construction, and integrations are available in llama.cpp and SGLang, with on-device gains for the 8B-A1B model limited by its current MoE Metal implementation.

Takeaways

1. 1

   LFM2.5-2.6B averaged 2.67x speedup on an H100 and 2.27x on an M4 Max across MATH500, HumanEval, MBPP, GSM8K, and MT-Bench.

2. 2

   Greedy speculative decoding preserves baseline output exactly because rejected draft tokens are replaced by the target model's own token.

3. 3

   The LFM2.5-8B-A1B draft model averaged only an 18% on-device improvement because verifying multiple tokens activates more experts and increases weight traffic in the current Metal MoE implementation.