# How We Optimized the Qwen 3.6 Model for Our Junie Agent

[Jetbrains](https://yomu.fyi/company/jetbrains) · Stanislav Erokhin · Aug 24, 2026

**Type:** Problem & solution

## Summary

Deploying the Junie coding agent locally on Apple MacBook M5 hardware requires overcoming steep performance bottlenecks during model execution. Engineers optimized the entire stack using Qwen3.6-27B rather than Qwen3.8-27B, which severely degrades without token-heavy reasoning enabled. At the agent layer, Junie appends requests directly into a rolling context and caches initial prompt prefixes to maximize KV-cache reuse across tasks. For model inference, the team disabled reasoning, selected 4-bit quantization, and patched MLX-VLM to execute self-attention prefill matrix operations using faster 8-bit instructions on the M5 processor. Combining these prefill improvements with simultaneous multi-token prediction and n-gram speculative decoding delivered up to a 2x generation speedup.

## Context

Enabling local inference for the Junie coding agent on MacBook M5 hardware was constrained by slow prefill speeds, high token generation counts, and memory bottlenecks compared to discrete cloud GPUs.

## Approach / What changed

Engineers integrated Qwen3.6-27B with 4-bit quantization, disabled reasoning, extended the agent's rolling context to preserve KV caches, and cached initial system prompt prefixes. They also patched MLX-VLM to use 8-bit matrix arithmetic for self-attention layers during prefill on the M5 and enabled both Multi-Token Prediction and n-gram speculative decoding.

## Takeaways

- Patching MLX-VLM to run self-attention matrix operations in 8-bit precision on the M5 chip increased prefill speed by approximately 40%.
- Disabling reasoning mode in Qwen3.6-27B reduced token generation by 2x to 3x, yielding a 2x speedup on task execution with negligible impact on quality.
- Combining Multi-Token Prediction with a draft model alongside n-gram speculative decoding yielded up to a 2x speedup in generation.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Caching](https://yomu.fyi/topic/caching), [Developer Experience](https://yomu.fyi/topic/developer-experience), [LLMs](https://yomu.fyi/topic/llm), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://blog.jetbrains.com/junie/2026/08/qwen-for-junie)