Junie Can Now Run Entirely on Your Mac – No Credits, No Cloud

Summary

JetBrains has introduced Junie Local, a feature that runs the Junie coding agent entirely on local hardware without cloud dependencies, subscriptions, or credit meters. Users execute a single command inside Junie to download a 4-bit quantized Qwen3.6-27B model, requiring an Apple M5 Mac with 64 GB of RAM. The team prioritized prefill throughput over pure generation speed by utilizing 8-bit arithmetic instructions on the M5 Neural Accelerator, which increased prefill throughput by approximately 40%. Performance optimizations also include KV-cache reuse, speculative decoding to double generation speed, and disabling reasoning to preserve performance. In JetBrains' evaluations, the local configuration performed comparably to cloud-based Sonnet 4.5 on everyday tasks while ensuring source code, diffs, and prompts never leave the local environment.

Context

Connecting coding agents to local models previously required manual setup, runtime installation, profile configuration, and model tuning, while small models remained constrained to simpler tasks. Developers were also constrained by token consumption, cloud costs, and vendor privacy concerns.

Approach / What changed

JetBrains launched Junie Local using a tuned 4-bit Qwen3.6-27B model on Apple M5 hardware with 64 GB of RAM. The implementation leverages M5 Neural Accelerator 8-bit arithmetic instructions for prefill speed, disables model reasoning, applies KV-cache reuse, and uses speculative decoding.

Takeaways

• Junie Local runs a 4-bit Qwen3.6-27B model locally on an Apple M5 Mac with 64 GB of RAM via a single /local command.
• Optimizing prefill throughput using the Apple M5 Neural Accelerator's 8-bit arithmetic instructions yielded a 40% performance gain.
• JetBrains selected Qwen3.6 over Qwen3.8 because 3.8 required reasoning to operate reliably, which made tasks run roughly four times slower.

Related reading

• Jetbrains · Aug 24, 2026

  How We Optimized the Qwen 3.6 Model for Our Junie Agent

  Deploying the Junie coding agent locally on Apple MacBook M5 hardware requires overcoming steep performance bottlenecks during model execution. Engineers optimized the entire stack using Qwen3.6-27B rather than Qwen3.8-27B, which severely degrades without token-heavy reasoning enabled. At the agent layer, Junie appends requests directly into a rolling context and caches initial prompt prefixes to maximize KV-cache reuse across tasks. For model inference, the team disabled reasoning, selected 4-bit quantization, and patched MLX-VLM to execute self-attention prefill matrix operations using faster 8-bit instructions on the M5 processor. Combining these prefill improvements with simultaneous multi-token prediction and n-gram speculative decoding delivered up to a 2x generation speedup.

  Stanislav Erokhin

• Grab · Nov 4, 2025

  How we built a custom vision LLM to improve document processing at Grab

  Document processing for identity verification across Southeast Asia presents challenges due to varied layouts and non-Latin scripts. Traditional OCR and off-the-shelf vision models struggle with accuracy, high latency, or lack of regional language training data. Grab addressed this by creating synthetic regional datasets, using an automated labeling pipeline named Documint, and evaluating open-source multimodal architectures. After initial LoRA fine-tuning failed on complex scripts like Thai and Vietnamese, full-parameter fine-tuning of Qwen2-VL 2B yielded substantial gains. To optimize deployment costs and latency, the team constructed a custom 1B parameter model pairing a Qwen2-VL vision encoder with a Qwen2.5 0.5B language decoder, achieving performance within 3 percentage points of the 2B model at significantly lower latency.

  Jia Chen

• Meta · Aug 3, 2026

  GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model

  Meta's Generative Ads Recommendation Model (GEM) trains across thousands of GPUs using a hybrid architecture featuring trillions of sparse embedding parameters and billions of dense parameters. Standard training infrastructure built for large language models struggles with recommendation-specific workloads characterized by jagged user activity histories, asymmetric attention patterns, and numerical sensitivity. To overcome these bottlenecks, engineering teams introduced custom kernels like Jagged Flash Attention and Generalized Dot-Product Attention alongside MXFP8 mixed-precision execution. Pairing these kernel optimizations with topology-aware 5D parallelism doubled end-to-end training efficiency to 20–25% Model FLOPs Utilization while scaling total training FLOPs fourfold over twelve months.

• Grab · Jul 24, 2026

  Agent platform (Part 1): How we help Grab build and run AI agents at scale

  Grab scaled its internal AI tooling by transforming the infrastructure behind a single technical support bot into a standardized agent framework named LLM-Kit. The original Go-based bot revealed critical bottlenecks, including the lack of automated evaluation, painful provider migrations, fragmented observability, and excessive production scaffolding overhead. To address these frictions, engineers developed a FastAPI template pre-wired with LangGraph ReAct agents, remote Model Context Protocol servers, and built-in ROUGE, BLEU, and LLM-as-judge evaluation suites. The framework centralizes model access through an OpenAI-compatible GrabGPT Gateway and unifies observability across services, outbound calls, and agent steps using OpenTelemetry. This extracted scaffolding reduced day-one production setup time from two weeks to roughly an hour while supporting hundreds of production agents.

  Kendrick Tan