---
title: "LiteRT-LM"
description: "3 posts about LiteRT-LM, summarised, each linking to the original."
---

# LiteRT-LM
> 3 posts about LiteRT-LM, summarised, each linking to the original.

## Articles

### [Introducing Support for Local AI Models in the Antigravity SDK](https://yomu.fyi/post/introducing-support-for-local-ai-models-in-the-antigravity-sdk.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Sachin Kotwani, Taylor Mullen
- Published: Sep 23, 2026
- Type: Announcement

The Antigravity SDK now supports local AI models, starting with Gemma 4 26B A4B via Google AI Edge’s LiteRT, enabling offline agentic assistance on local GPU and RAM with a recommended machine of 24GB VRAM or unified memory. Local execution eliminates API costs and rate limits while keeping code and requests on-device, offering privacy and offline resiliency. The SDK supports hybrid workflows where a cloud model like Gemini 3.8 Flash plans while local Gemma 4 instances execute, as in a code auditing task where 97.2% of tokens (3,322 of 3,417) ran locally. A built-in example generates a live-updating CLI resource monitor using psutil and rich from a single prompt, with the agent autonomously writing the script, creating requirements.txt, and testing the code entirely offline. The SDK also supports any OpenAI-compatible server like Ollama, LM Studio, or vLLM via LocalOpenAIAgentConfig without changing agent orchestration.


### [Gemma 4 12B: The Developer Guide](https://yomu.fyi/post/gemma-4-12b-the-developer-guide.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: André Susano Pinto, Andreas Steiner, Karolis Misiunas, Karsten Roth, Michael Tschannen, Omar Sanseviero
- Published: Jun 3, 2026
- Type: Announcement

Gemma 4 12B is a dense multimodal model with a unified, encoder-free architecture, native audio input, and a size intended for local execution on dedicated GPU laptops with 16GB VRAM or unified memory. Instead of separate vision and audio encoders, it uses a 35M-parameter vision embedder that projects raw 48x48 patches and a linear projection for raw 16 kHz audio sliced into 40ms frames; both feed a decoder-only transformer with shared weights. That design reduces multimodal latency and fragmented memory footprints, while allowing downstream adapters such as LoRA or full tuning to update the multimodal token loop in one pass. Capabilities include automatic speech recognition, agentic reasoning, diarization, video understanding, and coding, with examples covering a locally built image-processing app and five minutes of video processed at 1 FPS with audio; LiteRT-LM adds offline macOS apps, sandboxed Python execution, and an OpenAI-compatible local API server with stateless prefix caching.


### [Blazing fast on-device GenAI with LiteRT-LM](https://yomu.fyi/post/blazing-fast-on-device-genai-with-litert-lm.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Tenghui Zhu, Yu-hui Chen, Ram Iyengar
- Published: May 19, 2026
- Type: Announcement

LiteRT-LM is presented as an on-device runtime for deploying Gemma 4 across Android, iOS, and the web despite constrained memory, limited compute, and varied hardware. It combines LiteRT inference, quantization, XNNPACK and MLDrift kernels, CPU/GPU/NPU backends, transfer-avoiding pipelines, Multi-Token Prediction (MTP), and session management. Reported decode rates reach 52 tokens/sec on Android GPU, 56 on iOS Metal, and up to 76 on WebGPU; MTP provides up to a 2.2x speedup, while a ~2.58GB E2B model uses 607MB of physical memory on Apple mobile CPUs. The runtime also supports save/restore of KV caches, native vision and audio loading, Thinking Mode, constrained decoding, and function calling. New Swift and JavaScript APIs extend the integration surface, including WASM/WebGPU browser inference designed for serverless, secure, privacy-preserving applications.