---
title: "Welcome Gemma 4: Frontier multimodal intelligence on device"
description: "Google DeepMind’s Gemma 4 family is available on Hugging Face as Apache 2-licensed, multimodal models intended for use across agents, inference engines, fine-tuning libraries, and on-device deployments. The family accepts image and text inputs across five sizes, while E2B, E4B, and 12B Unified also support audio; context windows range from 128k to 256k tokens. Its design combines alternating local and global attention, dual RoPE configurations, Per-Layer Embeddings, shared KV caches, and model-specific vision and audio handling, including an encoder-free unified 12B variant. The post documents integrations for Transformers, llama.cpp, MLX, WebGPU, Rust, and other tools, alongside fine-tuning examples and demos. Reported instruction-tuned results include a 1452 estimated text-only LMArena score for 31B and benchmark results spanning reasoning, coding, vision, audio, and long-context tasks."
---

# Welcome Gemma 4: Frontier multimodal intelligence on device

[Hugging Face](https://yomu.fyi/company/hugging-face) · merve, Pedro Cuenca, Sergio Paniego, ben burtenshaw, Steven Zheng, Alvaro Bartolome, Nathan Habib · Apr 2, 2026

**Type:** Announcement

## Summary

Google DeepMind’s Gemma 4 family is available on Hugging Face as Apache 2-licensed, multimodal models intended for use across agents, inference engines, fine-tuning libraries, and on-device deployments. The family accepts image and text inputs across five sizes, while E2B, E4B, and 12B Unified also support audio; context windows range from 128k to 256k tokens. Its design combines alternating local and global attention, dual RoPE configurations, Per-Layer Embeddings, shared KV caches, and model-specific vision and audio handling, including an encoder-free unified 12B variant. The post documents integrations for Transformers, llama.cpp, MLX, WebGPU, Rust, and other tools, alongside fine-tuning examples and demos. Reported instruction-tuned results include a 1452 estimated text-only LMArena score for 31B and benchmark results spanning reasoning, coding, vision, audio, and long-context tasks.

## Context

The post announces the availability of the Gemma 4 multimodal model family and describes efforts to make it usable across popular agents, inference engines, fine-tuning libraries, and deployment environments, including on-device hardware.

## Approach / What changed

Gemma 4 combines long-context decoder models with alternating sliding-window and global attention, dual RoPE configurations, Per-Layer Embeddings, and shared KV caches. Depending on the variant, it uses vision and audio encoders or an encoder-free unified architecture, and is integrated with Transformers, llama.cpp, MLX, WebGPU, Rust, and other tools.

## Takeaways

- Gemma 4 has five model sizes: E2B, E4B, 12B Unified, 31B, and 26B A4B MoE, with 128k or 256k context windows.
- Shared KV caches let selected final layers reuse key-value states from earlier layers, reducing inference memory and compute with minimal reported quality impact.
- The 12B Unified model projects raw image patches and audio waveforms directly into the LLM embedding space, avoiding separate vision and audio encoders and enabling one-pass fine-tuning.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

- Source: [Hugging Face](https://huggingface.co/blog/gemma4)
- Source URL: https://huggingface.co/blog/gemma4
- Ingested by Yomu: 2026-08-29T01:47:22.033Z

[Read original post](https://huggingface.co/blog/gemma4)