# Welcome EmbeddingGemma, Google's new efficient embedding model

[Hugging Face](https://yomu.fyi/company/hugging-face) · Tom Aarsen, Joshua, Alvaro Bartolome, Aritra Roy Gosthipaty, Pedro Cuenca, Sergio Paniego · Sep 4, 2025

**Type:** Announcement

## Summary

Google DeepMind released EmbeddingGemma, a multilingual embedding model with 308 million parameters and a 2048-token context window designed for on-device applications. Based on the Gemma3 transformer backbone, the architecture replaces causal attention with bidirectional attention to function as an encoder, followed by mean pooling and two dense layers producing 768-dimensional vectors. The model incorporates Matryoshka Representation Learning, allowing outputs to be truncated down to 512, 256, or 128 dimensions for reduced memory and storage footprints. Trained on approximately 320 billion multilingual tokens across more than 100 languages, the quantized model operates under 200 MB of RAM. In domain-specific evaluations on the MIRIAD dataset, fine-tuning increased NDCG@10 from 0.8340 to 0.8862, outperforming larger baselines.

## Context

Text embedding models enable semantic search, clustering, and retrieval across corpora, but on-device use cases and mobile retrieval-augmented generation pipelines require compact, low-latency multilingual models that operate within strict memory constraints.

## Approach / What changed

Google adapted the Gemma3 transformer backbone into a bidirectional encoder, adding mean pooling and two dense layers to produce 768-dimensional embeddings. The model was trained on roughly 320 billion tokens covering over 100 languages using Matryoshka Representation Learning, which enables dynamic truncation down to 512, 256, or 128 dimensions while requiring task-specific prompt prefixes during inference.

## Takeaways

- EmbeddingGemma contains 308 million parameters, supports a 2048-token context window across more than 100 languages, and consumes under 200 MB of RAM when quantized.
- The architecture modifies Gemma3 from a causal decoder to a bidirectional encoder, using mean pooling and dense layers to yield 768-dimensional embeddings truncatable to 512, 256, or 128 dimensions via Matryoshka Representation Learning.
- Fine-tuning EmbeddingGemma on the MIRIAD medical retrieval dataset boosted retrieval performance from 0.8340 to 0.8862 NDCG@10, exceeding the performance of multiple larger baseline models on the task.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python), [Search](https://yomu.fyi/topic/search)

[Read original post](https://huggingface.co/blog/embeddinggemma)