Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality

Summary

Granite Embedding Multilingual R2 introduces two Apache 2.0 multilingual embedding models built on ModernBERT: a 97M-parameter compact model and a 311M full-size model. Both cover more than 200 languages, receive enhanced retrieval training for 52 languages and nine programming languages, and support context lengths up to 32,768 tokens, compared with 512 for the R1 models. The compact model scores 60.3 on MTEB Multilingual Retrieval, while the full-size model scores 65.2; the reported results place them among the strongest open models in their parameter ranges. R2 replaces the XLM-RoBERTa architecture with alternating attention lengths, rotary position embeddings, newer multilingual and code tokenizers, and Flash Attention 2.0 support, while the 311M model adds Matryoshka embeddings and both models provide ONNX and OpenVINO weights.

Takeaways

1. 1

   The 97M model scores 60.3 on MTEB Multilingual Retrieval across 18 languages, compared with 50.9 for multilingual-e5-small, according to the post.

2. 2

   The 311M model produces 768-dimensional embeddings and supports Matryoshka truncation, allowing a storage and speed trade-off; the 97M model produces 384-dimensional embeddings.

3. 3

   Both models support more than 200 languages, enhanced retrieval quality for 52 languages, 32,768-token contexts, and cross-lingual code retrieval across Python, Go, Java, JavaScript, PHP, Ruby, SQL, C, and C++.