# How we improved translation experience with cost efficiency

[Grab](https://yomu.fyi/company/grab) · Jie Zhang · Aug 5, 2024

**Type:** Problem & solution

## Summary

Grab faced translation quality and cost challenges in its booking chat system when tourism resumed after COVID restrictions eased. Device language settings proved unreliable for determining message language, prompting the team to use Lingua alongside booking context heuristics and third-party fallbacks for real-time detection. To replace expensive and inaccurate general translation APIs, Grab distilled a large open-source Hugging Face model into lightweight, language-specific models trained on ten million synthetic chat examples. A post-translation validation step verifies non-translatable entities such as numbers and emojis before falling back to external services if mutations occur. Caching layers were also introduced across translation paths to curb redundant on-the-fly execution.

## Context

Post-COVID travel surges exposed missed and inaccurate translations in Grab's booking chat, driven by inaccurate device language assumptions, shared-vocabulary confusion between Malay and Indonesian, and high costs from third-party translation APIs.

## Approach / What changed

Grab implemented dynamic language detection with Lingua and context-aware heuristics, distilled an open-source LLM into 99% smaller in-house translation models using synthetic data, added programmatic post-translation integrity checks, and deployed caching.

## Takeaways

- Device language settings do not reliably reflect message language, necessitating real-time detection augmented by trip context to distinguish closely related languages like Malay and Indonesian.
- Fine-tuning language-specific models that were 99% smaller on 10 million synthetic examples generated by a large LLM achieved 98% of the larger model's translation effectiveness.
- Programmatic validation that counts non-translatable tokens like numbers, symbols, and emojis catches translation mutations and triggers safe third-party fallbacks.

**Tags:** [Caching](https://yomu.fyi/topic/caching), [Go](https://yomu.fyi/topic/go), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning)

[Read original post](https://engineering.grab.com/improved-translation-experience-with-cost-efficiency)