# User foundation models for Grab

[Grab](https://yomu.fyi/company/grab) · Abhinav Rai · Sep 26, 2025

## Summary

Traditional recommendation systems at Grab relied on siloed, manually engineered features that struggled to capture sequential interaction data and cross-service user intent across its superapp ecosystem. To resolve these limitations, Grab developed a custom user foundation model built on a transformer architecture designed to learn from both tabular profile data and time-series clickstream interactions. The architecture unifies diverse data modalities—including categorical identifiers, numerical values, text, and geographical coordinates—by framing inputs as key-value token pairs. Custom positional embeddings and attention masks ensure the model processes tabular attributes as unordered sets while maintaining the sequential order of time-series events. The pre-trained model generates holistic user embeddings and supports direct fine-tuning across downstream tasks such as fraud detection, churn prediction, and advertisement optimization.

## Takeaways

- Off-the-shelf LLMs lack domain-specific context for superapp data structures like geohashes, requiring custom architectures capable of handling heterogeneous modalities including geographic coordinates, transaction amounts, and entity IDs.
- Grab standardizes input representations into universal key-value token pairs, pairing column names with attributes for tabular data and event types with entities for sequential clickstream data.
- The transformer architecture uses custom positional embeddings and attention masks to process tabular key-value tokens as an unordered set while preserving the strict chronological sequence of time-series events.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems)

[Read original post](https://engineering.grab.com/user-foundation-models-for-grab)