# Making User-Sequence Data More Cost-Efficient, Faster, and Easier to Use

[Pinterest](https://yomu.fyi/company/pinterest) · Pinterest Engineering · May 21, 2026

**Type:** Problem & solution

## Summary

User sequences serve as core primitives powering machine learning models across ranking, retrieval, and recommendation surfaces at Pinterest. Managing these sequences previously caused high infrastructure costs, operational fragility, and data drift between offline training pipelines and online serving systems. In response, engineering teams redesigned the platform around a single definition executed across multiple runtimes, including real-time streaming, batch backfills, and online inference. The updated architecture pairs a unified enrichment engine with columnar time-partitioned storage and standardized validation tiers. Consequently, the redesign achieved lower storage and network costs, faster signal onboarding via configuration, and improved engagement metrics across major recommendation surfaces.

## Context

At Pinterest, user sequences power offline training, offline analysis, and online inference across Home feed, Related Pins, and Search Results. Managing user sequences across multiple teams and models had become expensive, fragile, and prone to split-brain data drift between batch training datasets and real-time serving systems.

## Approach / What changed

Pinterest redesigned its user-sequence infrastructure around a 'one definition, many runtimes' architecture using a single configuration surface and shared execution engine across streaming, batch, and online serving. The system incorporates columnar time-partitioned storage, event-level and sequence-level shadow validation, and configuration-driven signal onboarding.

## Takeaways

- Defining signals and events once across batch, real-time, and serving runtimes eliminates data drift between offline training and online inference.
- Validating legacy-to-new pipeline migrations requires a two-tiered comparison strategy spanning event-level field matching and end-to-end sequence output comparisons alongside A/B testing.
- Adopting columnar, time-partitioned storage and reducing network transfer per request lowered infrastructure costs while boosting engagement metrics on major surfaces.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://medium.com/pinterest-engineering/making-user-sequence-data-more-cost-efficient-faster-and-easier-to-use-2a56a928cae1)