# Lyft’s Feature Store: Architecture, Optimization, and Evolution

[Lyft](https://yomu.fyi/company/lyft) · Rohan Varshney · Jan 6, 2026

**Type:** Explainer

## Summary

Lyft built a centralized Feature Store to optimize feature management, deployment, and data consistency for machine learning workflows across its rideshare platform. The system ingests batch features defined by Spark SQL and JSON metadata into Airflow DAGs, which simultaneously write data to Hive for model training and to an online serving layer for low-latency inference. Real-time streaming features are processed with Apache Flink and Apache Kafka before being ingested into the serving infrastructure. The online layer, dsfeatures, couples DynamoDB with a ValKey write-through LRU cache and OpenSearch to serve embeddings. Infrastructure optimizations reduced standard P95 read latency by a third while supporting over a trillion additional read and write operations.

## Context

Lyft needed to centralize feature engineering and guarantee consistency across offline model training and real-time inference for dozens of critical rideshare workflows.

## Approach / What changed

Lyft implemented a platform of platforms covering batch, streaming, and online serving. Batch features use Spark SQL and JSON metadata to generate Airflow DAGs feeding Hive and dsfeatures. Streaming pipelines ingest Kafka events through Apache Flink applications. The online layer uses DynamoDB backed by a ValKey write-through LRU cache, OpenSearch for embeddings, and Golang and Python SDKs.

## Takeaways

- The batch ingestion pipeline automatically converts Spark SQL queries and JSON configuration files into production-ready Airflow DAGs that deliver data to Hive and online storage.
- The dsfeatures online serving layer utilizes DynamoDB as primary storage, an OpenSearch integration for embedding features, and a ValKey write-through LRU cache to minimize retrieval latency.
- Platform optimizations and cache TTL tuning reduced P95 read latency by one-third while total aggregate read/write activity increased by over one trillion operations.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Caching](https://yomu.fyi/topic/caching), [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://eng.lyft.com/lyfts-feature-store-architecture-optimization-and-evolution-7835f8962b99)