---
title: "Two years of vector search at Notion: 10x scale, 1/10th cost"
description: "Notion describes two years of scaling vector search for AI Q&A, where semantic embeddings retrieve workspace and connected-tool content by meaning rather than exact keywords. The initial system combined Apache Spark batch indexing with Kafka-driven real-time updates, workspace-based sharding, and generation routing; rapid demand drove a 600x increase in daily onboarding capacity and cleared a waitlist of millions of workspaces by April 2024. A serverless migration cut vector database costs 50 percent from peak usage, while a later move to turbopuffer reduced search-engine spend 60 percent, lowered AWS EMR costs 35 percent, and improved p50 query latency from 70–100ms to 50–70ms. Further work uses xxHash-based page-state caching in DynamoDB to reprocess only changed spans and shifts embeddings generation from Spark and query embeddings away from a third-party API to Ray on Anyscale, with early rollout results pointing to a projected 90+ percent reduction in embeddings infrastructure costs."
---

# Two years of vector search at Notion: 10x scale, 1/10th cost

[Notion](https://yomu.fyi/company/notion) · Preeti Gondi, Mickey Liu, Nathan Louie, Calder Lund, Jacob Sager · Feb 19, 2026

**Type:** Explainer

## Summary

Notion describes two years of scaling vector search for AI Q&A, where semantic embeddings retrieve workspace and connected-tool content by meaning rather than exact keywords. The initial system combined Apache Spark batch indexing with Kafka-driven real-time updates, workspace-based sharding, and generation routing; rapid demand drove a 600x increase in daily onboarding capacity and cleared a waitlist of millions of workspaces by April 2024. A serverless migration cut vector database costs 50 percent from peak usage, while a later move to turbopuffer reduced search-engine spend 60 percent, lowered AWS EMR costs 35 percent, and improved p50 query latency from 70–100ms to 50–70ms. Further work uses xxHash-based page-state caching in DynamoDB to reprocess only changed spans and shifts embeddings generation from Spark and query embeddings away from a third-party API to Ray on Anyscale, with early rollout results pointing to a projected 90+ percent reduction in embeddings infrastructure costs.

## Context

Rapid demand for Notion AI Q&A created a waitlist of millions of workspaces, capacity constraints, and operational complexity across multiple vector-database generations. The indexing pipeline also reprocessed entire pages when only individual spans or metadata changed, while separate preprocessing and embedding systems created extra compute costs and dependencies on external API reliability.

## Approach / What changed

Notion expanded onboarding with Airflow, pipelining, and Spark tuning; introduced generation-based routing; migrated from dedicated pod infrastructure to serverless storage and compute; moved the corpus to turbopuffer; added xxHash-based page-state caching in DynamoDB; and began shifting embedding generation and serving to Ray on Anyscale.

## Takeaways

- The initial indexing design paired Apache Spark batch processing for existing documents with Kafka consumers for near-real-time page edits, using workspace IDs for partitioning and generation IDs for routing.
- The turbopuffer migration reduced search-engine spend by 60 percent, cut AWS EMR compute costs by 35 percent, and improved p50 production query latency from 70–100ms to 50–70ms.
- DynamoDB stores per-page text and metadata hashes using 64-bit xxHash, allowing changed spans to be re-embedded selectively while metadata-only changes avoid unnecessary embedding work.

**Tags:** [Apache Spark](https://yomu.fyi/topic/apache-spark), [DynamoDB](https://yomu.fyi/topic/dynamodb), [Kafka](https://yomu.fyi/topic/kafka), [Ray](https://yomu.fyi/topic/ray), [Search](https://yomu.fyi/topic/search)

- Source: [Notion](https://www.notion.com/en-gb/blog/two-years-of-vector-search-at-notion)
- Source URL: https://www.notion.com/en-gb/blog/two-years-of-vector-search-at-notion
- Ingested by Yomu: 2026-09-09T05:20:54.591Z

[Read original post](https://www.notion.com/en-gb/blog/two-years-of-vector-search-at-notion)