Lakebase Search: State-of-the-art full text and vector search for Postgres

Summary

Traditional OLTP systems like Postgres face challenges meeting the low-latency, high-accuracy search demands of AI agents, often requiring standalone search engines with ETL pipelines. Lakebase introduces lakebase_vector and lakebase_text, two extensions that bring scalable approximate nearest neighbor search and BM25 full-text search directly into Lakebase Postgres. lakebase_vector overcomes pgvector’s memory-bound HNSW index by using hierarchical IVF clustering and binary quantization, achieving 71-millisecond P99 latency at 97 percent recall on the 100-million-vector LAION benchmark. The extensions decouple storage from compute for pay-per-use scaling, enable fast parallel index builds, and support native hybrid search that fuses vector similarity with keyword relevance in a single SQL query. Early adopters like Conexiom reduced infrastructure costs by three times while achieving five times higher throughput, and the service is generally available on AWS and Azure.

Takeaways

1. 1

   lakebase_vector outperforms pgvector at scale by using hierarchical IVF clustering and binary quantization, avoiding memory-bound HNSW; it achieves 71ms P99 latency at 97% recall on 100M vectors.

2. 2

   Lakebase Search decouples storage from compute, enabling pay-per-use scaling, fast cold starts (1.13s P90 after scale-to-zero for 100M vectors), and index builds that can be offloaded to distributed engines like Spark.

3. 3

   Native hybrid search in Lakebase Postgres combines vector similarity with BM25 keyword relevance and standard SQL filters in a single query, allowing consolidation of operational and search workloads into one serverless database.

FAQ