Announcing Lakebase Search: agent-native retrieval built into Lakebase Postgres

Summary

Lakebase Search is a beta offering on AWS and Azure that adds hybrid vector and full-text retrieval to Lakebase Postgres. It uses the lakebase_vector and lakebase_text extensions to keep retrieval, memory, operational data, and hybrid search in one backend. lakebase_vector retains pgvector types and operators, applies RaBitQ clustering and compression for 32x smaller indexes, and targets more than 1B vectors, while lakebase_text replaces GIN with object-storage-optimized BM25 ranking. A tiered cache keeps hot data on NVMe and places colder data in object storage; the source reports lower memory needs, faster index builds, and cold-cache startup than standard pgvector HNSW in its LAION-100M benchmark. The extensions also combine vector similarity and keyword relevance with reciprocal rank fusion in a single SQL query, enabling joins and tenant filtering alongside transactional workflows.

Context

Agents use search as part of an operational read/write loop: they retrieve context, write new memories, and need newly generated data to be searchable immediately. The source identifies cold, mostly idle data, vector-index bloat, and memory-bound traditional indexes as constraints for large-scale multi-tenant search.

Approach / What changed

Lakebase Search adds the lakebase_vector and lakebase_text Postgres extensions on a tiered Lakebase architecture. The vector extension uses native pgvector types with RaBitQ compression and clustering, while the text extension provides BM25 through an index designed for sequential object-storage reads. Both support hybrid search through reciprocal rank fusion in one SQL query.

Takeaways

• lakebase_vector preserves standard pgvector types and operators while using RaBitQ to reduce index footprint 32x; a 100-million-vector index described in the source fits in under 10 GB instead of 300 GB of RAM.
• On LAION-100M, lakebase_vector runs on a 192 GB instance with a 1.5-hour index build; standard pgvector HNSW is reported to require a 512 GB instance and about 40 hours to build.
• lakebase_text replaces RAM-resident GIN indexing with an object-storage-optimized index and native BM25 ranking, while reciprocal rank fusion combines keyword and vector results in one SQL query.

Related reading