LLM-assisted vector similarity search

Summary

Vector similarity search can struggle with queries containing negation, constraints, or conceptual requirements, even though it efficiently retrieves semantically similar records. The proposed method uses FAISS and OpenAI text-embedding-ada-002 embeddings to shortlist 15 candidates, then asks GPT-4o to rank the candidates against the original query and return the top three. On two GPT-4o-curated synthetic datasets—100 food dishes and 100 Asian tourist spots—the assisted method matched raw search on straightforward queries but produced more relevant results for complex ones, including excluding fish or shrimp and identifying wildlife-related places. The authors note limits from the small datasets and query set, report similar relevance improvements in an internal deployment involving more than 4,500 relational-database rows, and identify shortlist size and added LLM latency as areas for further evaluation.

Takeaways

1. 1

   LLM-assisted ranking filtered out fish- and shrimp-containing dishes for the query “food with no fish or shrimp,” while raw vector search returned seafood-related results.

2. 2

   For complex queries, the assisted method improved relevance; for straightforward queries aligned with textual content, it performed comparably to raw similarity search.

3. 3

   The approach adds an LLM query and therefore introduces latency considerations; future evaluation includes varying shortlist size and testing larger, more diverse datasets.