# LLM-assisted vector similarity search

[Grab](https://yomu.fyi/company/grab) · Md Riyadh · Oct 23, 2024

## Summary

Vector similarity search often struggles with conceptual nuances, negations, and complex constraints despite its efficiency in high-dimensional semantic retrieval. To address this limitation, a two-step retrieval pipeline combines Facebook AI Similarity Search (FAISS) using OpenAI's text-embedding-ada-002 embeddings with a large language model re-ranking step powered by GPT-4o. The initial FAISS search extracts a candidate shortlist of matches, which GPT-4o then re-ranks based on contextual relevance and query intent. Evaluations on structured datasets showed that while simple queries yielded comparable results across methods, the LLM-assisted approach effectively resolved negations and conceptual matching where raw vector search failed. Internal production trials on relational datasets exceeding 4,500 rows demonstrated similar relevance improvements, with additional query latency being the primary operational trade-off.

## Takeaways

- Raw vector similarity search can misinterpret queries with negations or conceptual nuances, whereas an LLM re-ranking stage correctly filters out excluded criteria.
- The hybrid architecture uses FAISS and text-embedding-ada-002 to retrieve a preliminary shortlist of candidates before prompting GPT-4o to select the final ranked results.
- While standard vector search matches the performance of the LLM-assisted approach on simple keyword queries, the multi-stage pipeline provides higher accuracy for complex queries at the expense of additional latency.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Python](https://yomu.fyi/topic/python), [Search](https://yomu.fyi/topic/search)

[Read original post](https://engineering.grab.com/llm-assisted-vector-similarity-search)