---
title: "Stripe’s new AI Assistant in VS Code"
description: "Stripe has introduced an AI Assistant in its VS Code extension to answer developer questions using current Stripe knowledge rather than relying on LLM pre-training. The extension retrieves API reference entries, integration guides, code examples, and curated summaries of developer Discord threads, while inserting the user’s API key into generated snippets and supporting either GitHub Copilot’s @stripe agent or its own chat interface. Its retrieval pipeline classifies queries, combines BM25 keyword and k-nearest-neighbor embedding search, reranks results, and sends selected sources, code, user code, and the question to Claude Sonnet through a RAG prompt. A nightly Temporal workflow refreshes indexed content, while evaluation uses golden and synthetic datasets, Mean Reciprocal Rank, and human and automated review. On its synthetic test dataset, the system includes the best source about 91.11% of the time and reports an MRR of about 78%."
---

# Stripe’s new AI Assistant in VS Code

[Stripe](https://yomu.fyi/company/stripe) · Mathew Varughese · Mar 18, 2025

**Type:** Announcement

## Summary

Stripe has introduced an AI Assistant in its VS Code extension to answer developer questions using current Stripe knowledge rather than relying on LLM pre-training. The extension retrieves API reference entries, integration guides, code examples, and curated summaries of developer Discord threads, while inserting the user’s API key into generated snippets and supporting either GitHub Copilot’s @stripe agent or its own chat interface. Its retrieval pipeline classifies queries, combines BM25 keyword and k-nearest-neighbor embedding search, reranks results, and sends selected sources, code, user code, and the question to Claude Sonnet through a RAG prompt. A nightly Temporal workflow refreshes indexed content, while evaluation uses golden and synthetic datasets, Mean Reciprocal Rank, and human and automated review. On its synthetic test dataset, the system includes the best source about 91.11% of the time and reports an MRR of about 78%.

## Context

Stripe’s documentation exceeds 130 MB and changes hundreds of times each week, while LLM pre-training can become outdated and produce incorrect information. Payments integrations are sophisticated and highly customizable, creating a need for answers grounded in current Stripe-specific knowledge.

## Approach / What changed

The VS Code extension uses Retrieval Augmented Generation. It classifies queries, retrieves document chunks through hybrid BM25 keyword and k-nearest-neighbor embedding search, reranks results using page traffic and other factors, and prompts Claude Sonnet with relevant sources, code examples, user code, and the question. A nightly Temporal workflow chunks and embeds documentation and curated Discord-thread summaries. The system supports GitHub Copilot’s @stripe agent and a standalone chat UI.

## Takeaways

- Hybrid retrieval combining BM25 keyword search with semantic embeddings improved accuracy over embedding search alone, while classification and reranking reduced irrelevant Stripe Apps results.
- A pipeline summarizes developer Discord troubleshooting threads with an LLM, then uses human evaluations to select high-quality summaries for the search index.
- Nightly evaluation reports that the best source is included about 91.11% of the time on the synthetic dataset, with a Mean Reciprocal Rank of about 78%.

**Tags:** [Developer Experience](https://yomu.fyi/topic/developer-experience), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Search](https://yomu.fyi/topic/search)

- Source: [Stripe](https://stripe.dev/blog/stripes-ai-assistant-vs-code)
- Source URL: https://stripe.dev/blog/stripes-ai-assistant-vs-code
- Ingested by Yomu: 2026-08-28T08:58:17.727Z

[Read original post](https://stripe.dev/blog/stripes-ai-assistant-vs-code)