Agent Memory: What Outlives the Run
Agent memory is the knowledge that survives a run, not the working state inside one. This explainer separates the two, then maps the write path, the read path, and the database limits that cap recall.
Loading…
Topic
Retrieval-Augmented Generation
34 posts about Retrieval-Augmented Generation. Every summary links to the original.
- 34 posts
- 17 companies
- 2 explainers
See the idea
Retrieval-Augmented Generation explainers
AI Search: Speed vs. Relevance Trade-off
AI Search balances retrieval speed and result quality by choosing between dense embeddings, sparse lexical search, and hybrid approaches. Understand which trade-off fits your workload.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
EmbeddingGemma 2: The Developer Guide
Modern search and RAG applications need to work across diverse content types, but finding models with strong retrieval accuracy and low latency without massive compute is challenging. EmbeddingGemma 2 is a compact open model under Apache 2.0, based on Gemma 4, that maps text, code, images, video, and audio into a unified 768-dimensional space with a modular architecture scaling from 270M to 740M parameters. Key capabilities include native multimodal retrieval, superior code understanding, modular memory footprint, and flexible vector storage via Matryoshka Representation Learning enabling dynamic truncation from 768 to 128 dimensions. Under the hood, it uses modular encoders—a text/code decoder, vision encoder, and audio encoder—that project into a shared space and can be loaded selectively to minimize memory. The developer guide demonstrates how to use sentence-transformers to load the model, embed with task prompts, handle media inputs, truncate dimensions, and choose configurations based on storage and recall needs.
Lower the Cost of Building and Running Visual AI Agents with NVIDIA VSS Blueprint 3.3
Vision-language models enable production-scale video understanding, but building maintainable systems that combine ingestion, stream processing, event detection, retrieval, summarization, and reporting with services like Kafka, Redis, and Elasticsearch is challenging. The NVIDIA VSS Blueprint 3.3 introduces the Build Vision Agent skill, which composes multi-workflow applications from a single natural-language prompt using one of four Foundation profiles. It also adds Adaptive EVS, which prunes unchanged video patches via cosine similarity before VLM processing and batches tokens around events. Adaptive EVS cuts alert latency by 17%, delivers 80% fewer VLM input tokens for a 60-minute summary, and enables 46% more concurrent streams on the same GPU. These updates reduce development effort, infrastructure duplication, and operating costs, allowing a single prompt to build a bottling-line agent in under 30 minutes.
Building a knowledge assistant over code
Building a knowledge assistant over code requires more careful chunking than indexing ordinary business documents, because code has nested structure, cross-file dependencies, and multiple file formats. The comparison built three Databricks Knowledge Assistants over the Casper’s Kitchens repository: fixed-size character chunks, LangChain language-aware splitting, and AST-based splitting with structure and metadata headers. The AST approach used ASTChunk to preserve units such as functions, classes, or statement blocks and prepend file paths plus class/function hierarchies to chunks. MLflow GenAI evaluation ran 46 questions with retrieval sufficiency, groundedness, and custom answer-correctness judges; AST chunking achieved 89% retrieval sufficiency and 70% fully correct answers, versus 85% and 59% for the naive baseline. The advantage was strongest for disambiguating structurally similar code, while value lookups and complete definition retrieval were comparable across methods.
What is Intelligent Document Processing?
Intelligent document processing (IDP) uses AI to extract, classify, validate, and structure information from PDFs, images, emails, forms, and other structured, semi-structured, or unstructured documents. Its pipeline combines document ingestion and preprocessing with OCR, document classification, NLP, layout analysis, data validation, human review for uncertain results, and delivery into downstream systems such as ERP, CRM, or accounting software. The post explains how embeddings support classification, how continuous learning adapts models to new layouts and corrections, and how metrics such as accuracy, throughput, confidence scores, and model drift support optimization. IDP can reduce manual effort, improve accuracy, lower costs, scale document-heavy workflows, and supply structured data for analytics, automation, AI agents, and retrieval-augmented generation, while high document variability and edge cases still make human-in-the-loop review relevant.
Tevogen Bio’s Journey to Streamlining Life
Tevogen Bio faced drug development costing upwards of $3 billion and requiring 10–12 years; selecting targets for its single-HLA-restricted SARS-COV2 product took 18–24 months of wet-lab work. Tevogen.AI partnered with Microsoft and Databricks to build and curate a multi-terabyte protein-sequence library and parallelize processing for machine-learning training. Databricks Data + AI Platform, Medallion Architecture, Unity Catalog, and distributed computing cut processing from 50 days to 24 hours under governed, fine-grained access controls. The dataset reached 24 million proteins, 16 billion datapoints, ~700 million unique peptides, and ~37 million cross-matched expert articles. An MLOps framework automated training, inference, monitoring, and retention; the alpha PredicTcell model used XGBoost and ESM models and reported 93–97% recall and 38–43% accuracy, while beta PredicTcell and alpha AdapTcell work began.
Building a RAG Pipeline for Semantic Code Search: A Developer Diary and Field Notes
JetBrains Context is presented as a production RAG pipeline for semantic code search, designed to give coding agents precise, citable evidence from repositories instead of relying on keyword tools such as grep. This first installment covers parsing, structure-aware chunking, and vectorization, with the goal of retrieving meaningful functions, symbols, or snippets rather than whole files or insignificant lines. Language parsers produce syntax nodes, and the algorithm retains the largest units that fit, splits oversized leaf nodes when necessary, groups small adjacent units, and keeps documentation, annotations, modifiers, and closing syntax with their constructs. JetBrains Context supports this approach for Kotlin, Java, Python, JavaScript, TypeScript, C#, PHP, Go, and Rust; other languages use line-based splitting. Vectors are generated with an open-weight model on company-operated GPUs, while indexed records store coordinates rather than source code, and the post reports that this setup outperformed hosted embedding APIs on internal code-retrieval benchmarks.
Glyph: A Multi-Strategy Agentic System for Column Description and Sensitivity-Ontology Tagging of Enterprise Data Catalogs
Glyph addresses enterprise data lakes where tables accumulate faster than human stewards can document or classify them, leaving column descriptions and governance labels incomplete. It models column description generation and sensitivity-ontology tagging as cooperating LLM agents orchestrated as stateful graphs. The Descriptor uses active Retrieval-Augmented Generation to retrieve pipeline source code from an enterprise GitHub, while the Tagger runs description, line-of-business regex, and metadata strategies in parallel before combining ranked outputs with Reciprocal Rank Fusion. A fine-tuned six-layer MiniLM metadata encoder raised same-tag retrieval on an in-distribution held-out split from NDCG@10 0.55 to 0.92 and MAP@100 0.19 to 0.90, relative to the stock base encoder. Glyph also evaluates end-to-end multilabel tagging with a recall-weighted F2 objective across three groups and an ablation, emphasizing value-free, code-grounded operation, per-tag provenance, and graceful degradation for auditable production cataloging.
Bringing custom knowledge to agents with AutoRAG
Enterprise LLMs can hallucinate when they lack company-specific vocabulary and documentation, so the guide builds a bank agent grounded in internal data. It configures Red Hat OpenShift AI 3.5 Technology Preview features, enables OGX, AutoRAG, and the MCP catalog, and deploys PostgreSQL, Milvus, and MinIO for metadata, vector embeddings, and artifacts. AutoRAG processes documents, evaluates pipeline configurations against a benchmark dataset, and identifies a preferred setup; the example uses Qwen3-8B and embeddinggemma-300m models. A MariaDB MCP server adds access to customer records, with a patch supplying writable logging storage and a warning that unrestricted ALLOWED_HOSTS is for demo/development only. The resulting Streamlit agent answers a customer-information query and a Gold-card-limit question using RAG context and database tools.
Enterprise AI Accuracy: Building a More Trustworthy RAG Application
An enterprise RAG pipeline scored above 90% on standard text-retrieval benchmarks but approximately 46% on complex, customer-representative documents, with failures spanning parsing, chunking, enrichment, embeddings, retrieval, and answer generation. The proposed diagnostic traces wrong answers backward from generated response to retrieved context, chunks, and parsed source, using human-validated end-to-end evaluation and changing one stage at a time. Intelligent Parsing routes pages according to complexity and preserves relationships in structures such as multi-page tables; in a staged benchmark, it raised accuracy from 65.1% to 84.4%, while Enriched Indexing raised it to 86.8%. Retrieval improvements include dynamic metadata pre-filters for explicit constraints and GraphRAG for multi-hop evidence, with GraphRAG slated for general availability in November 2026. Across broader improvements, complex-document accuracy exceeded 90%, and service-based Just-in-Time indexing reduced sub-100 MB payload indexing from 30+ minutes to approximately seven to ten minutes.
Rust AI in Practice: Building LLM Applications With Rig
Rig is an open-source Rust library that provides a common interface for LLM providers including OpenAI, Anthropic, and Gemini, avoiding application rewrites when switching services. It structures applications around a provider client, completion model, agent, preamble, and registered tools, with async APIs handling provider network requests. In the Rat Code demo, built with Ratatui, separate tools read and write files and run shell commands, while Rig’s streaming API updates the terminal as responses arrive. The library also supports RAG through vector store abstractions, local inference via Ollama, llama.cpp, or Candle, and cassette-based integration tests that replay recorded HTTP interactions in CI; live model quality requires separate scheduled testing.
An Organizational Second Brain: Building an AI That Learns From Experts
Organizations often retain specialist knowledge in experts’ heads, leaving routine compliance assessments slow, inconsistent, and difficult to scale. The proposed “second brain” combines a structured, auditable knowledge architecture with a separate reasoning layer and a self-improvement loop that converts expert feedback into verified, regression-tested text updates without model retraining. High-density organizational positions, taxonomies, routing indexes, and gateway files are maintained in a dependency-linked filesystem, while sparse supporting material remains available through semantic or lexical retrieval. Composable recipes encode expert procedures, and an evaluation pipeline proposes auditable pull requests, retries failed fixes, and expands regression coverage after approval. After six weeks, SMEs reportedly found outputs useful almost always, assessments fell from days to minutes, and improvement cycles produced zero regressions; the architecture is presented as applicable beyond compliance.
Why AI Agents Get the Right Facts but the Wrong Answer—and How GraphRAG Helps
The post examines why an agent can cite accurate return-policy evidence yet reject an eligible customer when decisive facts are distributed across systems. Its washing-machine scenario combines a product classification, an extended-policy rule, membership tier, purchase amount, and purchase date to show that retrieving enough evidence differs from retrieving relevant evidence. GraphRAG addresses the gap by extracting entities and relationships into a knowledge graph, then using multi-hop retrieval to follow connections from WM-500 to the applicable policy and supporting records. The implementation also uses explicit pointers from graph entities to structured data, while separating the TBox blueprint from ABox instances and requiring business validation of the schema. The diagnostic framework distinguishes missing retrieved context, omitted business rules, and broken record mappings, while noting that these examples are not benchmark results and that correct evidence can still be used incorrectly.
What is an AI Copilot?
An AI copilot is an assistant embedded in a software application that uses workflow context to offer suggestions, generated content, or approved actions while keeping the human in control. Unlike a standalone chatbot, it combines an LLM with contextual grounding, retrieval-augmented generation (RAG), access to enterprise data, and action layers that can call APIs, execute code, trigger workflows, or update records. The explainer surveys code, productivity, data and analytics, customer-facing, and domain-specific copilots, including uses such as SQL generation, dashboard creation, email drafting, and case summarization. Reported benefits include faster routine work, broader data access, more consistent outputs, and less context switching; a cited GitHub study found Copilot users completed a controlled coding task 55% faster. It also stresses hallucinations, privacy, bias, overreliance, integration complexity, cost, and the requirement for human review, governance, and accountability.
DE Dost: Teaching Machines to Be a Friend on the Last Mile
DE Dost is a GenAI onboarding companion embedded in Swiggy’s delivery-executive app, designed for new DEs who receive information but lack confidence-building, immediate guidance. It supports questions in nine languages across SOPs, earnings, payouts, app navigation, shifts, and support, adapting detail and tone to conversational context. Before the first LLM call, a graph state assembles user context, preprocessing resolves references, and embedding-based semantic routing separates general knowledge-base questions from personal diagnostics. Retrieval uses authored QnA pairs plus hybrid semantic and lexical scoring to handle paraphrases and terms such as “MG” and “COD,” while guardrails constrain follow-ups to available knowledge. Evaluation spans about 800 cases per language across fleet and payout combinations, with macro accuracy trending at 90%+ and a target P50 latency of 4.5 seconds after reducing an initial 15-second pilot.
RAG vs Fine Tuning: Enterprise Decisions for AI Models and AI Systems
The guide frames enterprise choice between retrieval-augmented generation (RAG) and fine tuning as a trade-off between injecting knowledge at inference time and changing model weights before deployment. RAG connects an LLM to external stores by embedding a query, searching vector databases for similar chunks, and adding retrieved context to the prompt, making current information and citations possible. Fine tuning uses curated input-output examples and supervised training to update parameters; full-model training is expensive, while LoRA trains a smaller set of added weights and can support a single-GPU pilot. The decision depends on whether the priority is stable behavior and output format or fresh, traceable knowledge, alongside data quality, latency, compute, and maintenance requirements. For many mature deployments, the guide recommends combining both: fine tuning for domain understanding and consistency, and RAG for real-time data.
Supercharge your integration workflow with the Google Pay & Wallet Developer MCP server
Google Pay & Wallet Developer MCP server is announced to help AI assistants and IDEs complete Google Pay and Google Wallet API integrations with less context switching. It uses Model Context Protocol to connect tools including Antigravity, Cursor, and Visual Studio Code with current documentation, account details, and integration status. Its search_documentation tool applies Retrieval-Augmented Generation to return answers and code samples from official developer sites, while other tools validate Wallet pass JWTs or JSON, expose performance metrics and error trends, and manage merchant accounts and Google Pay API integrations. The stated goal is to reduce friction, improve accuracy with current documentation and real account data, and speed troubleshooting from setup through launch. A public guide and tool reference are available, with further capabilities planned based on developer feedback.
How World Bank Group uses Databricks to eradicate poverty through shared knowledge
The World Bank Group is building a unified data and AI platform on Databricks to make tens of millions of repository documents and structured operational data findable for teams pursuing shared prosperity. Legacy databases and disconnected document repositories forced researchers to search manually, slowing decisions and limiting the ability to surface lessons learned across its global portfolio. The implementation migrates operational data, applies Unity Catalog governance, uses Databricks Volumes for unstructured content, Genie for natural-language structured queries, a metrics layer for deterministic reporting, and vector search for retrieval-augmented generation over project documents. Because each Genie serves a specific metrics domain, an agentic layer combines intent and domain classification with query decomposition, routing requests among domain agents, RAG, and visualization components before assembling one response; a multi-region feedback prototype was deployed in approximately two and a half days.
From manual to autonomous: how AI agents are transforming electric grid operations
Electric utilities are facing rising demand, retiring generation, extreme weather, aging infrastructure, and fragmented operational data that manual processes cannot manage at scale. AI agents are presented as a human-centered alternative that synthesizes heterogeneous data, learns from outcomes, and progresses from human-approved recommendations to exception-based control and eventually autonomous operations within defined parameters. Hawaiian Electric used a Retrieval Augmented Generation proof-of-concept with Databricks AI Search, Unity Catalog, and Lakeflow Declarative Pipelines to query regulatory documents and provide page-specific citations. The system reduced response times from five minutes to five seconds and was implemented in two weeks, while the article describes broader potential for predictive maintenance, outage response, load forecasting, and customer service.
Generative AI for Business: A Complete Strategy and Implementation Guide
Generative AI is presented as an enterprise technology shift with potential value across customer operations, marketing and sales, software engineering, and research and development, which together are described as accounting for approximately 75% of use-case value across industries. The guide distinguishes generative systems, which create text, images, code, audio, or structured data from predictive models, and explains how foundation models and LLMs support open-ended business applications. Its implementation strategy prioritizes reliable data infrastructure, high-impact low-complexity pilots, and governance covering sensitive data, model limitations, monitoring, human review, and compliance. Recommended execution includes a cross-functional squad, predefined KPIs, user training, baseline measurements, and a 90-day executive review with scale, iterate, or discontinue decisions. Retrieval-augmented generation is identified as the most widely adopted way to reduce hallucinations by grounding responses in verified proprietary data, while ROI is expected within six to twelve months for a well-structured pilot.
How to transform document activation workflows with Genie and Agent Bricks
Enterprise document workflows often require people to read PDFs, copy fields into spreadsheets, and re-enter data into business systems, creating delays, error risk, and weak auditability. The proposed alternative combines AI/BI Genie for natural-language queries over Delta tables, Agent Bricks for production agents, and Unity Catalog for permissions, lineage, and traceability. A five-phase workflow extracts fields into Lakeflow Bronze, Silver, and Gold layers, then supports structured queries, clause-level RAG answers, multi-agent routing, and actions through MCP servers. Examples include syncing validated rights data to ERP or CRM systems, updating billing entitlements, and triggering workflows from detected obligations. The approach targets media, advertising, ad tech, and telecommunications, where customers see improvements including faster month-end close, recovered revenue, reduced leakage, and lower operational risk.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.