Loading…
Topic
8 posts about RAG. Every summary links to the original.
See the idea
Consolidating agent memory, retrieval, and governance onto MongoDB Atlas removes sync and vendor layers, but ties scaling and control to one platform. This trade-off view shows the payoff, the cost, and the condition that decides it.
A comparison of Apache Lucene's BM25 scoring and StableTfl for paginated search: BM25 maximizes relevance, while StableTfl trades a small relevance drop for consistent ranking across replicas and pages.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Granite Switch 4.1 models combine chat, general and specialized decisions, and speech-to-text from a single hosted endpoint. The 8B and 30B models include a new Decision aLoRA for general-purpose decision-making and a Guardian aLoRA that outperforms Jev 1.13 and other closed models on safety tasks. They also embed Granite-Speech-5.0-Turbo-CTC, the fastest ASR model on the OpenASR Leaderboard, enabling efficient transcription. All components are composed into one model using cache-efficient Activated LoRAs, taking up about the same GPU VRAM as the base model alone, and the KV cache from base generation can be reused for any subsequent adapter decisions. Granite Switch 4.1 can switch between chat, decision, and spoken requests quickly, and a 3B variant achieved 60-100 decisions per second playing Doom, compared to Jev’s 10, while also narrating and answering questions.
FactSet’s Text-to-Formula (TTF) system converts natural-language investment queries from professionals into correct FactSet Query Language (FQL) formulas. Initially built on retrieval-augmented generation (RAG), the approach hit a quality ceiling as queries became more specialized and formulas more complex. FactSet adopted fine-tuned open models within a Compound AI architecture, reducing end-to-end latency by about 60 percent and improving quality for complex formulas. Fine-tuning runs on Databricks AI Runtime with serverless GPU infrastructure, where engineers choose compute based on prompt length and output needs, and the pipeline integrates with MLflow, the Lakehouse, and Lakeflow Jobs for a unified workflow. This setup cut model setup, training, and deployment from days to hours, enabling the team to retrain every two to three weeks as new data and models become available.
Many engineering teams mistakenly think building an internal AI knowledge context layer is as simple as standing up a vector database and wiring a RAG pipeline to documentation. In reality, a production-grade pipeline requires five continuously maintained stages—capture, deliver, govern, organize, and validate—where each stage’s long-term cost compounds as vendor APIs change and content ages. The critical innovation is the knowledge node, a living record carrying source, ownership, and an automatically computed trust score based on authorship, usage, recency, and conflicts. Trust decays over time, prompting owners for lightweight revalidation when confidence drops, and knowledge is automatically scoped to the correct audience without manual filing. For product companies not in the knowledge-infrastructure business, the compounding maintenance burden makes building this pipeline a distraction compared to leveraging existing platforms like Stack Internal.
Tool-augmented LLM agents must discover relevant APIs from large registries before planning, but finite context windows prevent injecting all tool definitions into a single prompt. Two competing architectures are Retrieve-then-Plan (RtP), which embeds the query and retrieves similar tools via nearest-neighbor search, and Plan-then-Retrieve (PtR), which first generates abstract tool stubs as retrieval queries. Evaluating on ToolBench with 46,984 API endpoints across 50 categories reveals a clear complexity-dependent crossover: RtP outperforms PtR on single-tool queries while PtR outperforms RtP on multi-tool queries. PtR’s stub-hallucination failure mode acts as a double-edged sword—on simple queries it abstracts away proper-noun signals, yet on complex queries it decomposes multi-faceted intent into functional sub-needs. PtR roughly doubles inference cost, which pays off on multi-tool queries but hurts on single-tool ones, motivating complexity-adaptive routing driven by RtP’s top-1 retrieval score.
AstaBrief is an open-weights 8B-parameter model developed by Ai2 for generating cited scientific reports from a research question and literature excerpts, designed for evidence-grounded outputs. It was trained from Qwen3-8B using supervised fine-tuning on 47K filtered real user queries and direct preference optimization, focusing on data quality. Unlike Asta’s Thinking mode, which uses a multi-step pipeline, AstaBrief writes the full report in a single pass to reduce generation time. In Asta’s Fast mode, it averages 51.1 seconds per report versus 178.5 seconds for Thinking mode, roughly 3.5× faster, while competitive on answer and citation quality. The model and training data are open-sourced on Hugging Face, enabling institutions to deploy behind their own firewall and supporting future open scientific models.
Barclays, a British universal bank, is expanding its collaboration with Anthropic to integrate Claude across global operations, aiming to accelerate software development, modernize legacy systems, and improve efficiency. By end of 2026, Barclays expects Claude Code adoption to reach 50% of its developer population, rising to a majority of software engineers in 2027. A live example is the Colleague Knowledge Assistant, powered by Claude via a retrieval-augmented generation architecture since 2025, adopted by over 16,000 employees and handling over one million searches. In Global Markets, Claude models classify, enrich, and route approximately 120,000 client emails daily, reducing manual handling and helping staff prioritize requests. Barclays applies robust governance, security controls, and human oversight to ensure responsible AI deployment.
Model versioning alone is insufficient for production AI because changes in inputs, preprocessing, prompts, retrieval, tool contracts, and serving settings can alter behavior independently. The post proposes a versioned release manifest that captures all interdependent components, including model revision, prompt revision, retrieval index revision, embedding revision, pipeline revision, and runtime revision covering token limits, batching, and timeouts. This manifest enables atomic release testing and rollback, with an evaluation gate that tests the full retrieval-generation path using deterministic checks and semantic rubrics on a versioned dataset. Practical steps include canary deployments that compare candidate and control signals, with explicit stop conditions and rollback procedures that restore compatible dependencies rather than just older model weights. The post concludes that the smallest useful implementation is a manifest, evaluation job, load test, release-aware traces, and a rehearsed rollback, and asks whether the on-call engineer can identify and restore a compatible known-good version.
The article demonstrates extending Red Hat OpenShift Lightspeed by integrating the Red Hat Satellite MCP server and bring your own knowledge (BYOK). The Satellite MCP server exposes live operational data, such as kernel versions, CVEs, and errata, through the Model Context Protocol. BYOK enriches the assistant with private enterprise knowledge, enabling retrieval-augmented generation (RAG) to combine internal documentation with infrastructure data. The integration requires installing the OpenShift Lightspeed operator and deploying the Satellite MCP server in the openshift-lightspeed project. Authentication is handled by storing the header value in a Secret, and the MCP server expects clients to include appropriate Accept headers for the text/event-stream media type.
The best engineering reads, once a week.
