# LLMs > 80 posts about LLMs, summarised, each linking to the original. ## Articles ### [How we optimized Dash's relevance judge with DSPy](https://yomu.fyi/post/how-we-optimized-dash-s-relevance-judge-with-dspy.md) - Company: [Dropbox](https://yomu.fyi/company/dropbox.md) - Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon - Published: Mar 17, 2026 Dropbox Dash optimized its LLM-as-a-judge relevance scoring system using DSPy to migrate from expensive proprietary models to cheaper open-weight alternatives. By establishing automated feedback loops based on human agreement and strict JSON format validation, the team systematically generated robust prompts for new models. This reduced human-score disagreement by 45% and slashed model adaptation time from weeks to days while enabling 10x to 100x more data labeling. ### [Enabling R8 optimization at scale with AI-assisted debugging](https://yomu.fyi/post/enabling-r8-optimization-at-scale-with-ai-assisted-debugging.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Nguyen Van Minh - Published: Mar 12, 2026 Grab experienced widespread Application Not Responding spikes across its Android superapp, driven by memory pressure and complex Jetpack Compose layouts embedded in legacy code. While switching to advanced R8 optimization promised significant performance gains, obfuscated stack traces and two-hour remote compilation cycles stalled investigation across nine million lines of code. To resolve this, engineers built Model Context Protocol tools to automate APK decompilation, deobfuscation, and code context extraction. The team paired these tools with an AI workflow that used the GitLab CLI to generate multiple solution branches and run verification builds in parallel. This strategy replaced hours of manual reverse engineering with minutes of automated analysis, allowing the team to debug and validate aggressive optimizations at scale. ### [Using LLMs to amplify human labeling and improve Dash search relevance](https://yomu.fyi/post/using-llms-to-amplify-human-labeling-and-improve-dash-search-relevance.md) - Company: [Dropbox](https://yomu.fyi/company/dropbox.md) - Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon,Dmitriy Meyerzon - Published: Feb 26, 2026 Dropbox Dash uses large language models (LLMs) to amplify human labeling efforts for training its search relevance and ranking models. By validating and optimizing LLM evaluators against a small set of human-labeled internal data, Dropbox creates massive, high-quality training datasets offline for production rankers like XGBoost without incurring high latency or latency costs at query time. ### [Cursor at Grab: Adoption and impact](https://yomu.fyi/post/cursor-at-grab-adoption-and-impact.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Akshay Misra - Published: Jan 29, 2026 Following a multi-tool AI strategy, Grab integrated the AI coding assistant Cursor into its engineering toolkit in late 2024 to accelerate software development. Technical staff adoption reached 98% monthly active usage with a 50% suggestion acceptance rate, supported by custom monorepo indexing and preconfigured rules aligned with internal coding conventions. Engineers frequently apply the tool to unit test generation, code refactoring, cross-repository navigation, and routine API scaffolding, with over a third of merge requests incorporating Cursor. The rollout also encompasses non-technical personnel and product designers who, after receiving Git training, submit direct production UI fixes. Statistical evaluations using fixed-effects regression indicate a dose-response relationship between Cursor usage intensity and measurable productivity gains. ### [From deployment slop to production reality: How BriX bridges the gap with enterprise-grade AI infrastructure](https://yomu.fyi/post/from-deployment-slop-to-production-reality-how-brix-bridges-the-gap-wi.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Sneh Agrawal - Published: Jan 16, 2026 Internal AI prototypes frequently fail enterprise rollouts due to diverging versions, security oversights, hardcoded credentials, and infrastructure bottlenecks. BriX addresses this deployment gap by turning AI rollout into a configuration-driven platform rather than an engineering rewrite. Built on a synchronous streaming architecture, it routes user prompts through a React frontend using Server-Sent Events, a FastAPI gateway, and LangGraph orchestration. The platform integrates model switching, centralized prompt locks, and standardized Model Context Protocols for governed enterprise data access. ### [Streamlining Security Investigations with Agents](https://yomu.fyi/post/streamlining-security-investigations-with-agents.md) - Company: [Slack](https://yomu.fyi/company/slack.md) - Author: Dominic Marks - Published: Dec 1, 2025 Slack's Security Engineering team handles billions of daily security events and needed a reliable way to streamline on-call alert triage. An initial prototype relying on a single 300-word prompt produced inconsistent results and frequently reached spurious conclusions without properly challenging assumptions. To gain precise control, the team decomposed the workflow into chained model invocations with structured JSON outputs organized across three agent personas: a Director, four domain experts, and a Critic. Domain experts gather raw evidence through tool calls, the Critic evaluates finding quality and synthesizes a timeline, and the Director steers investigation phases using tiered model costs. The multi-agent system enables engineers to supervise investigations via a real-time dashboard while uncovering emergent issues like credential exposures across process ancestry chains. ### [SpellVault’s evolution: Beyond LLM apps, towards the agentic future](https://yomu.fyi/post/spellvault-s-evolution-beyond-llm-apps-towards-the-agentic-future.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Felix Haryanto Lie - Published: Nov 21, 2025 Grab developed SpellVault as an internal no-code platform to democratize the creation of AI applications backed by Retrieval-Augmented Generation (RAG) and plugin integrations. To advance beyond static retrieval and linear input-output processing, the platform transitioned from its legacy executor to a graph-based execution model supporting branching, looping, and ReAct agent patterns. Capabilities like Python code execution and internal repository searching were unbundled from the prompt builder and consolidated alongside user plugins into unified Native and Community Built Tools. The platform also introduced a drag-and-drop deterministic workflow designer, automated task scheduling, and support for the Model Context Protocol (MCP). ### [How we built a custom vision LLM to improve document processing at Grab](https://yomu.fyi/post/how-we-built-a-custom-vision-llm-to-improve-document-processing-at-gra.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jia Chen - Published: Nov 4, 2025 Document processing for identity verification across Southeast Asia presents challenges due to varied layouts and non-Latin scripts. Traditional OCR and off-the-shelf vision models struggle with accuracy, high latency, or lack of regional language training data. Grab addressed this by creating synthetic regional datasets, using an automated labeling pipeline named Documint, and evaluating open-source multimodal architectures. After initial LoRA fine-tuning failed on complex scripts like Thai and Vietnamese, full-parameter fine-tuning of Qwen2-VL 2B yielded substantial gains. To optimize deployment costs and latency, the team constructed a custom 1B parameter model pairing a Qwen2-VL vision encoder with a Qwen2.5 0.5B language decoder, achieving performance within 3 percentage points of the 2B model at significantly lower latency. ### [From failure to success: The birth of GrabGPT, Grab’s internal ChatGPT](https://yomu.fyi/post/from-failure-to-success-the-birth-of-grabgpt-grab-s-internal-chatgpt.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Wenbo Wei - Published: May 19, 2025 Grab's machine learning platform team initially faced overwhelming volumes of repetitive user inquiries across their internal support channels. An initial attempt to automate answers using the open-source chatbot-ui framework and GPT-3.5-turbo failed to scale because the 8,000-token context limit could not accommodate extensive documentation, and embedding search proved inadequate. The project then pivoted to create an internal conversational AI platform called GrabGPT by wiring chatbot-ui with Google authentication and Grab's catwalk model-serving infrastructure. The resulting internal service rapidly expanded across the organization, providing auditable interactions, multi-model support across OpenAI, Claude, and Gemini, and private network routing to safeguard corporate data. ### [Streamlining RiskOps with the SOP agent framework](https://yomu.fyi/post/streamlining-riskops-with-the-sop-agent-framework.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Fujiao Liu - Published: May 8, 2025 Manual Account Takeover (ATO) investigations in Risk Operations traditionally demand intensive cross-referencing across systems, manual SQL execution, and high-pressure decision-making prone to human error. To resolve these bottlenecks, an SOP-driven LLM agent framework models investigative workflows as natural-language tree structures with explicit function notations like @function\_name. Execution is coordinated between an SOP planner, which traverses the tree using a Depth-First Search strategy, and a Worker Agent that parses JSON-formatted steps to invoke database queries and APIs. Once all steps evaluate their decision criteria, the framework synthesizes the collected data into an actionable summary report. Implementing this architecture automated 87% of ATO cases and dropped average ticket handling time from 22 minutes to 3 minutes. ### [Introducing the SOP-driven LLM agent frameworks](https://yomu.fyi/post/introducing-the-sop-driven-llm-agent-frameworks.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Fujiao Liu - Published: Apr 25, 2025 Standard operating procedure (SOP) driven Large Language Model agent frameworks address common generative AI challenges such as hallucinations, non-standard output formats, and branching navigation errors. Workflows are represented as hierarchical trees where nodes encapsulate actions or decision points that can be created using a visual editor and annotated with explicit external function calls. Execution relies on a tripartite architecture consisting of a Depth-First Search planner module with backtracking, an adaptive worker agent that limits API exposure and compresses context, and a multilingual user agent. Supporting tools include a Graph Retrieval-Augmented Generation pipeline, a plugin system integrating Python and SQL, and a state stack for pausing workflows during human intervention. In production deployments for fraud and account takeover investigations, the framework automated up to 87% of cases while cutting handling times substantially. ### [Grab AI Gateway: Connecting Grabbers to multiple GenAI providers](https://yomu.fyi/post/grab-ai-gateway-connecting-grabbers-to-multiple-genai-providers.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Bjorn Jee - Published: Feb 19, 2025 Grab built the AI Gateway to centralize access, cost control, and security across external and open-source Generative AI providers such as OpenAI, Azure, AWS, and Google. Designed as a set of lightweight reverse proxies, the gateway manages authentication, rate limiting, and authorization while translating payloads into a unified OpenAI-compatible interface. The platform archives request metadata and calculated per-call costs into a central data lake for auditing and showback, dynamically routing traffic across shared reserved capacity and regions to mitigate quota throttling. Supporting over 300 internal use cases, the system integrates directly with internal development notebooks and deployment tools to power applications ranging from real-time audio safety analysis to automated content moderation. ### [Supercharging LLM application development with LLM-Kit](https://yomu.fyi/post/supercharging-llm-application-development-with-llm-kit.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Boon Zhan Chew - Published: Nov 29, 2024 Grab developed LLM-Kit to resolve standardization, security, observability, and infrastructure provisioning bottlenecks across its generative AI development efforts. Submitting an initial application request automatically triggers project generation in GitLab, outputting a modular codebase along with Terraform definitions for Amazon EKS and ECR. The scaffolded architecture incorporates FastAPI, LangChain, OpenID Connect authentication helpers, and PGVector alongside HashiCorp Vault for credential management. Observability and quality tracking are built in through direct integrations with Datadog and LangSmith evaluations. By providing these standardized paved-road templates, the framework has onboarded hundreds of generative AI applications and saved teams an estimated 1.5 weeks of initial setup time. ### [Metasense V2: Enhancing, improving and productionisation of LLM powered data governance](https://yomu.fyi/post/metasense-v2-enhancing-improving-and-productionisation-of-llm-powered.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Nick Buhrer - Published: Nov 14, 2024 Grab scaled Metasense to automate metadata generation, column-level classification, and sensitivity tiering across its entire data lake. Post-rollout analysis revealed that high tag volumes, wide tables, and mixed content—such as nested JSON and customer communications—strained model capacity and led to missed Personally Identifiable Information (PII). To overcome these limitations, the team split the classification process into distinct PII and non-PII tasks, reduced prompt word counts, and partitioned tables with over 150 columns into smaller units. Integrating LangChain and LangSmith modernized the architecture by enabling direct prompt experimentation, custom metric tracking, and version-controlled deployments. The updated pipeline achieved low misclassification rates alongside automated alert thresholds to trigger model improvement protocols if errors increase. ### [LLM-assisted vector similarity search](https://yomu.fyi/post/llm-assisted-vector-similarity-search.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Md Riyadh - Published: Oct 23, 2024 Vector similarity search often struggles with conceptual nuances, negations, and complex constraints despite its efficiency in high-dimensional semantic retrieval. To address this limitation, a two-step retrieval pipeline combines Facebook AI Similarity Search (FAISS) using OpenAI's text-embedding-ada-002 embeddings with a large language model re-ranking step powered by GPT-4o. The initial FAISS search extracts a candidate shortlist of matches, which GPT-4o then re-ranks based on contextual relevance and query intent. Evaluations on structured datasets showed that while simple queries yielded comparable results across methods, the LLM-assisted approach effectively resolved negations and conceptual matching where raw vector search failed. Internal production trials on relational datasets exceeding 4,500 rows demonstrated similar relevance improvements, with additional query latency being the primary operational trade-off. ### [Leveraging RAG-powered LLMs for analytical tasks](https://yomu.fyi/post/leveraging-rag-powered-llms-for-analytical-tasks.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Edmund Hong - Published: Oct 9, 2024 Data analysts at Grab faced an increasing volume of stakeholder data requests that required manually writing and executing repetitive SQL queries with minor parameter changes. To automate routine tasks like metric reporting and fraud investigations, the Integrity Analytics team paired an internal prompt-management platform, Spellvault, with Data-Arks, an in-house Python-based middleware that packages SQL queries and Python functions into APIs. A scheduler triggers automated report summarization, while an analytical bot named A\* bot executes relevant investigation queries via RAG and returns summarized findings in Slack. Automated report generation saves an estimated three to four hours per report and shortens multi-query fraud investigations to minutes. Grab favored RAG over fine-tuning because it avoids continuous retraining costs, scales faster across diverse use cases, and surfaces current production data. ### [Enabling conversational data discovery with LLMs at Grab](https://yomu.fyi/post/enabling-conversational-data-discovery-with-llms-at-grab.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shreyas Parbat - Published: Sep 26, 2024 Grab faced major bottlenecks in dataset discovery across its 200,000 data lake tables due to low documentation coverage and vanilla Elasticsearch limitations in its internal tool, Hubble. To resolve this, engineers first tuned Elasticsearch by boosting frequently queried P80 tables and certified schemas while deboosting deprecated datasets. They then deployed a GPT-4 pipeline that generated documentation from table schemas and sample data, raising P80 documentation coverage from 20% to roughly 90%. Finally, the team developed HubbleIQ, an LLM-powered assistant built via Glean Apps and integrated into Slack and Hubble search to answer semantic queries. These initiatives increased search click-through rates to 94% and lifted the proportion of users finding discovery easy to 73%. ### [Chimera Sandbox: A scalable experimentation and development platform for Notebook services](https://yomu.fyi/post/chimera-sandbox-a-scalable-experimentation-and-development-platform-fo.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Arkar Min Aung - Published: Aug 27, 2024 Grab's Chimera team built Chimera Sandbox to address resource accessibility challenges and enable rapid machine learning experimentation across technical and non-technical teams. The platform runs on multi-cluster Kubernetes infrastructure capable of scaling to thousands of nodes per cluster during peak loads. Users can provision customizable Docker environments with varying CPU and GPU configurations, collaborate in real time, and query data using Spark-on-Kubernetes integrations and helper magic cells. The workspace integrates with Grab's AI Gateway for LLM access alongside tools like MLflow, Optuna, RStudio, and LabelStudio on shared storage, backed by user-level cost attribution. ### [How we improved translation experience with cost efficiency](https://yomu.fyi/post/how-we-improved-translation-experience-with-cost-efficiency.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Jie Zhang - Published: Aug 5, 2024 Grab faced translation quality and cost challenges in its booking chat system when tourism resumed after COVID restrictions eased. Device language settings proved unreliable for determining message language, prompting the team to use Lingua alongside booking context heuristics and third-party fallbacks for real-time detection. To replace expensive and inaccurate general translation APIs, Grab distilled a large open-source Hugging Face model into lightweight, language-specific models trained on ten million synthetic chat examples. A post-translation validation step verifies non-translatable entities such as numbers and emojis before falling back to external services if mutations occur. Caching layers were also introduced across translation paths to curb redundant on-the-fly execution. ### [LLM-powered data classification for data entities at scale](https://yomu.fyi/post/llm-powered-data-classification-for-data-entities-at-scale.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Hualin Liu - Published: Jul 15, 2024 Grab needed to classify sensitive data at the table and column level across petabytes of database tables and streaming schemas. Manual schema-tiering campaigns had resulted in half of all schemas receiving overly strict Tier 1 access controls, while an initial automated service using regex patterns and third-party machine learning produced high false-positive rates and lacked customizability. To address this, the Caspian data engineering and governance teams enhanced their internal orchestration service, Gemini, by integrating GPT-3.5 via Azure OpenAI. Gemini aggregates classification requests into mini-batches, handles API rate limits, and uses prompt engineering—including few-shot examples, curated tag libraries, and explicit JSON DTO schemas—to reliably tag columns for data owner verification. [Newer posts](https://yomu.fyi/topic/llm/page/3.md)