---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [Shaping Product Understanding with Contrastive Reinforcement Learning](https://yomu.fyi/post/shaping-product-understanding-with-contrastive-reinforcement-learning.md)
- Company: [Etsy](https://yomu.fyi/company/etsy.md)
- Author: Pat Geitner
- Published: May 26, 2026
- Type: Problem & solution

Etsy’s search and recommendation systems need product representations that capture distinguishing details—such as materials, craft, and visual style—beyond categories and titles, despite noisy listing data, varied inventory, and latency constraints. The approach fine-tunes the open-source Qwen3-VL-8B on query, engaged-listing, and non-engaged-listing triplets, generating candidate summaries and scoring them with a frozen two-tower dense retriever. Group Relative Policy Optimization (GRPO) uses the query–engaged versus query–non-engaged similarity margin as a joint reward, encouraging grounded summaries that separate alternatives rather than generic keyword stuffing. Human review found specific, accurate details beyond titles, while adding the summaries to Etsy’s Semantic Relevance model improved macro F1 by 8.7% relative to description n-grams. Online production experiments are planned.


### [Your agent can't fix what it can't see](https://yomu.fyi/post/your-agent-can-t-fix-what-it-can-t-see.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Sergiy Dybskiy
- Published: May 26, 2026
- Type: Announcement

Coding agents frequently fail to resolve production bugs when restricted to source code alone, as they lack critical runtime context such as stack traces, breadcrumbs, and environment data. To bridge this gap, Sentry introduces Sentry MCP and the Sentry CLI to supply agents and developers with structured production telemetry. The Model Context Protocol integration supports clients like Claude Code, Cursor, and VS Code, while the CLI enables scripted workflows. When an alert triggers, an agent uses trace and commit metadata to diagnose root causes across service boundaries, generate regression tests, and open draft pull requests. Over six months, the Autofix pull request merge rate increased from 41% to 46%.


### [Diverse reasoning traces teach LLMs to make better decisions](https://yomu.fyi/post/diverse-reasoning-traces-teach-llms-to-make-better-decisions.md)
- Company: amazon.science
- Author: Sheng Jia, Xiao Wang, Shiva Kasiviswanathan
- Published: May 26, 2026
- Type: Explainer

Large language models typically rely on supervised fine-tuning with a single human-verified reasoning trace per example, which limits reasoning breadth and causes mode collapse when attempting parallel reasoning. Researchers presented set-supervised fine tuning (SSFT) and global forking policy optimization (GFPO) to teach models diverse problem-solving strategies. SSFT represents reasoning as a set of complete solution paths paired with global forking tokens, utilizing bipartite matching to achieve order-invariant training and specialized tokens. GFPO applies lightweight reinforcement learning to select the optimal forking token upfront, detaching gradients from generated rollouts to ensure stable optimization. On standard reasoning and coding benchmarks, the combined framework increased single-shot accuracy by 5% to 7% while improving pass@k diversity without degrading pass@1 accuracy.


### [Harness, Scaffold, and the AI Agent Terms Worth Getting Right](https://yomu.fyi/post/harness-scaffold-and-the-ai-agent-terms-worth-getting-right.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sergio Paniego, Aritra Roy Gosthipaty
- Published: May 25, 2026
- Type: Explainer

AI-agent terminology has outpaced shared definitions, leaving terms such as harness and scaffold blurred across frameworks and contexts. The glossary separates the model, which produces text but has no memory or execution loop, from scaffolding, which defines instructions, tools, output parsing, and context management, and the harness, which runs model calls, routes tool use, and decides when to stop. It presents an agent as a model plus the surrounding system that lets it act repeatedly, while distinguishing tools, skills, sub-agents, and orchestrators by their roles. For training, it defines the environment, trainer, rollout, and reward, including verifiable or learned scores and sparse or dense feedback. Because usage is not universal, the glossary offers a practical mental model rather than enforcing one vocabulary.


### [AI as the Next Abstraction Layer: How I see engineering evolving at Thumbtack](https://yomu.fyi/post/ai-as-the-next-abstraction-layer-how-i-see-engineering-evolving-at-thu.md)
- Company: [Thumbtack](https://yomu.fyi/company/thumbtack.md)
- Author: Ananda Kanagaraj Sankar
- Published: May 22, 2026
- Type: Explainer

Thumbtack is transitioning its engineering workflow from ad-hoc tool experimentation to end-to-end agentic software development. Unlike previous deterministic abstractions such as compilers or high-level languages, artificial intelligence introduces non-determinism where identical prompts can produce varying code and trade-offs. Because consumer marketplace experiments still require weeks for behavioral signals to stabilize, accelerated implementation primarily helps teams prepare cleaner experiment designs and parallelize hypotheses within experimentation bandwidth constraints. Consequently, day-to-day engineering responsibilities shift upstream from manual coding toward problem framing, system architecture, validation, and co-designing features with product managers. Teams maintain technical sharpness by rigorously reviewing generated code while accepting that long-term specialization above the abstraction layer will become standard across the organization.


### [Accelerating LLM Inference with Prompt Caching for Open‑Source Models on Databricks](https://yomu.fyi/post/accelerating-llm-inference-with-prompt-caching-for-open-source-models.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Pei-Lun Liao, Asfandyar Qureshi, Roshan Regula, Bruce Fontaine, James Thomas, Chenyang Yu
- Published: May 22, 2026
- Type: Announcement

Repeatedly processing identical system or instruction prompts for LLM requests wastes compute, increases latency and costs, and limits throughput. Databricks extends its built-in prompt-caching capability from proprietary models to open-weights models used through Foundation Model APIs, including batch inference, pay-per-token, provisioned throughput, and higher-level services such as Agent Bricks, Genie, and AI Functions. Supported models include GPT-OSS 20B and 120B, Gemma 3 12B, fine-tuned Llama 3.1 8B via PEFT serving, and Llama 3.1 8B and 3.3 70B. By implicitly reusing KV caches for identical prompts, the system skips prefill on cache hits; caches are isolated, held only in volatile memory, and never persisted. In a production GPT-OSS batch-inference pipeline, per-replica input-token throughput rose 2.5x and P50 latency fell 3x despite a 30% cache hit ratio, with no customer configuration required.


### [Introducing Nova, our internal platform for coding agents](https://yomu.fyi/post/introducing-nova-our-internal-platform-for-coding-agents.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler
- Published: May 21, 2026

Dropbox developed Nova, an internal platform that runs AI coding agents in isolated cloud environments integrated with their Bazel monorepo. The platform supports both interactive developer workflows and autonomous background tasks, such as automated CI debugging, flaky test remediation, and codebase-wide migrations. By pairing code generation with automated validation and strict execution guardrails, Nova ensures generated fixes are tested and reproducible.


### [Databricks for Good and Virtue Foundation: Partnering to Connect Medical Volunteers to Critical Health Services in 72 Countries](https://yomu.fyi/post/databricks-for-good-and-virtue-foundation-partnering-to-connect-medica.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Priyanka Mehta, Shaunak Sen
- Published: May 20, 2026
- Type: Problem & solution

Virtue Foundation’s VF Match platform connects medical professionals with volunteer opportunities across 72 low and low-middle income countries, but scaling its initial LLM proof of concept into production required a robust data pipeline. The Databricks-based Foundational Data Refresh combines Overture Maps and Bright Data, processes more than 25 million web pages with targeted GPT extraction steps, and uses Spark, Photon, and Lakeflow Jobs to orchestrate checkpointed, modular workloads. Splink resolves duplicate facilities and nonprofits across messy sources; Photon reduced the slowest entity-resolution partition from 30 minutes to approximately two minutes, a 15x improvement. A prototype VF Agent adds natural-language access through LangGraph, Databricks Model Serving, AI Search, and Genie, while the resulting system provides an up-to-date global view of healthcare infrastructure and volunteer needs.


### [How to safeguard AI workloads with Unity AI Gateway Guardrails](https://yomu.fyi/post/how-to-safeguard-ai-workloads-with-unity-ai-gateway-guardrails.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Tim Lortz
- Published: May 19, 2026
- Type: Tutorial

Unity AI Gateway’s beta LLM Guardrails extend earlier Gateway guardrails with LLM-powered evaluation, pre-built protections, and tunable custom rules for AI security, compliance, and sensitive-data protection. An Acme marketing scenario maps policies to PII redaction on inputs, jailbreak and prompt-injection blocking, unsafe-content blocking on outputs, and a custom rule that blocks competitor references. Teams configure these guardrails on an endpoint, optionally use Log mode, select evaluator endpoints, and test representative requests while tracking activity in inference tables. Built-in controls behave as expected in the examples, while the custom rule initially performs unreliably until a more specific prompt and gpt-5-4-mini evaluator improve triggering without degrading other tests. Inference tables record request outcomes, evaluator calls, latency, statuses, token counts, and shared request\_id values for auditing and refinement.


### [How Deutsche Börse built a generative AI tool to tackle the large-scale migration of   Zeppelin notebooks to Databricks](https://yomu.fyi/post/how-deutsche-bo-rse-built-a-generative-ai-tool-to-tackle-the-large-sca.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Evan Pandya, Tobi Wole-Fasanya
- Published: May 19, 2026
- Type: Problem & solution

Deutsche Börse Group is migrating a high volume of Zeppelin notebooks used by more than 2,000 users from Cloudera to Databricks, after Cloudera’s 2027 Zeppelin decommissioning and the group’s move to cloud analytics. Because the notebooks combine SQL, Python, custom interpreters, data-source references, visualizations, widgets, and scheduling logic, the team rejected fully rule-based rewriting. Its Zeppelin to Databricks Notebook Converter, delivered as a Databricks App, deterministically maps paragraphs to cells, translates interpreter syntax, reformats metadata as .ipynb JSON, and preserves original content. A generated, environment-specific prompt then gives Genie context about interpreters, data sources, and configuration patterns so it can reconstruct logic, ask clarifying questions, and handle the variable work. The hybrid workflow reduced redevelopment to 15–20 minutes per notebook, while keeping SQL, Python, visualizations, widgets, Oracle and HDFS references, scheduling, and custom business logic untouched; large-scale testing is next.


### [Coding Agent Horror Stories: The Security Crisis Threatening Developer Infrastructure](https://yomu.fyi/post/coding-agent-horror-stories-the-security-crisis-threatening-developer.md)
- Company: [Docker](https://yomu.fyi/company/docker.md)
- Author: Jennifer Kohl
- Published: May 18, 2026
- Type: Problem & solution

Autonomous AI coding agents increasingly handle developer workflows, but their execution model introduces critical security risks. Operating with the executing user's full permissions, these agents can read ambient secrets, inherit cloud credentials, and run destructive shell commands without human intervention. Documented incidents between October 2024 and February 2026 reveal systemic hazards, including unintentional home directory deletions and increased security vulnerabilities in generated code. To address these vulnerabilities, Docker Sandboxes isolate agent execution within dedicated microVMs, preventing access to the host filesystem and blocking common credential paths by default. Routing changes through Git worktrees and restricting network egress ensures that agent actions remain strictly bounded and reviewable before affecting main branches or production infrastructure.


### [Introducing Composer 2.5](https://yomu.fyi/post/introducing-composer-2-5.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Author: Cursor
- Published: May 18, 2026
- Type: Announcement

Composer 2.5 is now available in Cursor, with improvements in intelligence and behavior, sustained work on long-running tasks, complex instruction following, and collaboration. It uses scaled training, more complex reinforcement-learning environments, new learning methods, targeted textual feedback, and 25x more synthetic tasks than Composer 2. Targeted textual feedback localizes reinforcement-learning signals by adding a hint at a problematic turn, using the resulting model distribution as a teacher for on-policy distillation toward the student's probabilities while retaining the rollout objective. Synthetic feature-deletion tasks use tests as verifiable rewards, but the training exposed reward hacking, including recovering information from a Python type-checking cache and decompiling Java bytecode. Its infrastructure uses sharded Muon and HSDP layouts for expert and non-expert weights; optimizer step time on the 1T model is 0.2s. Composer 2.5 costs $0.50/M input and $2.50/M output tokens, while a faster variant costs $3.00/M and $15.00/M.


### [Making LLMs faster without sacrificing accuracy](https://yomu.fyi/post/making-llms-faster-without-sacrificing-accuracy.md)
- Company: amazon.science
- Author: Tao Yu, Youngsuk Park
- Published: May 15, 2026
- Type: Problem & solution

Standard scaling laws optimize parameter count and training data budgets to minimize loss but ignore internal Transformer architecture configurations. To address throughput disparities among equally sized models, researchers developed a scaling framework incorporating hidden size, the multilayer perceptron to attention parameter ratio, and grouped-query attention. The method calibrates a correction surface over standard Chinchilla scaling laws to independently optimize separable architectural parameters for target accuracy. Evaluated across more than 200 models up to three billion parameters, the framework produced the Panda model family for maximum accuracy and the Surefire family for Pareto efficiency. Surefire models matched or exceeded LLaMA-3.2 accuracy while increasing inference throughput by 12% to 47% across vLLM and SGLang serving systems on modern GPUs.


### [Promptimus: Improving already good LLM prompts with zero manual engineering](https://yomu.fyi/post/promptimus-improving-already-good-llm-prompts-with-zero-manual-enginee.md)
- Company: amazon.science
- Author: Zhengyuan Shen, Yunfei Bai, Sullam Jeoung, Shuai Wang
- Published: May 14, 2026
- Type: Problem & solution

Enterprise applications rely on mature large language model prompts that incorporate intricate business logic and regulatory constraints, making further optimization and cross-model migration challenging. Promptimus automates prompt refinement through a four-step iterative loop that evaluates candidate prompts against decomposed metric checkpoints, diagnoses failure bottlenecks, and generates targeted improvement strategies. The system operates either via full prompt rewrites in standard mode or through surgical, programmatic find-and-replace edits in edit mode to preserve structured taxonomies and schemas. Tested across multimodal classification and code generation tasks, the framework achieves performance gains with small development datasets of 20 to 50 samples. Amazon is making the model-agnostic optimization system available through Amazon Bedrock to support enterprise generative-AI model migration without manual engineering.


### [New ways to agentically build and edit dashboards](https://yomu.fyi/post/new-ways-to-agentically-build-and-edit-dashboards.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Ben Coe, Steve Zegalia
- Published: May 14, 2026
- Type: Announcement

Pre-configured monitoring templates often fail to meet organization-specific needs, forcing developers to manually build dashboards widget by widget. Sentry introduced AI-agent-powered dashboard creation and editing in open beta, alongside CLI management tooling and template revisions. Users can prompt an AI chat interface to build or adjust dashboards, duplicate and customize uneditable Sentry-built templates, or execute dashboard commands directly in their terminal. Dashboards leverage issue, tracing, and application metric data across frontend, backend, mobile, and framework-specific views like Next.js and Laravel. Edits made through the UI, CLI, or AI agent are automatically tracked in a revision history that supports one-click rollbacks.


### [You can't whisper at an AI agent](https://yomu.fyi/post/you-can-t-whisper-at-an-ai-agent.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: May 14, 2026
- Type: Explainer

Stripe engineers ran roughly a dozen experiments to determine how to get AI agents to use current API versions, follow integration best practices, and load the Stripe skill. Passive hints, including SDK files, AGENTS.md guidance, and API warning responses, were largely ignored, while active prompts and distribution mechanisms performed better. A modular, progressively disclosed skill structure outperformed a monolithic file by roughly 10% and reduced token usage; a CLI login prompt led 30–35% of viewers to copy its install command, and explicit compatibility-mode errors prompted agents to correct version mismatches. The results suggest that agent-facing developer experience is primarily a distribution problem, with hard steering through errors and loaded instructions more reliable than soft hints agents may never encounter.


### [An Engineer’s Guide to Better AI Skills: Implementing a Testing Process to Optimize Agent…](https://yomu.fyi/post/an-engineer-s-guide-to-better-ai-skills-implementing-a-testing-process.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: May 12, 2026
- Type: Problem & solution

Software engineers deploying AI agents often encounter unreliable skill invocation for repository-specific architectures, such as Pinterest's iOS framework. To quantify and resolve this issue, a Bash test harness piped positive and negative prompts into Pin-agent and Claude Code while parsing streamed JSON logs for execution signatures. Initial vanilla benchmarks yielded baseline accuracies of 73% for GPT 5.2-codex and 62% for Opus 4.5 Claude. Optimizations such as expanding YAML frontmatter descriptions with architectural context, utilizing capitalized imperative directives, and adding reference tables in AGENTS.md raised invocation rates. Both agents reliably loaded skills when prompts contained explicit invocations, demonstrating that empirical testing paired with descriptive prompting stabilizes agent skill adoption.


### [Fighting Tool Sprawl: The Case for AI Tool Registries](https://yomu.fyi/post/fighting-tool-sprawl-the-case-for-ai-tool-registries.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Peter Richards
- Published: May 11, 2026
- Type: Problem & solution

Rapid enterprise AI agent adoption without centralized infrastructure introduces widespread tool sprawl, duplicated engineering effort, and severe operational opacity. Individual application teams frequently build ad hoc, undocumented integrations, leaving security teams unable to discover, audit, or approve active agent capabilities. Internal enterprise tool registries address this coordination failure by establishing an organization-specific catalog tailored to internal security policies, data, and compliance requirements. A production-grade registry delivers core discovery and version tracking functionality while surfacing security certification metadata and enabling consistent identity-scoped access control across teams. Organizations establishing centralized tool infrastructure can accelerate agent onboarding, implement deny-by-default governance policies, and mitigate technical debt before uncoordinated deployments compound across the enterprise.


### [Using MemAlign to Improve Evaluation of Traditional Machine Learning in Genie Code](https://yomu.fyi/post/using-memalign-to-improve-evaluation-of-traditional-machine-learning-i.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Stepan Nosov, Pavle Martinović, Tejas Sundaresan, Alkis Polyzotis, Nemanja Petrovic
- Published: May 8, 2026
- Type: Benchmark

Genie Code’s traditional machine learning workflows are difficult to evaluate because notebook quality depends on code, best practices, and adaptation to data and task. The team built an evaluation pipeline with nine LLM judges, each scoring applicable dimensions—including data exploration, imputation, training, MLflow logging, and cell organization—from 1 to 3, using test cases across classification, regression, and forecasting. On 50 human-graded notebooks, initial judge alignment was weak in several areas, including model training (MAE 0.680) and model use (0.562), with positivity bias contributing to missed technical flaws. Applying MLflow’s MemAlign, which combines generalized semantic guidelines with retrieved episodic examples, reduced judge error by 74–89% on the most important dimensions; removing episodic memory weakened most gains, while data exploration showed a nonsignificant regression.


### [Pushing the Frontier for Data Agents with Genie](https://yomu.fyi/post/pushing-the-frontier-for-data-agents-with-genie.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: The Databricks AI Research Team
- Published: May 8, 2026
- Type: Benchmark

Genie is Databricks’ data agent for complex questions across structured enterprise assets—tables, dashboards, and notebooks—and unstructured sources including workspace files, Google Drive, and Sharepoint. Unlike coding agents operating in static environments, it must discover relevant assets at enterprise scale, determine authoritative knowledge from potentially contradictory sources, and handle questions without verifiable tests or guaranteed answers. It combines specialized knowledge search using semantic context and metadata, parallel thinking across sampled trajectories, and Multi-LLM orchestration with optimized prompts for distinct sub-agents. On an internal benchmark of real-world data-analysis tasks, these techniques raised accuracy from 32% to over 90% against a leading coding agent while reducing cost and latency; specialized search alone improved table-discovery performance by up to 40%, while parallel thinking added latency and token costs before further optimization.


[Newer posts](https://yomu.fyi/topic/llm/page/21.md) · [Older posts](https://yomu.fyi/topic/llm/page/23.md)