---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [Innocent until combined: Blocking the lethal trifecta with Omnigent Contextual Policies](https://yomu.fyi/post/innocent-until-combined-blocking-the-lethal-trifecta-with-omnigent-con.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Nishith Sinha, Arun Pamulapati, Omar Khawaja
- Published: Aug 10, 2026
- Type: Problem & solution

An AI agent can exfiltrate private data when one session combines access to confidential information, attacker-controlled content, and an external communication channel, even though each capability is individually authorized. Omnigent's contextual policy tracks those three legs as session state: private data, untrusted content, and exfiltration, with human-defined tool or argument-based classifications. It allows ordinary actions until both prerequisite legs are lit, then denies the outbound call that would complete the trifecta. In the support assistant example, a user-supplied ticket instructs the agent to read internal Q3 revenue figures and email them externally; without the policy, the confidential number leaves in a reply, while the protected run blocks that email. A password-reset ticket still proceeds because it lights only the untrusted-content leg, and an empty internal lookup does not light the private-data leg.


### [Run Local Agentic AI Workflows with Meta’s Muse Glimmer on NVIDIA](https://yomu.fyi/post/run-local-agentic-ai-workflows-with-meta-s-muse-glimmer-on-nvidia.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 10, 2026
- Type: Announcement

Meta has released Muse Glimmer, a 30B open-weight dense model with a 120K+ context window aimed at local agentic AI workflows. The model uses a dense architecture that activates every parameter per token, avoiding routing overhead and targeting reliable instruction following, long-context coherence, and predictable latency for multi-step agent tasks. On NVIDIA Blackwell Ultra hardware, Muse Glimmer reports over 20K tokens/sec/GPU at BF16/NVF4 precision, with the full model fitting in a single GPU's VRAM. Deployment options span NVIDIA GeForce RTX 5090, DGX Spark, DGX Station, and Jetson edge devices, supported by vLLM, SGLang, and NVIDIA NIM containers. Fine-tuning is available through NVIDIA NeMo AutoModel for SFT and LoRA, with NeMo RL for reinforcement learning, and the NemoClaw harness enables long-running assistants in a sandboxed environment.


### [Making Knowledge Distillation Cheap Enough to Run at Scale](https://yomu.fyi/post/making-knowledge-distillation-cheap-enough-to-run-at-scale.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Antonio Tiene, Iker García-Ferrero, Ali Hashemi, Bakbergen Ryskulov
- Published: Aug 10, 2026
- Type: Problem & solution

Knowledge distillation trains a smaller student to match a larger teacher's capabilities, but conventional online training keeps both models loaded and materializes full vocabulary distributions at every step, making long-context recovery expensive. The proposed system caches the teacher's top-100 logits once and introduces a fused, chunked KL-divergence loss that avoids storing the full vocabulary-by-sequence grid or the student's full logits. In benchmarks with an 8K-token context on one H200, offline fused chunked KL used 58.3GB peak memory versus 102.8GB for online training, while all four methods reached near-identical training loss. At 32K tokens, the fully chunked loss reduced isolated kernel memory from 85.2 GiB to 5.45 GiB; distilling GPT-OSS 20B at that context cut step time from 57.0 to 12.23 seconds and reduced the setup from four GPU nodes to one.


### [Meta is back with Muse Glimmer: local, agentic, multimodal, and open source](https://yomu.fyi/post/meta-is-back-with-muse-glimmer-local-agentic-multimodal-and-open-sourc.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pedro Cuenca, merve, ben burtenshaw, Aritra Roy Gosthipaty
- Published: Aug 10, 2026
- Type: Announcement

Meta’s Muse Glimmer is a 30B-parameter multimodal model released under the Apache 2.0 license for local, agentic applications including coding, document analysis, and personal assistants. Its dense architecture combines a 2B ViT-style perception encoder with a 28B text decoder, while an optional DFlash speculative-decoding drafter targets faster generation for structured content. The decoder uses alternating sliding-window and full-attention layers, grouped-query attention, and query-key normalization with extra query scaling. Day-zero support is available in Transformers, llama.cpp, vLLM, Inference Endpoints, and other libraries, with examples for text, images, videos, deployment, and agent workflows. Published benchmarks show varied results: Muse Glimmer leads on MCP Atlas and SWE-Bench Pro, while Qwen3.6 leads on OSWorld-Verified and SWE-Bench Verified.


### [Unveiling good and bad behaviors on the Agentic Internet](https://yomu.fyi/post/unveiling-good-and-bad-behaviors-on-the-agentic-internet.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Jin-Hee Lee
- Published: Aug 7, 2026
- Type: Explainer

Web traffic increasingly shifts between human interaction and automated agents, rendering point-in-time security checks ineffective against sophisticated automation. Cloudflare addresses this shift through continuous behavioral analysis designed to evaluate trust across entire user sessions rather than relying on static point-in-time hurdles. Its CDN-injected client-side system, Precursor, tracks continuous behavioral indicators like cursor movements to catch mid-session automated takeovers. To counter rapid bot adaptation, Cloudflare is introducing self-adjusting machine learning models under Adaptive Intelligence and non-deterministic defenses such as the AI Labyrinth. These tools allow site owners to poison scraper training data, trap crawlers in synthetic webs, or queue legitimate agents while dynamically penalizing abusive traffic.


### [Introducing Radar Researcher: An AI tool for exploring Internet data in plain language](https://yomu.fyi/post/introducing-radar-researcher-an-ai-tool-for-exploring-internet-data-in.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Lai Yi Ohlsen
- Published: Aug 7, 2026
- Type: Announcement

Cloudflare launched the beta of Radar Researcher, an AI tool designed to let users explore global Internet traffic data using plain-language queries. Built entirely on Cloudflare's developer platform, the system uses the Cloudflare Agents SDK within a Cloudflare Worker, maintaining conversation state and SQLite databases inside Durable Objects. The inference layer relies on Workers AI running models like Kimi K2.7 through an ordered fallback chain across three model families to prevent outages, routed via AI Gateway for logging, caching, and guardrails. Radar's frontend renders retrieved data using its existing native interactive charts, while smaller dedicated models generate conversation titles and follow-up suggestions asynchronously. Additionally, Radar introduced support for the WebMCP standard, exposing imperative and declarative tools to allow external browser agents to navigate and interact with the platform directly.


### [Unifying Workers AI and AI Gateway into a single AI control plane](https://yomu.fyi/post/unifying-workers-ai-and-ai-gateway-into-a-single-ai-control-plane.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Michelle Chen
- Published: Aug 7, 2026
- Type: Announcement

Workers AI and AI Gateway are converging into a unified AI control plane to simplify model access, billing, and observability. Developers can now access both services through a single entrypoint using unified Workers bindings and the /ai/ REST endpoint. Specifying a default gateway automatically instruments requests with payload logging, token tracking, and cost attribution without manual dashboard setup. Cloudflare has also rolled out unified prepaid billing, allowing teams to spend AI Gateway credits on Workers AI models while unlocking higher rate limits. Planned future features include model-first routing for automated provider failover and smart routing powered by task classifiers on Workers AI.


### [Managing AI Coding Costs at Scale](https://yomu.fyi/post/managing-ai-coding-costs-at-scale.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Patrick Wendell, Akshat Bhatia, Vinay Gaba, Erich Elsen, Ivan Zhou
- Published: Aug 7, 2026
- Type: Explainer

AI coding tools have improved velocity metrics at Databricks, but broad deployment creates exponentially growing costs that can threaten the efficiency gains they deliver. The post presents a cost-management playbook aimed at preserving broad, low-friction access while keeping aggregate spending within a roughly fixed per-user envelope. It emphasizes the coding-model efficiency frontier, automated evaluations of internal workloads, model-flexible harnesses or meta-harnesses, and request- or task-level routing to the cheapest capable model. Databricks reports that Unity AI Gateway Smart Router reduced average task cost by more than 30% while roughly matching the most expensive model’s quality, and that harness and caching changes cut generated tokens and associated costs by almost 50% without observed quality degradation. The proposed AI Gateway centralizes model access, budget policies, tool configuration, and coding-session traces for ongoing optimization.


### [What is an AI Assistant?](https://yomu.fyi/post/what-is-an-ai-assistant.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Aug 7, 2026
- Type: Explainer

AI assistants are software systems that use language models, data retrieval, and reasoning to understand natural-language requests and return information or perform tasks. For enterprise data teams, they can generate SQL, build dashboards, troubleshoot pipelines, and automate repetitive work, but their usefulness depends on integration with data catalogs and governance rather than generic text generation alone. The described pipeline captures input, parses intent and context, retrieves knowledge, generates an output, and either delivers it for review or executes an action; advanced systems incorporate tool and API use plus feedback from corrections and approvals. The article categorizes assistants as voice, conversational, general-purpose, domain-specific, data and developer, or autonomous agents, while noting risks including hallucinations, privacy exposure, integration complexity, over-reliance, and bias. It presents assistants as augmenting human work through faster, more accessible, and more scalable decision support, not replacing human oversight.


### [What are Agentic Workflows?](https://yomu.fyi/post/what-are-agentic-workflows.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Aug 6, 2026
- Type: Explainer

Agentic workflows are AI-driven processes that let one or more agents plan, execute, and refine multi-step tasks toward a defined goal with minimal human intervention. Unlike chatbots and fixed automation, they evaluate context and intermediate results at runtime, gather data, select tools dynamically, and adapt when outcomes are incomplete, incorrect, ambiguous, or unexpected. The described pattern combines agents and LLMs with APIs, databases, code executors, prompts, feedback mechanisms, memory and state management, and, for complex work, coordinated specialized agents. Production reliability depends on clear success criteria, action guardrails, human review points for high-stakes actions, audit trails, governance, and production-grade infrastructure. Examples include customer-service resolution, fraud investigation, supply-chain optimization, and event-driven document processing across systems.


### [What is Tool Calling?](https://yomu.fyi/post/what-is-tool-calling.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Aug 6, 2026
- Type: Explainer

Tool calling lets an AI model interact with external tools, APIs, and systems to retrieve information or perform actions beyond its training data, turning a conversational model into an agent that can act. The process is an application-mediated loop: the model recognizes when outside help is needed, selects a tool from schema-described options, emits a structured request—typically JSON—and receives the result as context for its response or next step. Available tools can support information retrieval, code execution, process automation, and smart-device or IoT monitoring, while complex tasks may chain multiple calls into a workflow. The piece also distinguishes tool calling from function calling, describing the latter as a subset, and presents MCP as a standardization layer for describing, discovering, and invoking tools, with Databricks Agent Bricks offering governed integrations and agent-development support.


### [Introducing OfficeQA Pro V2: A New Benchmark for Enterprise Grounded-Reasoning](https://yomu.fyi/post/introducing-officeqa-pro-v2-a-new-benchmark-for-enterprise-grounded-re.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks AI Research Team
- Published: Aug 6, 2026
- Type: Benchmark

OfficeQA Pro V2 benchmarks whether AI agents generalize grounded reasoning to unfamiliar enterprise document collections rather than specialize in one corpus or task distribution. It contains 90 questions grounded in roughly 120,000 pages of U.S. Treasury accounting records spanning 1793–2024, including dense tables, charts, revised values, and changing reporting conventions. Questions were generated with asynth from seeded inputs, corpus evidence searches, traceable reasoning, and quality gates, then evaluated with deterministic exact match at 0.0% tolerance. Out-of-the-box frontier agents averaged 37.5% in one matched comparison, while competition systems averaged 41.1% and the winning team reached 63.3%; Databricks Genie raised matched-model accuracy by 24.0 percentage points on average, although failures remained in parsing, temporal reconciliation, and entity or category interpretation.


### [Kimi K3 from Moonshot AI is now available on Databricks through Unity AI Gateway](https://yomu.fyi/post/kimi-k3-from-moonshot-ai-is-now-available-on-databricks-through-unity.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: The Databricks Engineering Team
- Published: Aug 6, 2026
- Type: Announcement

Kimi K3 from Moonshot AI is now available on Databricks through the Foundation Model API and Unity AI Gateway, giving customers native access to an open-weight model alongside proprietary models from Anthropic, OpenAI, and Google Gemini. The announcement positions Kimi K3 as a frontier-quality option for coding, agentic reasoning, and document understanding, with deployment flexibility, customizability, and zero data retention coverage. Databricks says the model runs against governed enterprise data through Unity Catalog, while Unity AI Gateway provides permissions, safety guardrails, audit trails, spend controls, routing, and a standardized REST API within Databricks’ security perimeter. The post cites a score of 57 and rank of #4 in Artificial Analysis’ Intelligence Index, internal benchmarks showing performance at the level of leading proprietary models, and 50–72% lower cost per task; availability includes US hosting for AWS and GCP workspaces.


### [Cloudflare AI Search: give your agents a search engine for your data](https://yomu.fyi/post/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Nelson Duarte
- Published: Aug 6, 2026
- Type: Announcement

Cloudflare introduced developer experience improvements and a preview pricing model for Cloudflare AI Search. Previously, users had to manually connect separate Cloudflare primitives like Workers AI, AI Gateway, Vectorize, R2, and Browser Run to build a search solution. The platform now automates crawling, ingestion, embedding, and retrieval across structured and unstructured data sources. Developers can index sites without sitemaps using link discovery, expose public /search and /mcp endpoints across namespaces, bind search instances directly to Workers, or secure endpoints with Cloudflare Access. Cloudflare uses this system for developer documentation, its corporate blog, and the Cloudflare Dev Stack MCP server to supply coding agents with cited documentation.


### [The next generation of MCP](https://yomu.fyi/post/the-next-generation-of-mcp.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Matt Carey
- Published: Aug 6, 2026
- Type: Announcement

Model Context Protocol previously required stateful connections between clients and remote servers, creating operational complexity around sticky sessions, message replay, and open streams. The newly released MCP 2026-07-28 specification transitions MCP to a fully stateless protocol alongside updated SDKs across TypeScript, Python, Go, and C#. The protocol eliminates mandatory handshakes and session identifiers from core request paths while enabling servers to run on request-scoped infrastructure like Cloudflare Workers. Interactive elicitation is restructured through Multi Round-Trip Requests, which allow servers to return input requirements without sustaining persistent connections. Standardized headers for method names and improved OAuth authorization mechanisms further enable standard web infrastructure to inspect and manage MCP traffic efficiently.


### [From ranking to recommended: get your site ready to thrive in the age of AI agents](https://yomu.fyi/post/from-ranking-to-recommended-get-your-site-ready-to-thrive-in-the-age-o.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Matthew Conroy
- Published: Aug 6, 2026
- Type: Announcement

Discoverability is shifting from traditional search engine rankings to recommendations generated by AI agents acting on behalf of users. To address this shift, Cloudflare integrated Agent Readiness diagnostics and an Answer Engine Optimization tool into its dashboard. The Diagnostics feature evaluates hostnames against machine-readable standards across tiers ranging from robots.txt and Markdown rendering to OAuth discovery and agent payment protocols. Simultaneously, the Answer Engine Optimization tool measures how frequently leading assistants like Claude and GPT cite, mention, and prominently position brands across industry categories. By evaluating prompt responses using Cloudflare AI Gateway and Workers AI alongside access logs, site owners can systematically track agent traffic, benchmark against competitors, and remediate crawl blockers.


### [Building an open Agentic Internet: readable, discoverable, callable, and payable](https://yomu.fyi/post/building-an-open-agentic-internet-readable-discoverable-callable-and-p.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Jack Galilee
- Published: Aug 6, 2026
- Type: Announcement

The modern web produces billions of bot requests that repeatedly fetch unchanged pages, highlighting an architecture fundamentally misaligned with automated visitors acting on behalf of paying users. Cloudflare outlines an open Agentic Internet structured around four foundational capabilities: readability, discoverability, callability, and payability. Identity and access rely on open specifications such as Web Bot Auth and Private Access Control Tokens, while Markdown for Agents and the Kitesurf browser reduce token overhead and context window bloat. For execution and monetization, WebMCP exposes structured tools directly through the browser, and protocols like x402 paired with Monetization Gateway enable per-fetch transactions. These open standards allow domain owners to define tool contracts, verify bot identities, and monetize agent traffic without relying on legacy advertising models.


### [How we built an automated debugging workflow at Sentry](https://yomu.fyi/post/how-we-built-an-automated-debugging-workflow-at-sentry.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Dhrumil Parekh
- Published: Aug 6, 2026
- Type: Problem & solution

Sentry automated its internal debugging and issue triage workflows to handle bugs and pull requests generated by AI agents. The company deployed Seer, an AI-powered debugging agent, across several repositories to determine root causes, create fix plans, and open pull requests. To manage the resulting influx of pull requests, Sentry built an automated Slack routine powered by Claude that periodically scans for unaddressed pull requests opened within four hours. The routine asks Seer to identify relevant reviewers using GitHub commit history, verifies pull request status, and tags engineers to request merges, closures, or feedback. Early metrics show a 21 percent increase in pull request action rates, a 13 percent increase in 48-hour response rates, and a 12.5 percent rise in pull requests closed without merging due to duplicate or expanded fixes.


### [DeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and Coding](https://yomu.fyi/post/deepseek-v4-flash-0731-vs-gpt-5-6-luna-on-deepswe-cost-and-coding.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Aug 6, 2026
- Type: Benchmark

On the DeepSWE coding benchmark, GPT-5.6 Luna achieves 67.2% pass@1 versus DeepSeek-V4 Flash’s 53.3%, a 14-point gap. DeepSeek costs $0.10 per task, roughly six times less than Luna’s $0.61, delivering 532 solves per $100 versus Luna’s 110. A cascade that runs DeepSeek first and escalates only on failure solves 78.9% of tasks at $0.385 each, beating Luna’s solo accuracy and cutting cost by 37%. DeepSeek fails more gracefully (9% broken tests vs 15%) but is slower and dramatically weaker on JavaScript (35% vs 60%). Luna dominates seven of eight task domains, while DeepSeek leads only on query and config languages, illustrating how inexpensive models can combine with flagships for superior price-performance.


### [Baseten on Hugging Face Inference Providers 🔥](https://yomu.fyi/post/baseten-on-hugging-face-inference-providers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Alex Ker, Roland Crosby, Sid Shanker, Johan, Célina Hanouti, Simon Brandeis, Lucain Pouget, merve
- Published: Aug 6, 2026
- Type: Announcement

Baseten is now a supported Inference Provider on the Hugging Face Hub, extending serverless inference on model pages and through the Hugging Face Python and JavaScript SDKs. As part of the initial integration, it supports conversational and text-generation tasks, including access to open-weight models such as Kimi K3, latest DeepSeek V4 Flash, and GLM-5.2, with additional tasks planned to roll out soon. Developers can use a custom provider key for direct calls or authenticate with a Hugging Face token so requests are routed to Baseten; model pages and account settings support provider compatibility and preference ordering. Python and JavaScript OpenAI-compatible examples use the Hugging Face router, while integrated agent harnesses can access Baseten-hosted models without extra glue code.


[Newer posts](https://yomu.fyi/topic/llm/page/11.md) · [Older posts](https://yomu.fyi/topic/llm/page/13.md)