--- title: "Model Routing" description: "9 posts about Model Routing, summarised, each linking to the original." --- # Model Routing > 9 posts about Model Routing, summarised, each linking to the original. ## Articles ### [Smart Routing in Unity AI Gateway: Match frontier quality with 30%+ lower cost per task](https://yomu.fyi/post/smart-routing-in-unity-ai-gateway-match-frontier-quality-with-30-lower.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Ankit Mathur, Ivan Zhou, Bryan Qiu, Rohit Agrawal, Elise Gonzales, Kelly Albano - Published: Aug 13, 2026 Unity AI Gateway’s Smart Routing, now in Beta, addresses the cost and choice overload of assigning coding tasks to increasingly diverse models and harnesses. It uses task-aware routing: a small, low-latency model classifies the initial task description and metadata, then a policy selects a cheaper, medium, or frontier model; Omnigent can also choose the coding harness and route sub-agents. The design keeps consecutive turns on one model to preserve cache-hit rates, while escalating complex work and delegating simpler tasks to lower-cost options. On internal coding workloads, Smart Routing achieved 35% savings and outperformed any single model at 65% of Opus 5’s cost; public benchmarks reported matched Opus 5 performance with 56% savings. The team identifies mid-session reassessment, real-session evaluation, and affordable switching as continuing challenges. ### [Managing AI Coding Costs at Scale](https://yomu.fyi/post/managing-ai-coding-costs-at-scale.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Patrick Wendell, Akshat Bhatia, Vinay Gaba, Erich Elsen, Ivan Zhou - Published: Aug 7, 2026 AI coding tools have improved velocity metrics at Databricks, but broad deployment creates exponentially growing costs that can threaten the efficiency gains they deliver. The post presents a cost-management playbook aimed at preserving broad, low-friction access while keeping aggregate spending within a roughly fixed per-user envelope. It emphasizes the coding-model efficiency frontier, automated evaluations of internal workloads, model-flexible harnesses or meta-harnesses, and request- or task-level routing to the cheapest capable model. Databricks reports that Unity AI Gateway Smart Router reduced average task cost by more than 30% while roughly matching the most expensive model’s quality, and that harness and caching changes cut generated tokens and associated costs by almost 50% without observed quality degradation. The proposed AI Gateway centralizes model access, budget policies, tool configuration, and coding-session traces for ongoing optimization. ### [Kimi K3 from Moonshot AI is now available on Databricks through Unity AI Gateway](https://yomu.fyi/post/kimi-k3-from-moonshot-ai-is-now-available-on-databricks-through-unity.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: The Databricks Engineering Team - Published: Aug 6, 2026 Kimi K3 from Moonshot AI is now available on Databricks through the Foundation Model API and Unity AI Gateway, giving customers native access to an open-weight model alongside proprietary models from Anthropic, OpenAI, and Google Gemini. The announcement positions Kimi K3 as a frontier-quality option for coding, agentic reasoning, and document understanding, with deployment flexibility, customizability, and zero data retention coverage. Databricks says the model runs against governed enterprise data through Unity Catalog, while Unity AI Gateway provides permissions, safety guardrails, audit trails, spend controls, routing, and a standardized REST API within Databricks’ security perimeter. The post cites a score of 57 and rank of #4 in Artificial Analysis’ Intelligence Index, internal benchmarks showing performance at the level of leading proprietary models, and 50–72% lower cost per task; availability includes US hosting for AWS and GCP workspaces. ### [Unity AI Gateway is Generally Available](https://yomu.fyi/post/unity-ai-gateway-is-generally-available.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: David Nasi, Stefania Leone, Kevin Stumpf, Kasey Uhlenhuth, Kelly Albano - Published: Aug 4, 2026 Unity AI Gateway is generally available as a centralized way for enterprises to manage AI costs, security, governance, and model choice across agents and other AI assets. It positions Unity AI Gateway alongside Unity Catalog: Unity Catalog supplies identity, permissions, lineage, and auditing, while the gateway applies runtime guardrails and contextual policies across AI interactions. The gateway centralizes usage data in Unity Catalog, offering dashboards, Genie-powered analysis, granular cost attribution, budgets, hard spend caps, and a single-query API for frontier models from providers including Anthropic, OpenAI, Gemini, Kimi, and GLM. Smart Routing is in beta and dynamically selects models using quality, cost, performance, availability, and budget; the announcement says customers already use the gateway at scale, including more than one quadrillion tokens passing through it in the past year. ### [The New Monday Morning Report: How Generative AI can deliver the insights your executives need.](https://yomu.fyi/post/the-new-monday-morning-report-how-generative-ai-can-deliver-the-insigh.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Roberto Robles Nacif - Published: Aug 3, 2026 Retail and CPG joint planning meetings often spend time reconciling fragmented, stale, and inconsistent data instead of deciding how to adjust spend, inventory, or forecasts. The proposed Monday Morning Brief replaces the manually stitched weekly deck with an overnight agent-generated brief built from continuously refreshed point-of-sale, shipment, and inventory data alongside trade, promotion, forecast, and external signals. It uses Delta Sharing for a shared view, Genie Ontology for business context, Unity AI Gateway for permissions, guardrails, logging, rate limits, and human approval, and supports models across AWS, Azure, and Google Cloud. The agent scans item-store combinations, ranks material watchouts, answers cited natural-language follow-ups, and can draft actions such as purchase-order changes, while humans retain approval for consequential decisions. The proposed progression from report to ritual to intelligent decision system includes a 90-minute workshop, a day-30 pilot scope, and a first live Monday targeted for day 90. ### [Online Learning for Cost-Efficient LLM Routing](https://yomu.fyi/post/online-learning-for-cost-efficient-llm-routing.md) - Company: [Ramp](https://yomu.fyi/company/ramp.md) - Author: Kedar Thakkar - Published: Jul 20, 2026 Ramp describes an internal LLM gateway that processes trillions of tokens daily and uses dynamic failure-aware routing to reduce AI spend. It combines an EWMA of genuine provider-side failure rates with Thompson Sampling over a Normal-Inverse-Gamma posterior for the mean and variance of log-latency. For each request, the router estimates the probability of outright failure or missing the caller's deadline, incorporates model cost, and orders options while preserving caller-defined model preferences and fallbacks. In a reranker with a six-second deadline, the strategy reacted to Gemini 3.1 Flash Lite rate limiting by switching to GPT 5.4 Nano Flex, whose latency matched Standard at half the price. Larger experiments reported more than 25% cost savings without degradation, while a streaming-equivalent deployment for Ramp Inspect produced 30% savings. ### [You're Spending Too Much on AI. You're Also Using Too Little.](https://yomu.fyi/post/you-re-spending-too-much-on-ai-you-re-also-using-too-little.md) - Company: [Ramp](https://yomu.fyi/company/ramp.md) - Author: Anand Kuchibotla, Kedar Thakkar, Rahul Sengottuvelu - Published: Jun 17, 2026 The post argues that a large AI bill does not show excessive use: companies can overspend on routine work while using too little AI where advanced models could create value. It proposes measuring work in atomic tasks—such as invoices coded or pull requests reviewed—rather than tokens, with cost defined by tasks attempted and value by successful tasks. The operating model uses defaults that pair routine work with the cheapest model meeting quality benchmarks, medium reasoning, and flexible latency, while escalating ambiguous, high-stakes work to frontier models at higher effort. It also recommends attributing spend by provider, product, team, and workflow, finding concentrated costs, benchmarking repeated tasks, repricing after model releases, and centralizing controls in one gateway. The conclusion is that efficiency should be treated as an engineering achievement, so cheaper routine execution funds ambition on rare tasks where extra intelligence can materially change outcomes. ### [Building a Unified Pipeline for AI Token Spend](https://yomu.fyi/post/building-a-unified-pipeline-for-ai-token-spend.md) - Company: [Ramp](https://yomu.fyi/company/ramp.md) - Author: Kedar Thakkar, Richard Wang, Veeral Patel - Published: Apr 16, 2026 AI token spend is volatile, consumption-based, and distributed across teams, making usage and cost difficult to monitor. Ramp's pipeline ingests LiteLLM and OpenRouter events through authenticated, tenant-scoped webhooks, Kafka, and ClickHouse, then aggregates them for REST dashboards and AI-generated forecasts. LiteLLM uses callbacks with token counts and metadata, while OpenRouter sends OTLP traces; ReplacingMergeTree deduplicates replayed events using business\_id, source, and event\_id. Customers can connect gateways with an API key, configure LiteLLM callbacks or OpenRouter Broadcast, and pass attribution metadata. The resulting visibility supports breakdowns by provider, model, department, user, project, and use case, and exposed phantom Gemini reasoning tokens after a LiteLLM upgrade, geo pricing premiums, runaway loops, and prompt bloat. ### [Agent Bricks: The governed enterprise agent platform](https://yomu.fyi/post/agent-bricks-the-governed-enterprise-agent-platform.md) - Company: [Databricks](https://yomu.fyi/company/databricks.md) - Author: Kasey Uhlenhuth - Published: Apr 14, 2026 Agent Bricks is presented as Databricks’ enterprise platform for building, deploying, and governing agents that operate on business data under real identities, permissions, and operational constraints. The platform combines multi-model and framework support, execution, routing, fallback, cost optimization, and unified governance through Unity Catalog and AI Gateway, including on-behalf-of token passing and observability across data, models, MCPs, and APIs. Its context layer uses metadata such as schemas, business definitions, lineage, permissions, and data-quality signals, while Genie Spaces, Document Intelligence, Knowledge Assistant, and Agent Mode address structured and unstructured business information. The announcement includes general availability for Document Intelligence, Custom Agents on Apps, and Supervisor Agent, plus AI Gateway guardrails, managed OAuth MCP Connectors, web search, and MLflow’s CLEARS evaluation framework; the post reports 70% higher accuracy than standard RAG and a 30% improvement in multi-step workflows.