Model Routing Is Simple. Until It Isn’t.

Summary

Model routing in agentic systems is presented as a systems optimization problem rather than a simple classification task. On 417 AppWorld Test Challenge tasks with the same CodeAct agent, Claude Sonnet 4.6 cost $79 total ($0.19 per task), versus $155 ($0.37 per task) for GPT-4.1, despite GPT-4.1 having lower listed token prices; cache-read pricing and repeated context changed the effective result. The discussion also shows why task difficulty and model speed are incomplete signals: hidden execution work, specialization, governance constraints, routing overhead, cache state, hardware, and endpoint load all affect decisions. The described router optimizes cost, quality, and latency together, producing configurable cost-accuracy operating points; one latency-oriented configuration reached 84% accuracy for $93 and 83 seconds, while adding about 6 ms and 2 kB per task.

Takeaways

1. 1

   Across 417 AppWorld Test Challenge tasks using the same CodeAct agent, Claude Sonnet 4.6 cost $79 total, while GPT-4.1 cost $155, because Sonnet benefited more from lower cache-read pricing despite higher listed token prices and longer trajectories.

2. 2

   Task difficulty is often unknown at routing time because apparently simple requests can trigger retrieval, compliance checks, tool use, and refinement, while technical prompts may be handled efficiently by smaller specialized models.

3. 3

   A latency-optimized router configuration achieved 84% accuracy for $93 and 83 seconds, reducing cost by 21% and latency by 9% versus running Opus alone, with a 4% accuracy drop.