Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading post…
CloudflareFrank MeszarosAnnouncement
Identify AI model overuse with User Insights
Summary
Cloudflare AI Gateway’s User Insights now provides context beyond model names and request counts, using a dedicated Worker to classify conversations by task type such as coding, research, summarization, or formatting. The “model overkill” view identifies conversations where a high-capability reasoning model is used for simple tasks, showing which users or agents are driving that usage and the associated cost. Teams can analyze conversation turns to see how many back-and-forth exchanges a task requires, helping uncover inefficiencies in prompts or workflows. The Potential Savings view and Auto Router (closed beta) use task and model-fit signals to automatically route requests to appropriate models without a separate rule for each workload. These features are free for AI Gateway users and require stable user_id and session_id for analysis, with optional Access integration.
Context
Teams struggle to understand AI usage beyond token counts and request volumes, making it hard to identify overspending or model misuse. User Insights adds task classification, model overkill detection, turn analysis, and auto-routing to help teams optimize model selection and cost.
Approach / What changed
User Insights uses a dedicated Cloudflare Worker to classify AI Gateway conversation logs by task type (coding, research, summarization, etc.), task complexity, and model fit. It surfaces model overkill patterns (high-capability models used for simple tasks) and provides turn analysis, cost comparison, and user/agent insights. The Potential Savings view and Auto Router (closed beta) automatically apply these signals to route requests to appropriate models without per-workload rules.
Takeaways
User Insights classifies AI conversations into task categories (coding, research, summarization, etc.) using a dedicated Worker that examines conversation trajectory and returns a confidence score.
The “model overkill” view identifies conversations where a high-capability model is used for simple tasks, showing which users or agents are driving that usage and allowing cost comparison.
The Auto Router (closed beta) uses task complexity and model-fit signals to automatically route requests to an appropriate model, not always the cheapest, without requiring separate routing rules for each workload.
FAQ
How does User Insights classify AI traffic?
User Insights uses a dedicated Cloudflare Worker that processes AI Gateway logs. It examines the conversation trajectory including requests, responses, tool calls, and results to identify task type (e.g., coding, research, summarization) and returns a confidence score.
How do I configure User Insights for custom apps?
Custom applications must include stable user_id and session_id in the cf-aig-metadata header. Optionally, idp_group and application fields can be added. Cloudflare Access can automatically provide identity context for tools like Claude Code and Codex.
What is the Auto Router and how does it decide?
The Auto Router is a closed beta feature that uses conversation trajectory, task category, task complexity, and model-fit signals to select an appropriate model. It does not always choose the cheapest model but routes based on task needs, such as using capable reasoning models for complex coding work.
Source and freshness
- Source
- blog.cloudflare.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.