Langfuse, the open-source AI observability platform acquired by ClickHouse in January 2026, launches the AI Gateway in public beta. The gateway serves as a central entry point to govern all AI traffic, logging every model call natively in Langfuse for cost tracking and analysis. It is built in Rust, adds under 3ms median latency with tracing on, and supports 1,000+ concurrent streams on a single 2 vCPU node. The gateway is MIT-licensed, deploys as a single 22MB Docker image, and requires only changing the base URL and using a gateway key. OpenAI and Anthropic are supported in beta, with more providers planned.
Context
Langfuse, the open-source AI observability platform acquired by ClickHouse in January 2026, launches the AI Gateway in public beta to address the challenge of tracking AI usage across multiple teams, tools, and providers within a company.
Approach / What changed
Langfuse AI Gateway is a central entry point that logs every model call natively in Langfuse, turning each call into a full trace. It is built in Rust, deploys as a single 22MB Docker image, and requires only changing the base URL and using a gateway key. The gateway supports OpenAI and Anthropic in beta, with more providers planned.
Takeaways
1
Langfuse AI Gateway turns every model call into a full Langfuse trace, enabling search, evaluation, and improvement beyond simple cost accounting.
2
The gateway is built in Rust, adds under 3ms median latency with tracing on, and supports 1,000+ concurrent streams on a single 2 vCPU node.
3
By default, the gateway records only metadata, usage, and cost; full prompt and response logging is opt-in, and pass-through mode records nothing.
FAQ
Is Langfuse AI Gateway open source?
Yes. The gateway is MIT-licensed and runs on Langfuse Cloud and self-hosted deployments.
Do I need to change my code?
No. Change the base URL and use a gateway key. Requests and responses stay in the provider's native format.
Does Langfuse record my prompts?
Only if you opt in. By default, the gateway records metadata, usage, and cost, not content. Pass-through mode records nothing.
An AI runtime is the execution layer between model code and hardware: it feeds data to accelerators, places and reshards tensors, schedules work, and holds latency steady. Here are its four core jobs.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.