---
title: "Introducing uFowarder: The Consumer Proxy for Kafka Async Queuing"
description: "Uber introduces uForwarder, an open-source Consumer Proxy for Kafka Async Queue workloads, after deploying a push-based system across more than 1,000 consumer services. The proxy fetches messages through Kafka’s binary protocol, sends them individually to consumer services over gRPC, collects status codes and processing results, and commits offsets when safe. Production challenges included head-of-line blocking from messages that fail before reaching handlers, growing hardware costs, external data sources that are not ready when events arrive, and requirements for production/non-production and zone isolation. The implemented solutions include context-aware routing through message and gRPC headers, active blocking detection and mitigation, hardware-efficiency improvements, and DelayProcessManager, which pauses only delayed partitions and guarantees at-least delay rather than exact timing. The project is open source, with planned work on consumer-lag handling and direct Protobuf support."
---

# Introducing uFowarder: The Consumer Proxy for Kafka Async Queuing

[Uber](https://yomu.fyi/company/uber) · Jul 2, 2026

**Type:** Explainer

## Summary

Uber introduces uForwarder, an open-source Consumer Proxy for Kafka Async Queue workloads, after deploying a push-based system across more than 1,000 consumer services. The proxy fetches messages through Kafka’s binary protocol, sends them individually to consumer services over gRPC, collects status codes and processing results, and commits offsets when safe. Production challenges included head-of-line blocking from messages that fail before reaching handlers, growing hardware costs, external data sources that are not ready when events arrive, and requirements for production/non-production and zone isolation. The implemented solutions include context-aware routing through message and gRPC headers, active blocking detection and mitigation, hardware-efficiency improvements, and DelayProcessManager, which pauses only delayed partitions and guarantees at-least delay rather than exact timing. The project is open source, with planned work on consumer-lag handling and direct Protobuf support.

## Context

uForwarder was developed to address partition scalability, head-of-line blocking, programming-language support, hardware efficiency, delayed upstream data, and message-isolation requirements that emerged as more services adopted the Consumer Proxy.

## Approach / What changed

The Consumer Proxy fetches Kafka messages using Kafka’s binary protocol, dispatches them individually over gRPC, aggregates processing results, and commits offsets when safe. uForwarder adds context-aware routing, active head-of-line blocking detection and mitigation, hardware-efficiency improvements, and DelayProcessManager for partition-level delayed delivery.

## Takeaways

- The Consumer Proxy hides Kafka consumer-management complexity behind a gRPC interface and commits offsets after aggregating consumer-service processing results.
- Context-aware routing carries subsetting context from Kafka message headers into gRPC request headers, allowing load balancers to isolate production/non-production and zone traffic without splitting topics.
- DelayProcessManager pauses only partitions whose first unprocessed message has not met its delay, buffers polled messages, and provides an at-least delay rather than an exact delay.

**Tags:** [Distributed Systems](https://yomu.fyi/topic/distributed-systems), [gRPC](https://yomu.fyi/topic/grpc), [Kafka](https://yomu.fyi/topic/kafka), [Open Source](https://yomu.fyi/topic/open-source)

- Source: [Uber](https://www.uber.com/us/en/blog/introducing-ufowarder)
- Source URL: https://www.uber.com/us/en/blog/introducing-ufowarder
- Ingested by Yomu: 2026-09-11T12:01:21.516Z

[Read original post](https://www.uber.com/us/en/blog/introducing-ufowarder)