Fly.ioExplainer
Deploying unmodified UDP applications across an anycast edge and worker network is harder than proxying HTTP, because UDP carries no convenient metadata for forwarding the original socket addresses. The initial implementation used userland services, AF_PACKET sockets, and custom proxying, but the post argues that kernel-level packet handling is a better fit than adding and monitoring another service. It explains BPF’s evolution into eBPF, then describes using XDP for fast ingress processing, XDP_REDIRECT around WireGuard limitations, and TC BPF on the worker side; Firecracker tap devices make reply traffic appear as host ingress. The verifier requires bounds checks and debugging benefits from dropwatch and perf-based packet tracing. The result is a beta deployment path for UDP applications on Fly.io.
Unmodified UDP applications are difficult to proxy across edge and worker servers because, unlike HTTP, UDP does not provide convenient metadata for carrying the original source and destination socket addresses. A userland implementation also adds a service that must be monitored on every server.
The implementation uses kernel packet processing with eBPF, XDP, XDP_REDIRECT, and TC BPF. XDP handles ingress traffic, Firecracker tap devices allow VM replies to appear as host ingress, and TC BPF handles the worker side because XDP does not work directly on WireGuard. dropwatch and perf-based packet tracing support debugging.
The initial UDP implementation relied on AF_PACKET sockets and userland services to add and remove proxy metadata, but the post describes service overhead and operational monitoring as the main drawbacks.
XDP handles ingress packets but not ordinary transmitted packets; Firecracker tap devices provide a path for VM replies, while WireGuard’s lack of link-layer headers requires XDP_REDIRECT and TC BPF workarounds.
eBPF programs must satisfy the verifier’s bounds-safety checks, and packet snapshots sent through perf proved more useful for debugging than printk(); dropwatch also diagnosed many packet-handling bugs.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
BPF is not one hook but a set of attachment points, and each one sees different traffic. This diagram maps XDP, TC, socket-level BPF LSM, and sched_ext so you can pick the layer that can actually enforce your rule.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
