Loading…
How GitHub uses eBPF to improve deployment safety
GithubLawrence Gripper
Summary
Deployment scripts can introduce dangerous circular dependencies when they rely on services or assets from platforms that are currently experiencing outages. Blocking network access at the host level is impractical because stateful nodes continue serving live traffic during rolling deployments. To solve this, GitHub isolates deploy scripts into dedicated Linux cGroups and attaches custom eBPF programs via the cilium/ebpf Go library. The system uses socket-address hooks to redirect DNS queries to a userspace proxy that checks a domain blocklist, while egress packet hooks map DNS transaction IDs to process IDs. This approach successfully prevents deploy-time circular dependencies, provides full command-line audit logs for blocked requests, and speeds up incident recovery.
Context
Deployment scripts executing on stateful production nodes risked failing during incidents due to circular dependencies on GitHub, but completely blocking host-level network access would break live customer traffic.
Approach / What changed
GitHub isolated deployment scripts within Linux cGroups and applied eBPF programs (BPF_PROG_TYPE_CGROUP_SOCK_ADDR and BPF_PROG_TYPE_CGROUP_SKB) alongside a userspace DNS proxy to intercept, filter, and audit outbound domain requests per process.
Takeaways
- Using BPF_PROG_TYPE_CGROUP_SOCK_ADDR enables rewriting connect4 syscalls on port 53 to route cGroup DNS queries to a local userspace proxy.
- Correlating skb_buff DNS transaction IDs with process IDs in eBPF maps allows looking up /proc/{PID}/cmdline to identify the exact command triggering a blocked request.
- Placing deploy scripts in dedicated cGroups allows applying network egress blocklists without impacting the host's primary production traffic.
Related reading
Github ·
From latency to instant: Modernizing GitHub Issues navigation performance
GitHub Issues addressed navigation latency by shifting workloads to the client using a local-first, stale-while-revalidate architecture. To evaluate perceived delays, the team tracked user transitions through Highest Priority Content thresholds, aiming for instant rendering in under 200 milliseconds. The core implementation added a persistent client-side cache using IndexedDB, paired with a synchronous in-memory tier to serve hot issue payloads without asynchronous overhead. A selective preheating mechanism resolves missing cache entries for high-intent links without overburdening backend capacity with redundant requests. Following broad rollout, the proportion of instant React soft navigations increased from 4% to approximately 22%, yielding an overall cache-hit ratio of around 33%.
Natalie GuevaraGithub ·
The cost of saying yes has changed
Generating initial code patches has become significantly cheaper with AI agents, shifting the primary expense of small feature requests from writing code to debating scope in meetings. Teams can use agent-generated patches as diagnostic probes rather than final deliverables, turning abstract scope debates into concrete artifacts that reveal true system touchpoints and risks. However, low generation costs do not translate to low ownership costs, as changes touching areas like authorization, compliance, or public contracts still demand significant human review. Constrained attempts allow engineers to price uncertainty quickly and shift scope discipline from pre-implementation speculation to evidence-based code review.
Dalia AbuadasGithub ·
Using the GitHub Copilot SDK for Java
The GitHub Copilot SDK for Java offers a framework-agnostic client library to orchestrate AI agent sessions and tool execution directly from server-side Java code. Unlike framework-dependent alternatives, the SDK supports direct model providers such as OpenAI, Azure, and Anthropic through custom endpoint configurations without requiring a Copilot subscription. Developers can register tools declaratively using the experimental @CopilotTool annotation processor or dynamically through inline lambda definitions with ToolDefinition.from. When integrated into a Jakarta EE 11 application on Open Liberty, agent workflows run on container-managed virtual threads that propagate CDI and transaction contexts during blocking calls like sendAndWait. Real-time event subscriptions capture model execution steps and tool invocations to stream updates over WebSockets without exhausting platform threads.
Edward BurnsGithub ·
Better tools made Copilot code review worse. Here’s how we actually improved it.
When migrating GitHub Copilot code review from custom navigation tools to shared Unix-style CLI utilities—grep, glob, and view—benchmarks revealed higher review costs and fewer caught issues. Rather than an issue with the underlying tools, trace analysis showed the agent used general-purpose coding assistant instructions that triggered expansive repository browsing loops. In response, the team rewrote the tool guidance to enforce a review-specific workflow anchored to pull request diffs. The updated instructions direct the agent to narrow candidate call sites and files using batched discovery before reading minimal line ranges with view. This workflow tuning reduced average review costs by approximately 20% in production while maintaining review quality.
Napalys Klicius