CloudflareHrushikesh DeshpandeIncident / postmortem
Security researcher Oren Yomtov reported a vulnerability in Cloudflare Containers and Sandboxes through the bug bounty program, demonstrating that a Workers Paid account could recover residual disk blocks from previous containers on the same host. The issue arose from Linux device mapper thin provisioning with skip_block_zeroing enabled, which left unzeroed 64 KiB blocks when a 4 KiB write triggered allocation, allowing the remaining 60 KiB to retain prior data. Researchers recovered directory structures, database pages, and structurally complete SQLite databases from 18 of 24 placements across four continents, using ext4 checksums to distinguish foreign blocks. Cloudflare removed skip_block_zeroing, retired all running container disks, and cleared cached image snapshots created before the mitigation, completing cleanup by September 19. Retained disk-I/O telemetry showed no evidence of malicious exploitation, and the researchers confirmed secure deletion of recovered data.
Cloudflare Containers used Linux device mapper thin provisioning with skip_block_zeroing enabled, which allowed residual data from previous containers to be recovered by a new container on the same host. The vulnerability was reported by security researcher Oren Yomtov through the bug bounty program.
Cloudflare removed skip_block_zeroing from the dm-thin pool configuration across the fleet, retired all running container disks, and cleared cached image snapshots created before the mitigation. This restored default zeroing behavior and eliminated residual data exposure.
The vulnerability exploited dm-thin’s skip_block_zeroing option, which left unzeroed 64 KiB blocks when a small write triggered allocation, allowing residual data from previous containers to be read.
Researchers used ext4 directory block checksums to distinguish foreign blocks from their own test filesystem, recovering directory structures, database pages, and SQLite databases.
Cloudflare’s remediation required no customer-side changes and involved removing skip_block_zeroing, retiring running disks, and clearing cached snapshots across the fleet.
The vulnerability used Linux device mapper thin provisioning with skip_block_zeroing enabled. When a new container wrote a small 4 KiB block into an unmapped 64 KiB region, dm-thin allocated a physical block without zeroing the remainder, leaving up to 60 KiB of residual data from a previous container readable through raw device reads.
No, an attacker could not select a particular victim or access an actively attached disk. Exposure depended on workload placement and which previously released blocks dm-thin reassigned, making targeting impossible.
Cloudflare reviewed retained disk-I/O telemetry and found no evidence of malicious exploitation. Activity attributed to the technique came from the researchers and Cloudflare engineers conducting authorized validation.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
An architecture view of a bug bounty program: scope, triage, validation, containment, durable fix, and payout, plus the loop that turns each report into stronger defaults.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
