Bot Management as an Adaptive Control Loop
Bot management is an architecture, not a rule list: classify traffic, score it with continuously retrained models, enforce with challenges or blocks, then feed the outcomes back into the model.
Loading…
Topic
49 posts about Cloudflare. Every summary links to the original.
See the idea
Bot management is an architecture, not a rule list: classify traffic, score it with continuously retrained models, enforce with challenges or blocks, then feed the outcomes back into the model.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Cloudflare announced an experimental Emscripten target for wasm-bindgen, enabling native Rust code compiled with wasm-bindgen to run directly on Workers with full platform feature virtualization. This resolves the long-standing incompatibility between Emscripten and wasm-bindgen by having Emscripten drive the build while wasm-bindgen produces portable bindings included in Emscripten’s library system. The integration virtualizes timers, file system operations, and sockets atop Workers’ Node.js APIs, bringing significantly improved library compatibility for many previously unsupported Rust crates after minor patches to libraries like libc and socket2. Tokio async runtime support is achieved through WebAssembly JavaScript Promise Integration, which suspends blocking calls and returns control to the event loop without stalling. In testing, a Rust Minecraft server (Pumpkin) was deployed inside a Durable Object with TCP ingress using real Tokio sockets, persisting world state through Durable Object storage and supporting multiplayer connections.
Cloudflare’s 2026 founders’ letter, marking the company’s 16th birthday, examines rapid changes in internet traffic driven by AI. Since mid-2025, new website creation has exploded, fueled by ‘vibe coding’ tools that allow non-programmers to build apps, with over 7 million developers now using Cloudflare’s platform. Automated traffic from AI agents and crawlers is projected to surpass human traffic by May 2026 and could reach 1,000 times human levels within five years. This surge risks a tragedy of the commons: agents consume resources from many sites but benefit only one, threatening small businesses that lack the information volume older competitors hold. Cloudflare proposes reducing crawler load by fetching only changed content and enabling creators to receive payment when agents use their content, alongside partnerships to shape a fairer internet.
Security researcher Oren Yomtov reported a vulnerability in Cloudflare Containers and Sandboxes through the bug bounty program, demonstrating that a Workers Paid account could recover residual disk blocks from previous containers on the same host. The issue arose from Linux device mapper thin provisioning with skip_block_zeroing enabled, which left unzeroed 64 KiB blocks when a 4 KiB write triggered allocation, allowing the remaining 60 KiB to retain prior data. Researchers recovered directory structures, database pages, and structurally complete SQLite databases from 18 of 24 placements across four continents, using ext4 checksums to distinguish foreign blocks. Cloudflare removed skip_block_zeroing, retired all running container disks, and cleared cached image snapshots created before the mitigation, completing cleanup by September 19. Retained disk-I/O telemetry showed no evidence of malicious exploitation, and the researchers confirmed secure deletion of recovered data.
The Vary response header, often called the ugliest part of HTTP, is now supported in Cache Rules on every Cloudflare plan. Vary tells caches which request fields may affect a response, but not which differences actually matter, leading to cache fragmentation or incorrect responses. For example, a server might deliver different image formats to different browsers, and a cache ignoring Vary risks serving the wrong bytes. The origin still names the request headers that may affect a response, but Cloudflare lets users decide how to handle each one. This support addresses the challenge of balancing correct caching with cache efficiency.
Worker Previews gives every Git branch its own production-like environment with an isolated URL, configuration, state, and observability. Developers deploy a Preview using npx wrangler preview, which creates a separate Worker version with its own variables, secrets, and bindings, distinct from production, on a stable URL that updates with each push. Durable Objects and Containers automatically receive new namespaces per Preview, preventing state contamination across branches during migrations or schema changes. Each Preview supports full observability—logs, errors, metrics, and traces—scoped to that branch, enabling agents to autonomously test, catch failures, push fixes, and verify before production. The feature enables an Agent Development Lifecycle where every change is atomic, independently deployable, and revisable before merging to production.
Cloudflare announced general availability of Python Workers, making Python a first-class language on its developer platform. The runtime now encapsulates type conversion so developers use Cloudflare bindings like D1 and Workers AI directly, without the previously required JavaScript glue code. Popular frameworks FastAPI, Django, and Flask run inside Workers via built-in ASGI and WSCI connectors that translate JavaScript requests into standard Python structures. A custom socket syscall implementation enables TCP connections for database drivers, unlocking Hyperdrive integration with PostgreSQL and MySQL, previously blocked by WebAssembly sandbox limits. Example projects include streaming from Bluesky Jetstream via Durable Objects, building an MCP server with the official Python MCP package, and creating a RAG system with Vectorize, with documentation now including Python examples alongside TypeScript.
Modern storefronts can appear healthy while malicious JavaScript diverts affiliate credit, hijacks searches or clicks, tampers with analytics, or loads further code. Cloudflare describes four operations spanning eight payloads that Page Shield ML detected in live traffic; humans verified the findings afterward, while seven payloads were absent from VirusTotal and none received a malicious verdict from URLScan. The detection system represents JavaScript as a graph with a graph neural network (GNN), sends fewer than 0.3% of analyzed scripts to a lightweight LLM on Workers AI, and uses a teacher cohort of frontier models for difficult investigations. This combination handles minification, renaming, some obfuscation, and behavior gated by device, time, geography, referrer, browser state, or cooldowns. The post concludes that continuous browser visibility and runtime context are needed because static, one-time scans can miss selectively executed attacks.
Cloudflare announces a Disallow AI Training setting intended to let website owners refuse AI training while keeping content available to search crawlers. The problem is mixed-use crawlers such as Applebot, Bingbot, and Googlebot, which can serve both search indexing and model training, making a simple block affect discoverability. Cloudflare says its network can publish preferences, identify and classify crawlers, block those that ignore controls, and report operator behavior on Radar; the setting publishes a no-training preference through robots.txt while allowing Accountable mixed-use crawlers for search. Apple, Google, and Microsoft meet or have committed to meet the Accountable requirements, which include training and summary opt-outs, URL-level visibility, metrics, and assurance that training opt-outs will not affect traditional search.
Ramp tested whether AI agents would relay a tracked referral incentive to people researching spend-management software. Across roughly 50 marketing pages, Cloudflare Workers served bot-specific variants—Markdown, stripped HTML, or schema-heavy content—while unique URLs and logs attributed downstream activity. Markdown surfaced most reliably, and bot classification proved difficult: major assistants were labeled “AI Assistants,” OpenAI SearchBot cached aggressively, and DeepSeek required TLS fingerprinting plus ASN detection. Over 32 days, agent relays rose from roughly 40 at 2.5 weeks to nearly 10 times that total, with Claude responsible for most matches; Perplexity moved from vague references to one specific branded mention, while ChatGPT remained at zero. The experiment concludes that agents can relay offers, but model-specific content strategies, citation-aware page selection, and better attribution are still needed.
The best engineering reads, once a week.