Bot Management as an Adaptive Control Loop
Bot management is an architecture, not a rule list: classify traffic, score it with continuously retrained models, enforce with challenges or blocks, then feed the outcomes back into the model.
Loading…
Topic
Cloudflare
49 posts about Cloudflare. Every summary links to the original.
- 49 posts
- 2 companies
- 1 explainer
See the idea
Cloudflare explainers
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Continuous accountability for Verified bots
Cloudflare introduces a system to continuously evaluate whether Verified bots honor website preferences, moving beyond identity verification to sustained trust by measuring compliance with robots.txt directives and HTTP 429 rate-limit responses over time. The system looks for repeated patterns of disregard rather than isolated violations, lowering a bot’s trust band if it ignores site preferences or fails to slow down after receiving rate-limit responses, while consistent compliance increases trust. Cloudflare previously de-listed Perplexity’s bot in 2025 for seeking to access opted-out resources and attempting to circumvent controls when blocked, illustrating the need for ongoing accountability. The company plans to share these trust scores publicly on Radar and eventually let customers choose the trust threshold for handling Verified bot traffic on their websites. This forms an accountability loop with Bot Preference Sync and Bot Management, ensuring trust is earned through sustained behavior rather than assumed from identity alone.
Deploy with Void
VoidZero has open-sourced Void, a toolkit for building and deploying full-stack Vite applications to Cloudflare. The release includes three products: Void Platform, Void SDK, and Void CLI, which integrate into a single Vite workflow. The SDK provides file-based routing, database integration via imports, and end-to-end type safety from the database to the user interface. The CLI handles building, migrating, provisioning resources, and deploying with a single command, automatically inferring bindings like databases or AI services. The Platform offers a shared deployment environment with access controls and resource limits, enabling teams to prototype and ship internal apps in minutes.
Cloudflare and OpenAI: making AI discovery better for everyone
Cloudflare and OpenAI launched a research pilot in July 2026 to help ChatGPT users discover content more efficiently using network signals about page existence changes and potential harm. The pilot ran on a cohort of about one percent of Cloudflare’s network and has enabled OpenAI to index hundreds of millions of new web pages that are useful but sparsely linked. Cloudflare provides these discovery signals to OpenAI’s crawler while OpenAI independently evaluates and ranks pages by relevance and helpfulness for ChatGPT Search. The collaboration also tests signals that flag harmful pages like fake login forms so crawlers avoid them without penalizing the legitimate domain. The next phase will refine page selection test faster change signals and measure business outcomes for site owners whose content is discovered.
Yandex outage: Data shows that instead of going dark, malicious bots went on the move
Reported drone strikes on Yandex data centers from October 7 to 11 caused network traffic to drop 99% below baseline for Yandex Search and 90% for Yandex.Cloud. Using Cloudflare Radar and Bot Management data, the analysis compared pre-event automation baselines with post-strike patterns on both ASNs. The Yandex Search ASN drop consisted almost entirely of verified search crawlers: 93.6% of its bot requests were verified, and 78.1% belonged to YandexBot. On Yandex.Cloud, vulnerability scanning fell 90% and plugin probing dropped 93%, but high-volume operations migrated to other Russian hosting providers, preserving identical JA4 fingerprints. Global bot traffic remained stable at 39.5%, demonstrating that malicious automation moves rather than disappears and that request behavior analysis is necessary to understand threats.
Rewinding the exponential: How we keep hardware usage down
Cloudflare’s Data team manages a pipeline handling over two billion events per second from tens of thousands of servers, powering Logs, SQL API, and observability. To counter exponential traffic growth against linear hardware expansion, they implemented several optimizations over five years, deferring two and a half years of hardware expansion. In 2022, they replaced a monolithic schema with product-specific schemas, reducing backbone bandwidth by 35-40%. In 2024, they switched compression from LZ4 to zstd, achieving a 40% reduction in compressed traffic with negligible CPU impact. In 2026, they decoupled pipeline transit from database redundancy via a shared ingest pool and late replication, applied sampling downstream, and introduced aggregations, eliminating about 70 GB/s of compressed traffic and halving projected 2031 data volume.
Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash
Cloudflare introduced Clef-omni, a decision model that natively handles audio, video, image, and text inputs in a single pipeline, built on a Qwen3-Omni mixture-of-experts backbone with text-to-speech components removed. Unlike cascading pipelines, it directly embeds media into a unified sequence and uses two-stage attention routing with lexical grammar to compute confidence scores, trained by freezing the backbone and applying LoRA adapters with label-smoothed cross-entropy and Brier score calibration. The company also cut Clef-flash’s price to $0.038 per million input tokens while reducing its context window from 64k to 24k for the hosted version, based on usage data showing only 0.24% of requests exceed that limit. Clef’s hosted version received speed optimizations at the infrastructure layer without new model weights, and Clef-omni is available open-weight on HuggingFace. Benchmarks show Clef-omni scoring 98.2 on BFCL case exact and 92.7 on API-Bank accuracy, with median text-only decision latency around 130 milliseconds.
Deno is joining Cloudflare
The Deno team is joining Cloudflare to merge workerd and celld, aiming to make self-hosting Workers and Durable Objects radically simpler for developers. Ryan Dahl, creator of Node.js and Deno, explains that Durable Objects provide a powerful abstraction: a distributed singleton with a SQLite database that scales by sharding state and WebSocket connections across machines. He started celld to run this model outside Cloudflare with a single Rust binary and object storage as its only external dependency. Kenton Varda counters the theory that Workers’ unique design creates lock-in, noting that workerd is open source and that celld provides an escape hatch for migration. The merged effort, led by Ryan and Bert Belder, will make self-hosting a first-class supported way to build and run apps using the Workers programming model.
Bridging technical depth and usability: The story behind Radar’s redesign
Cloudflare Radar was redesigned to make its real-time Internet data accessible to journalists, human rights advocates, and everyday users while preserving depth for network operators and researchers. The previous landing page used a bento box layout that gave everything similar visual weight, felt intimidating to newcomers, and lacked brand consistency. The new design introduces an interactive map as the primary navigational element, leading with traffic and outages to provide concrete, timely reference points. Below the map, a tabbed exploration flow replaces the old chart widgets, offering summary statistics and clearer calls to action. The design language blends marketing approachability, content structure, and product maintainability by migrating to Cloudflare’s Kumo component system, aiming for a human-centered experience that does not sacrifice technical rigor.
Building an evidence-grounded agentic security operations harness on Cloudflare
Cloudflare’s Managed Defense harness uses a team of specialized AI agents to analyze security alerts, separating deterministic evidence collection from model inference to improve accuracy. Before any AI analysis, fixed reconnaissance workflows collect customer identity, detection history, traffic baseline, and enforcement outcomes, making evaluation reproducible. A lightweight triage model called Clef filters high-confidence false positives, while a coordinator agent runs four specialist agents in parallel for deeper investigation. The specialist agents cover traffic analysis, customer context, global telemetry, and threat intelligence, with a synthesis agent combining their typed findings into an advisory. The system preserves customer privacy by using only aggregate global telemetry, and Managed Defense Analysts remain responsible for final decisions and mitigations. The approach addresses three problems from a single-agent prototype: context becoming authority, scope drift, and failure to distinguish missing data from negative results.
The keys to the Internet change on October 11. Are you ready?
On October 11, 2026, the DNS root will replace its DNSSEC key-signing key (KSK) for only the second time, moving from KSK-2017 to KSK-2024. The rollover changes the trust anchor that validating resolvers use to verify the root’s DNSKEY records. Cloudflare has published KSK-2024 in the root’s DNSKEY set since January 11, 2025, and added it to its own resolver’s built-in trust anchors in July 2024. To help operators check readiness, Cloudflare uses RFC 8509 root key trust anchor sentinels, which let a resolver report whether it trusts a given key. Most website operators need no action, but DNSSEC-validating resolvers should confirm they trust KSK-2024 before the switch.
Everything we launched during Birthday Week 2026
Cloudflare marked its 16th Birthday Week with 46 announcements spanning open source, application security, AI agent economics, and developer platform upgrades. The week began with a new agentic CLI called cf, an open-source generation pipeline called Forge, and EmDash, a serverless CMS with sandboxed plugins. Tuesday focused on Cloudflare’s intention to become a public certificate authority and post-quantum security tools. Wednesday introduced Pay Per Use and the Monetization Gateway, creating an economic model for AI agents to pay creators when using their content. The week closed with features for faster connectivity and broader accessibility.
Introducing Web Search API via AI Gateway
Cloudflare launched a Web Search API via AI Gateway, partnering with Ceramic.ai, Exa, and Linkup, to provide grounded intelligence by letting AI agents search the web instead of guessing URLs that return 404 errors. The API injects live, structured snippets into inference pipelines, giving models access to information beyond their training cutoff, such as new Birthday Week releases, and grounding agent responses with real-time data. Developers can integrate via AI Gateway credits, a direct REST endpoint with curl, or Workers bindings with one line of code. Partners commit to Cloudflare’s verified bot standards, including identifying crawlers, respecting robots.txt, and providing source links for transparency. Web search calls are logged in AI Gateway observability, billed at list pricing without markup, support Bring-Your-Own-Key and Zero Data Retention, with native server tools coming soon.
8 major updates to Cloudflare Observability
Cloudflare announced eight major updates unifying logs, traces, analytics, alerts, dashboards, and data export into one observability platform with simplified pricing. The new Logs home combines Workers Observability and Log Explorer, letting users query datasets like HTTP events, firewall events, Workers, and R2 using SQL or filters. Cloudflare Traces, now in open beta, provides request-level visibility across security rules, transformations, caching, routing, Workers, and origin, with OpenTelemetry export and W3C trace context propagation. A unified SQL API lets humans and agents query telemetry across Cloudflare products, and custom alerts can be set on any dataset using threshold, anomaly, or SLO conditions. Pricing shifts to a volume-based model for ingested and stored logs and traces, effective December 1, 2026, with free tier at 0.5 GB ingestion per day and paid plans starting at 50 GB ingestion per billing cycle.
Introducing Cloudflare Traces: follow requests through our entire platform
Cloudflare Traces, in open beta, extends tracing beyond Workers to the full request path, capturing spans for security, transformations, cache, routing, Workers, and origin handling in one timeline. Users set a baseline sampling rate and use Trace Rules to override it for specific traffic, enabling targeted debugging without increasing volume across the domain. The feature accepts and forwards W3C traceparent headers for distributed tracing and exports spans via OTLP to any compatible backend, with no instrumentation required. It reveals why a request was blocked, which rule acted, whether the URL was rewritten, and where time was spent between cache, origin, and the application. Pricing, effective December 1, 2026, is based on ingestion and retention: free tier 0.5 GB daily with 7-day retention, paid plans 50 GB ingestion and up to 365-day retention.
Updates on our pledge to make Cloudflare features accessible to everyone
Cloudflare CTO Dane Knecht pledged a year ago to make every Cloudflare feature available to all customers, not just Enterprise. The company has since expanded Logpush and Logpush Transformers, which use SQL to filter, redact, and reformat logs, to Free, Pro, and Business plans. Custom Dashboards for personalized metrics are now available to all customers. Role-Based Access Control (RBAC) has been extended to nearly all products, including resource-level roles for Workers, and a New Account button lets users create additional accounts directly. Organizations, which group accounts for unified analytics and shared configurations, are in beta for Enterprise and will reach free accounts in early 2027, and platform limits have also been raised, including Workers subrequests from 50 to 100 and Durable Objects idle time from 70–140 seconds to 15 minutes with active outbound connections.
Announcing Cloudflare OHTTP Gateway – expanding access to Cloudflare’s privacy-preserving infrastructure
Cloudflare announced the closed beta of its OHTTP Gateway, a managed service that decapsulates encrypted OHTTP requests for application servers while preserving client privacy and expanding access to privacy-preserving infrastructure. OHTTP is an IETF standard that uses two independent hops—a relay and a gateway—to hide client IP addresses and TLS fingerprints from app servers, ensuring no single party sees both client identifiers and request contents. The new Gateway is designed for customers whose app servers are already behind Cloudflare, allowing them to use a third-party relay while Cloudflare handles the cryptographic work, for example with Apple’s LiveCallerID. Cloudflare also renamed its existing Privacy Gateway product to Cloudflare OHTTP Relay to better distinguish the two product options. The Gateway runs on Cloudflare’s global edge network to minimize latency, and developers can implement OHTTP clients using sample libraries while joining a waitlist for the paid add-on.
Follow the thread: a new dashboard to investigate account abuse
Cloudflare’s new Account Abuse Protection dashboard helps fraud teams investigate account abuse by moving beyond stateless identity checks to a stateful trust model. The dashboard uses cryptographically hashed identifiers, called Hashed User IDs, created from customer-provided identifiers like email or username, to anchor account activity across login and signup events. It provides an account population overview with metrics like total events, unique IPs, and devices to spot anomalies such as leaked credential matches or unusual login failure rates, then allows analysts to filter and drill into individual accounts. Each account view shows historical behavioral, network, and device signals, enabling teams to reconstruct events and decide on responses like WAF rules. The dashboard also introduces two new roles—Account Abuse Protection and Account Abuse Protection PII—to enforce least privilege for dashboard access and PII exposure.
Protected Quick Tunnels: simple accountless authentication for your next dev project
Cloudflare’s Quick Tunnels now support email-based access control via the –allowed-mail flag, requiring no account on either side. The feature addresses the risk of exposing sensitive work, especially as AI coding agents increasingly use tunnels to share previews. Cloudflare Access verifies the visitor’s email with a one-time PIN, while cloudflared on the developer’s machine authorizes the request against a local rule list. The guest list never leaves the developer’s machine, and the tunnel remains accountless and free. The feature was built and shipped by two interns, Hugo Vicente and Alessandro Frigerio.
Building for good: How civil society organizations are automating on Cloudflare
Cloudflare awarded $7.5 million in credits to 30 civil society organizations to build automation tools using Workers AI and AI Gateway on its global network. LebTown replaced manual editorial workflows with a custom management system, and Kaya Guides built a WhatsApp mental health app processing 500,000 messages monthly with real-time counselor feedback. Larger groups like Freedom House and Article One received volunteer engineering support to build tools for tracking transnational repression and assessing human rights risks from factory audits. Cloudflare’s developer services include automatic DDoS protection and data minimization, addressing privacy and cost barriers that have limited AI adoption in the sector. The program is now open for a second cohort of non-profit organizations to apply for up to $250,000 in credits.
2026 Birthday week: network performance update
Cloudflare is now the fastest provider in 74% of the top 1,000 global networks, up from 60% in April 2026, representing an additional 150 networks and 38 countries. Performance is measured using connection time (TCP handshake) and the trimean of connection times, which combines the 25th, 50th, and 75th percentiles to reduce outlier influence. To expand data collection, Cloudflare introduced a new methodology using Challenge Pages, which run non-interactive background measurements fetching lightweight files from providers including Cloudflare, Amazon CloudFront, Google, Fastly, and Akamai. Challenge Pages are delivered by Turnstile and preserve privacy while providing broader reach than error-page measurements alone. This increased measurement volume narrows confidence intervals around trimean values, making rankings more stable in networks where top providers are separated by only a millisecond or two.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.