CloudflareKuber NandwaniBenchmark
Cloudflare built a WAF tester that uses frontier AI models to iteratively mutate attack payloads based on real-time response feedback. The tester, implemented in Python, orchestrates an LLM proposal and review loop against an authorized customer staging environment protected by Cloudflare’s WAF. It ran 45 scenarios across six attack categories—XSS, SQLi, CMDi, SSRF, path traversal/LFI, and Log4j—starting from known blocked exploits and varying encoding, request placement, or destination representation. The vast majority of 1,107 recorded attempts were blocked; the few non-blocked requests were triaged by humans and used to create new detections that harden security for all Cloudflare customers. The post stresses that a WAF bypass still needs an exploitable application to succeed, so patching software remains essential, and suggests combining Managed Rules, Attack Signature Detection, and positive security controls for defense in depth.
A Python-based tester that, for each scenario, takes a known blocked exploit payload and iterates by using an LLM proposal call to suggest an encoded or delivery variation, building and sending the request, then using an LLM review call to inspect the response (status, selected headers, body) and choose the next variation. The loop stops when mutations produce no useful variation or a hard limit is reached. Neither model call has access to WAF rules, rule IDs, WAF Attack Score details, or the identity of the security layer that acted. Code controls all HTTP replay, request sending (with allowlist and redirect disable), state tracking, and result collection.
The LLM-based WAF tester starts from a known blocked exploit and iterates by mutating encoding, delivery location, or host representation, using the WAF’s response to guide the next variation.
Non-blocked requests were treated as leads for human triage, not confirmed exploits, and results were used to create new detections that benefit all Cloudflare customers.
A WAF bypass alone is insufficient to compromise an application; maintaining up-to-date software and combining WAF with positive security controls and threat intelligence provides stronger defense-in-depth.
The LLM sees only selected HTTP response data: status code, selected headers, and response body. It never receives WAF rule IDs, rule expressions, Attack Score details, or which security layer acted.
The tester treats each request that is not blocked as a lead for human review, not a confirmed exploit. Human triage and validation are used to decide whether a detection gap exists and warrants a new rule.
The test ran on an authorized customer staging environment with an allowlisted test User-Agent so automated-traffic controls did not stop it before the WAF. Cloudflare also emphasizes that customers should test against a staging hostname protected by the same controls as production.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Application security is a closed loop: discover and prioritize risk, govern access and agent behavior, protect at runtime, then investigate and feed findings back into the next cycle.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
