Topic

Bots

1 post about Bots. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Cloudflare
Continuous accountability for Verified bots
Cloudflare introduces a system to continuously evaluate whether Verified bots honor website preferences, moving beyond identity verification to sustained trust by measuring compliance with robots.txt directives and HTTP 429 rate-limit responses over time. The system looks for repeated patterns of disregard rather than isolated violations, lowering a bot’s trust band if it ignores site preferences or fails to slow down after receiving rate-limit responses, while consistent compliance increases trust. Cloudflare previously de-listed Perplexity’s bot in 2025 for seeking to access opted-out resources and attempting to circumvent controls when blocked, illustrating the need for ongoing accountability. The company plans to share these trust scores publicly on Radar and eventually let customers choose the trust threshold for handling Verified bot traffic on their websites. This forms an accountability loop with Bot Preference Sync and Bot Management, ensuring trust is earned through sustained behavior rather than assumed from identity alone.

The best engineering reads, once a week.

Topic

1 post about Bots. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Cloudflare
Continuous accountability for Verified bots
Cloudflare introduces a system to continuously evaluate whether Verified bots honor website preferences, moving beyond identity verification to sustained trust by measuring compliance with robots.txt directives and HTTP 429 rate-limit responses over time. The system looks for repeated patterns of disregard rather than isolated violations, lowering a bot’s trust band if it ignores site preferences or fails to slow down after receiving rate-limit responses, while consistent compliance increases trust. Cloudflare previously de-listed Perplexity’s bot in 2025 for seeking to access opted-out resources and attempting to circumvent controls when blocked, illustrating the need for ongoing accountability. The company plans to share these trust scores publicly on Radar and eventually let customers choose the trust threshold for handling Verified bot traffic on their websites. This forms an accountability loop with Bot Preference Sync and Bot Management, ensuring trust is earned through sustained behavior rather than assumed from identity alone.

The best engineering reads, once a week.