---
title: "Cloudflare and OpenAI: making AI discovery better for everyone"
description: "Cloudflare and OpenAI launched a research pilot in July 2026 to help ChatGPT users discover content more efficiently using network signals about page existence changes and potential harm. The pilot ran on a cohort of about one percent of Cloudflare’s network and has enabled OpenAI to index hundreds of millions of new web pages that are useful but sparsely linked. Cloudflare provides these discovery signals to OpenAI’s crawler while OpenAI independently evaluates and ranks pages by relevance and helpfulness for ChatGPT Search. The collaboration also tests signals that flag harmful pages like fake login forms so crawlers avoid them without penalizing the legitimate domain. The next phase will refine page selection test faster change signals and measure business outcomes for site owners whose content is discovered."
---

# Cloudflare and OpenAI: making AI discovery better for everyone

[Cloudflare](https://yomu.fyi/company/cloudflare) · James Balfour (OpenAI) · Oct 15, 2026

**Type:** Announcement

## Summary

Cloudflare and OpenAI launched a research pilot in July 2026 to help ChatGPT users discover content more efficiently using network signals about page existence changes and potential harm. The pilot ran on a cohort of about one percent of Cloudflare’s network and has enabled OpenAI to index hundreds of millions of new web pages that are useful but sparsely linked. Cloudflare provides these discovery signals to OpenAI’s crawler while OpenAI independently evaluates and ranks pages by relevance and helpfulness for ChatGPT Search. The collaboration also tests signals that flag harmful pages like fake login forms so crawlers avoid them without penalizing the legitimate domain. The next phase will refine page selection test faster change signals and measure business outcomes for site owners whose content is discovered.

## Context

The web contains many specific pages that are valuable for AI responses but difficult to discover through conventional crawling. Cloudflare’s network signals about page existence changes and harm can help AI search crawlers find and index useful content more efficiently while reducing wasted requests.

## Approach / What changed

Cloudflare shares network-derived metadata signals with OpenAI to indicate whether pages exist have changed or are potentially harmful enabling the OpenAI crawler to discover new pages and avoid wasting resources. OpenAI then independently evaluates and ranks those pages for relevance and helpfulness in ChatGPT Search.

## Takeaways

- The pilot enabled OpenAI to index hundreds of millions of new web pages using Cloudflare network signals that indicate page existence and changes.
- Cloudflare’s change signals help crawlers avoid re-downloading unchanged pages reducing unnecessary load on website owners.
- Harmful pages such as fake login forms can be flagged at the page level without penalizing the rest of the domain improving safety while respecting site owners.

## FAQ

### What signals does Cloudflare share with OpenAI?

Cloudflare shares metadata about whether a page exists whether it has changed and whether it may be harmful. These signals help OpenAI's crawler decide what to fetch and what to avoid.

### Does this share page content for model training?

No. The research uses metadata about the web for search and discovery not page content for training. Site owners can use Cloudflare's AI Crawl Control to manage access.

### How can site owners control AI access to their site?

Cloudflare provides AI Crawl Control which lets domain owners manage how AI systems access their website including the ability to opt out of crawling.

**Tags:** [AI](https://yomu.fyi/topic/ai), [AI Search](https://yomu.fyi/topic/ai-search), [Cloudflare](https://yomu.fyi/topic/cloudflare), [Crawlers](https://yomu.fyi/topic/crawlers), [OpenAI](https://yomu.fyi/topic/openai)

- Source: [Cloudflare](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web)
- Source URL: https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web
- Ingested by Yomu: 2026-10-15T15:00:14.093Z

[Read original post](https://blog.cloudflare.com/cloudflare-openai-discovery-agentic-web)