Cloudflare AI Infrastructure: Crawlers, Workers AI, Cyber Defense

A reverse proxy for a fifth of the web

Cloudflare was incorporated in Delaware in July 2009 by Matthew Prince, Michelle Zatlyn and Lee Holloway, and went public on the New York Stock Exchange in September 2019. Its core business sits between website visitors and the servers those sites run on: a reverse proxy that filters malicious traffic and caches or accelerates the rest. Matthew Prince remains chief executive; Michelle Zatlyn is president. That position, in front of roughly a fifth of all websites by one widely cited measure, is what makes the rest of the company's product line possible: DDoS mitigation, a zero-trust access product, domain registration, and Workers, a serverless platform for running code at the network edge. [2][1]

Atlas interpretation: That vantage point is also why Cloudflare keeps showing up in AI coverage on both sides of the same fight: it sees a meaningful share of the traffic AI systems generate as crawlers, and a meaningful share of the traffic AI systems are now used to attack. [2]

Gatekeeping the crawlers

Cloudflare's Workers AI product, launched in 2023, lets developers run inference on GPUs inside Cloudflare's network rather than calling out to a separate provider, one of several ways the company has tried to sit inside the AI stack rather than just in front of it. More consequential for publishers has been Cloudflare's move on the crawling side: in July 2025 it introduced Pay per Crawl, letting a site owner charge, block, or freely allow individual AI crawlers per domain, using an HTTP 402 response and a bot-authentication signature to keep the charges from being spoofed. Cloudflare pitched it as a third option between blocking AI crawlers outright and letting them scrape for free, the choice publishers had been stuck with since large language models started training on the open web. [3]

Cloudflare on the timeline

In May 2026, Cloudflare published Project Glasswing, a report on turning Anthropic's Mythos model loose on more than fifty of its own internal code repositories. Where earlier models Cloudflare had tested stopped at describing a vulnerability, Cloudflare's account is that this model chained low-severity bugs into working exploits and produced proofs that compiled. Cloudflare described the result as a difference in kind, not a better version of the same tool, and the report is the timeline's main source for that claim; it is Cloudflare's own account of testing on its own infrastructure, not an independent audit. [4]

In August 2026, Cloudflare was one of more than a hundred companies, alongside OpenAI, Anthropic, Google, Microsoft, Amazon and others, that signed an open letter calling for collective action on cyber defense, published the day after Hugging Face's postmortem on the July 2026 agent intrusion into OpenAI's evaluation environment. The letter's premise, that AI currently helps defenders faster than attackers but that the advantage may not last, is a position statement from the signatories rather than a measured finding, and the timeline treats it as such. [5]