An editorial series that sets marketers, media buyers and publishers up for a successful 2026. More from the series →
When Cloudflare CEO Matthew Prince first started getting calls from distressed publishers about the threat of AI crawlers scraping their content, his knee-jerk reaction was to roll his eyes.
And honestly, a little eye-rolling is fair — Cloudflare’s day job is fending off botnets and nation-state cyberattacks, not debating how Google and other AI companies crawl publisher sites. That means any AI-focused crawling the company tracks represents a narrow slice of the overall traffic and data the cloud-edge company processes. “I remember being like, why is the media always so afraid of the next new technology?” he recalls.
But when publishers continued to press him, he pulled the data — and that’s when the scale and urgency of the problem snapped into focus for him. He was shocked. He could see that the business model of digital publishing was dissolving.
“Ten years ago, for every two pages that the Google [search] bot slurped down, copied, they would send you one visitor. Fast forward to today, it’s 18 pages scraped in return for one visitor. What’s changed? [Google] AI Overviews,” he says.
And that’s the good news.
For new players in the digital ecosystem, like OpenAI, that initial trade with publishers never existed. With OpenAI, it’s 1,500 pages crawled for every one visitor sent back to the publisher. If you look at Anthropic, it’s 40,000 to one, he stresses.
But this is just the kind of existential puzzle Prince seems to thrive on. Speaking from his house in Lisbon, Portugal, this fall, he sounds relaxed. Just the day before he was in London meeting with the Competition Markets Authority to argue for why Google must completely separate its AI and search crawlers.
Although his surroundings are obscured by a generic video-call backdrop, his affection for Portugal comes through clearly. He talks warmly about the locals and how glad he is that he moved the Cloudflare headquarters there from London after Brexit in 2016. It’s become a second home for him and his family, and they’ve spent roughly eight months there spread over the last two years. He uses the Portuguese capital as a base from which he can travel to meet customers across Europe and the Middle East.
But Prince’s energy spikes the moment the conversation turns to the problem consuming much of his time: what the future of the internet should be. This isn’t an abstract debate for him. It’s a live problem to be solved – one he wants to help shape into something better than the web architecture that came before.
“A lot of the criticisms of the internet, of social media, come back to this idea that the currency of the internet is traffic. And the problem with traffic is that the best way to get traffic is to appeal to humans’ worst instincts – you know, fear, greed, lust,” he says.
He points to how some publishers effectively fed this model by producing content as cheaply as possible and relying on increasingly provocative headlines to drive scale. “I don’t think that made humanity better, I think it made it worse…And I worry that if the company [Google] that taught us that traffic was the deity that we all have to worship, also controls the future of the internet, I’m not sure we’re going to fix some of those problems,” he says.
Let’s make news mistakes, he stresses, not repeat old ones.
Storytelling as a ‘superpower’
That moment when Prince, 51, saw the dire situation for publishers so starkly visible in the data, nearly two and a half years ago, set off a chain reaction. He hired former Adobe exec Will Allen as vp of product in October 2024. His remit: develop new products to assist publishers and help them defend themselves from the technological onslaught of unregulated AI scraping.
Allen works closely with Cloudflare’s chief strategy officer Stephanie Cohen, and reports to CTO Dane Knecht. He says he was struck by Prince’s personal involvement from the get-go, even before he was officially hired. Not expected from a CEO with 4,838 employees globally. “He just showed incredible patience and thoughtfulness,” says Allen. “He could have pushed me off to somebody else, but he wrote back amazing, thoughtful responses to the questions that I sent over.”
Allen says there wasn’t an exact role defined at that time, more that the vision was clear. “He knew that there was a center of gravity building around this sort of initiative… that we have an opportunity and maybe an obligation here, to build this better internet… so his message has been: we’re going to listen to our customers, listen to the world and try and take a swing at making this better for everyone,” says Allen.
Some of the fruits of those efforts bubbled more publicly to the surfac