The Internet is under threat

The cracks are already visible: traffic is evaporating, bots are ballooning, and creators are footing the bill

For thirty years, the web has been powered by a simple bargain - creators produce content and search engines drive visitors to it. Advertising or subscriptions fund the process. This model has underpinned everything from newspapers to travel reviews.

But now Generative AI is changing the rules. Instead of navigating through search results, Chat users now pose a question and receive a single answer with all the details neatly stitched-together. No click-throughs, no referral traffic, and no incentives for publishers.

ChatGPT and Google’s AI Overviews are a fundamental threat to the web’s core business model. Barclays data suggests that Google now requires 18 pages to be crawled for every one referred visitor, up from 2:1 a decade ago. For OpenAI, that ratio is estimated to be 100 times worse. The problem is, that if no one ever reaches the source, the fundamental economics of the web simply collapse.

The invisible cost of AI crawlers

The second challenge is bandwidth. AI models are scraping vast amounts of content, imposing real costs on the hosting platforms that websites reside on.

• Anthropic’s crawler visited Freelancer.com 3.5 million times in just four hours.
• iFixit received 1 million visits (from bots) in 24 hours, costing them money without delivering any value to the company.
• Wikimedia reports a 50% surge in bandwidth usage due to AI bots, which generate 35% of page views but 65% of the most expensive requests.
• F5 Inc. estimates that over half of all web requests now come from AI and LLM-driven bots.

This is not benign exploration – it is pure extraction and it delivers no compensation to the site owners.

The surge of Generative AI

Far from being a novelty, Generative AI is now central to business and consumer life.

• In 2024, GenAI traffic grew by 890%.
• Morgan Stanley predicts the market will expand from USD 45 billion in 2024 to USD 1.1 trillion by 2028, with firms seeing positive returns as early as 2025.
• Over 77% of businesses expect GenAI to drive future revenue, and it already accounts for 26% of analytics budgets.
• Advertising is shifting too – 86% of marketers say they are already using or plan to use GenAI for video ads, projected to make up 40% of all such ads by 2026.

This is not incremental change – it is a rewiring of the digital economy.

The core question: Who pays?

The internet has always been funded by eyeballs and clicks. But if AI engines provide the answers directly – and scrape content without paying for it – the funding model for the open web begins to break.

Without adaptation, we risk an outcome where:

• Referral traffic disappears, undermining advertising and subscriptions.
• AI bots drain infrastructure and budgets without return.
• Revenue consolidates around the gatekeepers who control AI models.

Possible futures

We are not doomed to watch the collapse passively. Several paths are already being explored:

• Licensing and compensation models: an “AI-as-Spotify” approach where creators are paid whenever their content is used to generate answers.
• Technical enforcement: Cloudflare now blocks AI bots by default unless they pay, forcing negotiations.
• Trust layers: content tagged to signal reliability and consent, giving users and AI models clarity on provenance.

The web has reinvented itself before – from portals to search, from desktop to mobile. But this time, the stakes are existential.

A fragile, strange future

The next iteration of the web may not be a patchwork of websites but a fluid fabric of data repositories, stitched together to catalyse bespoke responses to individual questions. Whether that fabric is woven with fairness and trust – or dominated by a handful of AI oligarchs – depends on what we do now.

Generative AI is not just a tool – it is becoming the default interface to knowledge. If we want the web to remain open, sustainable, and creative, we must build new ways of funding and trusting it before the old bargain disappears.

Got any questions?

If you’d like to understand how well your organisation performs in AI search and assess your authority, we'd be happy to connect you with our team.