Is web browsing as we know it about to disappear? Will AI agents replace traditional web access? Will they browse like humans, or will a new model emerge? And what does this mean for web data collection, analysis, and insights?

In this series of articles I'm going to try to answer these questions and more. Am I qualified to answer them? Honestly, who is? Things are moving so fast that predictions can become outdated within a week. But with 20 years in the data business, having built the first residential proxy network that opened up the web business data market and designed Bright Data’s product lineup for the last 6 years — now the largest public web data collection platform, handling four times more requests per day than Google Search worldwide — I’d say that, at the very least, I've got one of the best seats in the house to watch all of this unfold. Bright Data powers the web data pipelines of over 20,000 customers, including many Fortune 500 companies and major AI players. Providing us a unique position to spot emerging trends before they hit the mainstream.

These articles will be short and packed with insights and educated guesses based on emerging trends extracted from our platform and customer base.

Chapter One - The Chicken or the Egg

We’ll start with the subject closest to home - AI agents and their web access requirements. Have we seen the spike of web training data? Is it on the way down or growing? Is the type of data required changing? What are the current evolving trends? Does the name of the chapter have anything to do with its content? (Spoiler alert - no it doesn't) - Let's dive in.

Since the release of Chat GPT AI has been advancing at a phenomenal speed. It seems that all restraints have been removed and the race between the big LLMs is raging on with new releases on a weekly basis. But when the dust settles down you realize that AI is an amazing sidekick that can handle many small and focused tasks and can speed up your work significantly, but if you try to find AI agents that can successfully take over even a simple task end-to-end with a real service impact it will be hard to find — especially if it involves any financial risk at scale. It's not there yet (i.e. a travel agent booking non-refundable tickets or a data collection bot scraping tens of millions of pages at a cost of hundreds of thousands of dollars).

Regardless, AI agents are one of the biggest current hypes of the AI world. At Bright Data, we see firsthand how data access is becoming the defining factor in their success. The AI landscape is shifting from reliance on massive, centralized LLM generic training data to localized, specialized AI agents that require real-time, domain and location specific data. This shift will have profound implications for businesses, data infrastructure, and the broader AI ecosystem.

According to stats collected by Bright Data and extrapolated over the industry the number of new AI based agents created on a daily basis is staggering and revolves around 5,000 new Agents per day. And that’s not including self-replicating agents which can triple this number.

These numbers assume at least a 30 day survival period — the number of agents that are created and trashed within shorter periods of time is probably 3 times higher.

According to what we are seeing this is the rough breakdown:

• Research & Industry – estimated ~750 new AI agents created daily
• Open-Source & Developer Contributions – ~3,000 new agents added daily
• Automated AI agent generation – over 10,000 new agents created daily (Self replicating)

Even new Autonomous multi-agent systems (MAS – multiple AI entities working together to achieve goals, often with decentralized decision-making) are created daily and require massive amounts of data.

• Research & Academia – ~100 new MAS prototypes added daily
• Industry & Enterprise – ~300 new MAS deployed daily
• Open Source & Developer Projects – ~500 new MAS added daily
• Automated MAS generation – over 1,000 added daily (AI-driven agent creation tools)

Not all of these agents will survive for long, but even with a 0.1% long term survival rate, we are looking at thousands of new AI agents every month requiring continuous web access to function effectively.

The Data Bottleneck: AI’s Growing Appetite for Web Access

AI agents replacing traditional services will require at least the same amount of data as current non-AI systems use — though in fact it will probably be much more. As such Bright Data is especially situated to predict future web data usage by looking at the web data consumption required by its customers to fuel current web data based projects, add to this the trends by new customers that are already feeding AI agents both with web data for training and for direct web access and extrapolate the results for the whole market.

Based on the trends we are seeing, Agents require roughly 1TB of web data per month (only 1% of that for training and the rest for providing the service). And that number keeps growing from month to month at a rate that should eventually double by the end of the year and is very likely to continue in 2026 as well.

Based on this, by the end of 2026 the estimate is that AI agents will require:

• 10PB of web data consumption per month
• 3–5x increase in real-time web scraping and API-based data access
• A shift toward decentralized AI training, requiring more localized live data sources rather than relying solely on the generic static training data of LLMs

For comparison this is equivalent to a little more than a third of all Google Search traffic (not the amount of data that Google is crawling but the amount of search queries data requested by users).

All of these estimates just represent the building stages and first exposure of these agents — once adoption kicks in, the numbers will scale up exponentially.

To wrap up this chapter we’ll take a look at how these agents are accessing the web. Currently 99% of agents access the web directly using standard browsing methods. However, there is some movement towards using dedicated tools — service APIs programmed directly by the agent creators or advertised using MCP (Model Context Protocol) or a similar standard.

Both of these directions are not yet production ready for use in high scale where accuracy is critical. AI still makes too many mistakes. While it is reasonable for reading data from a web page and providing answers to user questions with a disclaimer that it can be wrong, it's not possible to provide the simplest paid service reliably where mistakes have consequences. On the other hand, tool-based APIs require standards and widespread adaptation which requires time and effort. In this path an agent will always be constrained by the scope of the provided tools while browsing provides freedom to do anything.

We’ll keep track of this trend but with OpenAI endorsing MCP and the emergence of agent-to-agent protocols (Google A2A), standardization is well on its way. It’s also a good way to keep AI in check — reducing the freedom makes the possible mistakes more predictable and, hopefully, manageable. I think it’s pretty clear that what we will see over the next couple of years is the creation of an internet AI overlay network (AIONet if you want) that will be completely separate from the internet we humans use — where agents will communicate more efficiently over their own standards of communication and will rarely need to use the clunky interface of standard “human” websites.

Just for fun, if we want to go a little further and wilder with the predictions — I assume that most websites will be replaced by agents of their own and they will just be talking to other agents, fetching information and ordering services. People will start viewing the world through the filtered eyes of their personal agent and the internet as we know it will mostly cease to exist. But that is at least 5–7 years away and most likely reality will take us by surprise as usual and it will be something else completely.

So. To summarize. We answered the question about AI Agent creation trends and got some insights on their web access requirements and future trends.

In the next article “Chapter 2: Beauty and the Beast” I’ll give you a peek into the very deep and complex areas of Security, Compliance and ethics revolving around both web access in general and AI agents in particular — and also a rearview update on the trends we’ve already discussed.