The Parallel Open Web // BRXND Dispatch vol 121
Is this the twilight or renaissance of the internet we grew up on?
Since we have a fair amount of new faces here, a reminder that you’re getting this email as a subscriber to the BRXND Dispatch, a newsletter at the intersection of marketing and AI. Forward it to a friend, they’d like that, and so would we.
We also host a flagship event every year for a select group of marketing leaders. If you’d like to attend, we still have a small handful of early bird tickets left at a discounted rate of $749. Any questions, I’m an email away at mike@brxnd.ai.
The Parallel Open Web
The argument for the obsolescence of the open internet we know and sometimes love is ultimately a pretty simple one.
When the largest technology companies in the world have a vested interest in protecting access to proprietary data, the web is destined to become a closed entity of fiefdoms that feed the AI beast. In a world where context provided to an LLM is king and distribution is the rate-limiting resource, the die has ostensibly been cast.
In response to the rational economic incentives AI creates, all of the best content providers will move behind paywalls to directly monetize their audiences and exert whatever leverage they can with LLMs for licensing deals. More than ever, information simply does not want to be free. It longs to be expensive and scarce to accrue maximum value to those who have it.
Coming out of I/O, it’s pretty clear that Google is fully at peace with ending any last vestiges of free content depending on search traffic. Ten blue links that direct users to external websites have served their purpose and in Google’s eyes, are no longer the best form factor for efficiently satisfying a user’s query and maximizing their own ARPU. The malthusian trap is worth the risk. Google does this matter of factly with malice towards none but with charity towards none as well.
But ironically, the thesis for the endurance of the open web is similar to the case for its demise, with one key twist. As agents come to massively outnumber humans online, there will be a Cambrian explosion of new demand for readily accessible content. This force will be stronger than the pull towards walled off centralization.
This idea is best encapsulated by Parallel Web Systems (and a few notable competitors such as Exa), founded by former Twitter CEO Parag Agrawal and valued at $2B. Parallel’s mission is simple– build the search and retrieval infrastructure that lets AI agents read the open web at scale.
As a result, Parallel has taken up the mantle as the champion of the “open web”, a term that has historically been co-opted by programmatic vendors to sell a chumbox of ad products from a bygone era that have rendered many media properties utterly impossible to consume. In Parallel’s recasting, the open web is “modern humanity’s living memory…and the ecosystem that made modern AI possible in the first place.”
Parallel’s thesis is effectively that for AI to fulfill its grand promise, the spirit of the open web must endure and find a new economic model to support openly indexed content that lives outside walled gardens. Without it, agents will become stale commodities.
Last month, Parallel launched a new product called Index, a crack at building a new economic layer for open web content. The platform shows content owners how agents discover, cite, and rely on their work and compensates them based on each source’s contribution to the agent’s output. It’s a manifestation of an idea Ben Thompson was all over a year ago when he argued that micropayments— a concept that fundamentally never made sense on a web built for human attention– could finally be viable on the agentic web.
Patrick Collison also has a good take:
What I’d emphasize is that while Index is nominally a publisher-side product, what it really ships is a new framework for influence. For a brand, economic value now increasingly accrues to your position in an agent’s reasoning at the moment of inference, not to a human’s attention at the moment of blog post read, ad impression or PDP view.
The end of the commodity internet
When it comes to which entities influence agents, it’s important to remember that the metric of 68% of searches now being “zero-click” disproportionately obliterates a specific segment of the open web ecosystem. SEO-dependent publications– which largely only existed in the first place to exploit keyword arbitrage– have been wrecked. Entities that publish timely news, analysis, entertainment or bizarre esoteria that serves real-time retrieval have been far more resilient than reported.
This last point is the most interesting to me because it lays out the path for the “open web” to once again return to its free-wheeling roots.
When monetizing web sites required reaching a critical scale of human eyeballs to sell ads to, you had to build within commoditized patterns that could both rank for SEO and serve necessary ads infrastructure. For 25+ years, the pendulum of web architecture has swung towards standardization, especially for commercial and marketing oriented builds.
If the business model gets rebuilt around serving AI at the point of inference, more of the “standardization” can exist at the invisible infrastructure level. Certain protocols will need to be in place for agents to retrieve information but effectively there’s a path for the open web to become a subliminal layer targeted mostly by agents on top of far bolder, wackier web interfaces built for humans.
I’m sympathetic to this case for open web renaissance for a few reasons:
- The arc of digital history generally bends towards content that can be accessed for free.
- There’s just a hell of a lot of pent up creative energy to build weird stuff right now with all of the new capabilities that AI unlocks.
- Simultaneously, there’s a vibe right now where a lot of people are just building for the pure love of the game, happy to worry about business viability later. The early blogosphere was crafted on a similar ethos.
- There’s massive uncertainty around what kind of information agents will ultimately gravitate towards as the models become more sophisticated. When nobody knows what the hell is going to work, the best strategy is for content creators and brands alike to maximize shots on goal.
Agents are weird…and perhaps the web will once again be too
If there’s a critical flaw in what Parallel is trying to build with Index, it’s that micropayments have also historically failed as a web business model precisely because they ascribe a rationality to the media business model that doesn’t exist. If media was a pure utilitarian purchase, most publishers would be extinct.
Quality media is largely sustained by people who buy the idea and self-actualization of being a New York Times, Atlantic or Stratechery reader more than the utility of actually reading. That is to say, they buy being perceived as erudite by their peers and pay hundreds of dollars for coffee table accessories. They buy the brand.
One popular theory of the future is that agents won’t be so sentimental. Agents, so the conventional logic goes, will rationally seek out the most relevant information to complete a given task and will pay fair market value for it, agnostic of “brand.” For media, that fair market value often doesn’t sustain the cost to create content.
Truth is, at this juncture, we have little idea how agents will ultimately hold a notion of brand preference. Hell, there may well be agents for maximally cost conscious consumers that render brand as more of a tax than a buying signal.
But a few things are likely to be true that point to agents building true brand associations:
- Agents will be increasingly trained to look for heuristic shortcuts that don’t swallow as much compute
- Agents will take on more quirky anthropomorphic tendencies as models get more sophisticated
- Influencing agents will become one of the most lucrative problems in tech. It will be far less straightforward that simply optimizing catalogs and stacking Reddit karma.
Taken together, it’s hard to understand how the increasingly complex context needs of agents will possibly be served if the web devolves into a morass of walled gardens, licensing deals and content marketplaces.
Perhaps it’s as simple as this— all signs point to the agentic web swinging us back towards things getting really weird again. And it’s the weird early 2000’s web, where people wrote entire blogs from the perspective of ficus trees, that so many of us long to see return.
If you have any questions, please be in touch. As always, thanks for reading.
— Mike