unbrowseA worry I hear about Unbrowse: "It only works if you index everything."
I'm Lewis Tham, founder of Unbrowse and first author of "Internal APIs Are All You Need". Unbrowse is an index of what websites can do: the requests behind their buttons, learned once and replayed by agents.
Netcraft's July 2026 survey counted about 1.5 billion sites. That's hostnames, and plenty of them are parked or dead, but the point stands. We won't index them all. Nobody will.
That's fine. Here's why.
A miss costs one slow call
Here's what happens when an agent asks for a site we don't have.
Agent needs a site
your own learned routes miss
public registry miss
browse the site once (cloud browser, LLM, retries)
learn the request under the button
save the route
Next call
replay over HTTP
The first call is slow. On our own demo booking site, browsing for the result takes a median 36.7 seconds. Replaying the learned request takes 1.97.
Learning also costs more. In our April 2026 paper, a task done by driving a browser cost $0.53, against $0.005 through a cached route: 106x.
So a miss is a slow, expensive first call, not a broken product.
And it happens once. The route the first agent learned is there for the second. Misses fix themselves, one request at a time, in exactly the places someone asked.
Demand is lumpy
An index built for completeness spends the same on every site. Requests don't arrive that way.
Agents ask for the sites their users use. A booking page, a store's search, a council's lookup, a supplier portal. The same ones, over and over.
I expect a small slice of sites to carry most of the calls. That's an expectation. We haven't measured it for agent traffic.
But the order of work follows either way: index what gets asked for, and let the long tail learn itself on first contact.
Routes nobody calls are upkeep
Indexing ahead of demand keeps costing after you've paid to learn the route.
Sites change. A field gets renamed, a request picks up a new header, search moves to a different endpoint. Every route you hold has to be re-checked or it quietly rots. Our registry re-runs its public tools on a schedule for exactly that reason, and pulls the ones that keep failing out of search.
A route nobody calls is maintenance with no one paying for it.
Index the web ahead of demand and you own a huge, decaying asset that you pay to keep warm.
Google never indexed everything either
Search had the same objection and the same answer.
Google says so in its own documentation: "Indexing isn't guaranteed; not every page that Google processes will be indexed." No search engine holds the whole web.
People judged search on one thing: did their query find something?
The metric is hit rate
Ours has the same shape. Hit rate: the share of real agent requests that find a working route on the first call.
We haven't measured it yet.
What we have measured is the supply side: how good routes are once we have them.
Measured (supply side)
100-site test: sites that replay a new query
without a browser 42 of 100
Public registry sample: tools that answered
a fresh input 166 of 200
Registry sample false successes 0
Not measured yet (demand side)
Hit rate on real requests ?
The 100-site test ran on 2026-09-23 with a generic scripted agent: find the search box, type, press Enter, twice, then replay with a third query. 42% replayed without a browser. The registry sample ran on 2026-09-26: 83% of 200 public tools answered an input they'd never seen, and none of the failures pretended to succeed.
Both describe routes we already have. Neither tells you how often a real request finds one. That takes real traffic with every miss logged against it.
The plan: every miss becomes a bounty
A miss log is a to-do list written by demand.
Nothing in this section is built. The plan is that every miss becomes a bounty, and busier misses pay more. A bounty that stays unfilled rises in value until someone picks it up, the way Bitcoin's difficulty adjusts to keep blocks coming. Indexing nobody asked for gets paid only when someone uses it.
That would pay people to close the gaps that cost hit rate, and nothing else. Today nobody is paid for routes. Unbrowse has no contributor payouts and no token.
For one customer, the web is small
The coverage question shrinks again when you look at a single customer.
An agent product doesn't need the web. It needs the sites its users actually touch. Our planning assumption is 10 to 50 sites per customer. It's an assumption, not a measurement.
That's a list you can read in one sitting. We can index it, check every route, and measure that customer's hit rate directly.
Where this actually is
As of 12 October 2026. Live: learn a site's request once in Unbrowse's cloud browser, replay it over HTTP, and count an outcome as a success only when it's verified, not when it returns a 200. A public registry of scrubbed, read-only routes, each published only after a clean replay with no cookies, and re-checked on a schedule. And unbrowse.index, which points our own agent at one site you name, learns its core read-only capabilities, and keeps each one only after a browserless replay proves it.
Plan, not built: bounties, demand-weighted payouts, and learning jobs run on other people's machines (that last one is in a staging pilot, not in production).
Not measured: hit rate.
So help us measure it on your own list. Connect the MCP server at unbrowse.ai/mcp, run your agent against the sites it actually needs, and index the ones that miss. Setup is in the docs. If you want the argument for why those routes exist in the first place, start with The Agentic Web Will Not Wait. For how a route earns its place in the public registry, read Trust the Replay, Not the Recorder.