Who keeps the agentic web's index working, and how they would get paid. Lewis Tham, Unbrowse, October 2026. Second version; the first appeared as a draft in July 2026.
Status. This is a design paper. As of 12 October 2026, nothing in it pays anyone, no part of the network described here is live in production, and Unbrowse has no token. The section "What is built, and what is not" says exactly where each piece stands.
The problem. Agents need to use websites, and on 362 of 425 sites we examined, no official API does the job an agent was sent to do. The site's own pages already call internal APIs that would. Somebody has to find those calls, keep them working when the site changes, and check that nobody is lying about them.
Why it is hard. The cheapest place to learn a route is a real person's browser, which has passed every bot check and costs nothing extra. The most expensive place is a cloud data centre, which gets challenged. But a stranger's browser cannot be trusted, and paying strangers usually means minting a token that they then sell.
The answer. Trust the replay, never the recorder. Pay contributors per call, in dollars, split by how much each of them improved the route. Override bad work instead of slashing deposits. No stake, no token and no land grab are needed, and every one of those choices reverses something the first version of this paper proposed.
What changed since the first version
In July we drafted a paper called The Economics of Proof of Indexing. It promised contributors 70% of every call, a 2× bonus to whoever indexed a domain first, and payment over x402 on Solana and Base, and a companion paper proposed that maintainers post a token bond that would be slashed when their routes failed.
Three months of building changed most of that.
| First version (July 2026) | This version | Why |
|---|---|---|
| Contributor earns 70% of each call | Site 50%, contributors 40%, platform 10% | It is the site's server and data. A network that pays sites gives them a reason to stop blocking agents |
| First indexer of a domain earns 2× | Each contributor earns in proportion to the improvement they made | A first-mover bonus pays for arriving early, so it invites squatting. An improvement can only be earned by making the route better |
| Highest-reliability version takes all traffic | Every contributor to a route shares its calls, by improvement | Winner-take-all erases the person who created the route the moment someone fixes it |
| Maintainers stake a token bond, slashed on failure | Wrong work is overridden and loses its pending earnings | A bond is a capital barrier, and a bond in a token is only as strong as the token's price |
| Passive capture of everything you browse | Your own tasks only, unless you install a separate program and opt in | Recording a person's whole browsing history is a privacy problem we will not ship |
| x402 per-call settlement, described as live | Card payments first; any per-call stablecoin payout behind a decision we have not taken | x402 was never live in our service. The first version said otherwise, and it was wrong |
The parts that survived are the ones that mattered: pay only for use, settle in a stable currency, and treat a route that works as the only proof that counts.
The first call is the expensive one
An agent asked to check a price on a site nobody has indexed has one option: open a browser, render the page, find the field and read it. In the 94-domain benchmark in our April 2026 paper, a cached route answered in 950 ms on average against 3,404 ms for a Playwright browser. The same paper estimated browser rediscovery at $0.10 to $0.53 a task, depending on how many steps and retries it takes. Once a route is known, every later agent can call it directly for a fraction of a cent.
That gap is the whole economy. A rational agent uses a shared route whenever its fee is below the cost of rediscovering it. At a $0.01 reference fee against $0.10 to $0.53, there is room for the route's contributors, the site and the platform to all be paid, and for the agent to still save most of its cost.
The $0.01 is a design reference. It is not what Unbrowse charges: the live price is free during the beta, up to 500 verified calls a month, then $10 per 10,000 verified calls.
The index this requires does not exist yet. Our registry answered 166 of 200 fresh inputs (83%) in a September sample, against a bar of 85% we have not yet reached. On 100 sites we had never seen, a generic script replayed a new query with no browser on 42. When we crawled the Tranco top 10,000 with plain HTTP, 57 sites gained a newly published route and 195 already had one. Most of the rest failed to load for the crawler or had no search it could find, 254 failed the clean replay, and 474 were skipped because robots.txt asked us not to.
The gate is strict on purpose. Publishing a route that does not work is worse than publishing nothing.
Our cloud is the wrong place to learn
Running 20 cloud indexers costs us about $600 a month, and that bill grows with every site we add. It also fails where it matters: in a September test, even residential proxy exits got through on only 29 of 40 hard sites, and they are billed per byte.
A person's own browser has none of these problems. It has already passed the site's checks, it holds the person's sessions for the sites they use, and it costs us nothing. So the design moves learning to contributors:
- Your own tasks would run in your own browser. That is the intended default, and it never touches anyone else's work.
- Network work would run in a separate program you install and switch on. It uses its own cookieless browser context, visits public sites only, honours robots.txt, never signs in, and holds an exclusive lease per site so a thousand contributors cannot hammer one site at once.
- Jobs carry dummy inputs. A requester who wants "flight search on site X" sends the site and the intent, never their real dates or destination. They run the finished route with their real values on their own side.
As of 12 October 2026, neither of the first two runs in production, and learning happens in Unbrowse's cloud browser.
The recording never leaves your machine
The first design had contributors return raw recordings for us to compile. We dropped that. A recording holds the person's cookies, what they typed and the pages they saw, and it does not need to travel.
In this design, contributors compile on their own machine with the same compiler the service runs, scrub the result with the same public scrub, and submit only the finished route object plus a hash of the recording it came from. Our tests check that the submitted object contains none of the cookie, the page text or the person's task.
We then treat every submitted route as untrusted data. A static check refuses a route that contacts another domain, performs a write, carries a session value, runs host code, or contains any text a contributor wrote, because a description an agent reads is a place to plant instructions. Every name and description is regenerated from the route's structure. Passing that check is necessary and never sufficient.
Trust the replay, not the recorder
Checking a route costs one HTTP call. Learning it costs a browser and minutes. Bitcoin rests on the same asymmetry: work is expensive to do and cheap to check.
A submitted route is replayed by a committee of three validators who did not record it. The committee is drawn from a random seed that did not exist when the route was submitted: the next signed checkpoint of the log. Seats are one per owner and one per network, so one person running many machines gets one seat, and the route's own contributor can never sit. Each validator replays the route with fresh inputs and builds a shape report: the status, the content type, a fingerprint of the response's keys and types, whether the declared fields are present, and whether a nonsense query comes back empty. Prices and timestamps change from call to call, so validators compare shapes, not values.
Each validator commits to a hash of its report before anyone reveals, so nobody can copy the first answer. Two matching passing shapes out of three publish the route.
Publishing goes into a public, append-only log in the format Certificate Transparency logs use, with signed checkpoints anyone can verify and independent witnesses who cosign them. A route counts as public only when logs run by two different operators include it. That is the rule browsers apply to certificates, and it means no single operator, including us, can quietly publish or censor a route. Nodes keep a signed snapshot of the routes they use, resolve agent requests locally and replay them from their own address. A takedown is a signed tombstone that every honest node applies as soon as it catches up.
The live registry already applies a simpler version of this rule: a route is published only after a scrubbed copy, with no cookies, answers fresh queries and fails a nonsense one. The committee and the two-operator log are the network version, and as of 12 October 2026 they exist only in tested code.
Pay for the delta
Here is the change at the centre of this version. The first draft paid "the indexer" of a route. But routes are built by several people over time: someone creates one, the site changes, someone else fixes it, a third person makes it faster. Paying only the first, or only the latest, is unfair to everyone else.
So each accepted version of a route has a measured value per call: its success rate across the committee's replays, discounted for cost and latency. A contributor's delta is how much their version raised that value. Every call's contributor share is split among everyone who contributed to that route, in proportion to their deltas, with older deltas weighing a little less so that ongoing upkeep earns.
| What happens | Delta | Effect |
|---|---|---|
| Alice creates a route; the committee measures 0.9 | 0.9 | Alice earns the whole contributor share while she is alone |
| The site changes and the route breaks | none | Nobody earns while no call succeeds |
| Bob fixes it back to 0.9 | 0.9 | Alice and Bob share the calls, Bob slightly more because his delta is newer |
| Carol makes it faster, 0.9 to 0.95 | 0.05 | Carol earns a small share, matching what she added |
| Dave copies Alice's route | 0 | Nothing: a copy adds nothing over what exists |
| Eve splits one fix into ten small ones | the same total | The same pay: deltas add, so splitting gains nothing |
Exhibit 1. How one route's contributor share would be divided. Values are illustrative; the measurement is the committee's.
Three properties fall out of this rule without extra machinery. Copies earn nothing, which is how we stop duplicates. Unused routes earn nothing, because pay follows calls. And the person who first built a route keeps earning for as long as what they built is still in use, while a fixer is paid for restoring value the network had lost. The original author gets the first 48 hours to fix their own route before it opens to everyone.
We learned this the hard way. A predecessor design of mine in 2024 rewarded novelty, and my own note on it at the time was that people could feed it creative rubbish. Paying for improvement in something callers use is the opposite incentive.
Where the money would come from
All of it would come from callers. Every verified call would carry a fee, and our design reference is $0.01:
- $0.005 to the site whose server answered, held in a public per-domain escrow from the first call. The owner claims it by publishing a DNS record, and the same record can opt the domain out of automation entirely. Unclaimed escrow after 12 months funds fixes for that site's own routes.
- $0.004 to the route's contributors, split by delta.
- $0.001 to the platform, half of which goes to a pool that funds advances and public work.
A failed call costs nothing. Payouts would be settled once per weekly epoch: the service publishes a Merkle root of who is owed what, logged so that anyone can recompute it from the public receipts, and each contributor claims their line with a short proof. The design pays in a dollar stablecoin; as of 12 October 2026 we have not chosen the network it would settle on, and nothing settles. Caller identities in the published receipts are salted per epoch, so a caller cannot be followed from one week to the next.
Contributors pay their costs up front and earn over months, so the pool would offer advances: cash at verification against a route's expected use, repaid from that route's own earnings. If the route earns less than the advance, the pool takes the loss, not the contributor. That keeps the pool honest about which routes it funds, because it only advances against requests that actually went unanswered.
Override, not slashing
The companion paper proposed that maintainers post a token bond and lose part of it when their routes failed. Its own simulation, which modelled 24 months, found something instructive: with rational maintainers, nobody was ever slashed. The threat of losing money was enough to keep routes maintained.
Per-use pay creates the same threat without asking anyone to lock up capital. A broken route stops earning, so its contributors have every reason to fix it, and if they do not, someone else earns the fix. Work that turns out to be false is overridden: a majority of independent replays, or a better version, replaces it, and the contributor loses everything still pending from that route plus most of their reputation. Earnings stay pending for one epoch before they pay out, so an override always lands before the money leaves.
What replaces the bond is time. Committee seats and lease limits depend on reputation, which takes months of verified work to build and is lost at once. A new identity starts at zero, so an attacker with a thousand fresh keys has a thousand keys with no seats. Coordinators and gateways, which hold other people's leases and route their calls, still post a bond. Ordinary contributors and validators do not.
Do we need a token?
No, and Unbrowse does not have one. Callers would pay in dollars and contributors would be paid in dollars. With override in place of slashing there is nothing left for a token to secure. A token paid as a reward would be sold, because contributors pay for electricity, bandwidth and their time in dollars. Selling it is not disloyal; it is arithmetic.
So this design has no token, and nothing in it depends on one.
The objection: isn't this scraping?
It reads public pages the way the site's own frontend does, and it asks first. Every job checks robots.txt before the lease and again at run time. Network work is reads and search forms only: no sign-ins, no purchases, no writes. A site that wants out publishes one DNS record and every route on its domain is removed. A site that wants in publishes the same record with a payout address and would receive half of every call.
Most of the web's "no" to agents comes from their being a cost with nothing in return: they skip the pages, the ads and the upsells that pay for the site. Sending half of every call's fee back to the site is the most direct answer we know to that.
What is built, and what is not
Status as of 12 October 2026.
Built and tested, not deployed. The protocol core is written and tested as pure code: local compile and the submission check, the log with two-operator publishing, signed snapshots and local resolve, per-site leases, delegated-learning escrow, committees with commit-reveal, the fee split and site escrow, epoch payout roots, and bounty pricing. Fifteen of its seventeen acceptance stories pass, with 159 tests, including one test that runs a route from an unanswered request to a paid, published and finally removed route. The two open stories are switching on card payments and running the network on real machines.
Designed, not built. The delta split, override, advances and reputation-only committee seats described above are the next change to that core. The version in our tests still splits between a first indexer and a maintainer, still weights seats by bond, and still slashes bonds.
Not live in production. A first contributor node that runs index jobs is merged behind a switch and runs only as a staging pilot. No contributor has been paid, no site has been paid, and no payout of any kind has been made.
The limits
There is no revenue yet. Unbrowse is free during its beta, card payments are not switched on, and revenue is $0. Until callers pay, the pool is empty. In the design, contributors would earn points until it is funded; whether and how points convert is one of the questions for the legal review below. No mechanism in this paper creates demand; it only distributes what demand pays.
Measuring value is the hard part. A route's success rate wobbles from replay to replay, and a small fake "improvement" must not be paid. The design sets a minimum delta and measures over enough replays, but we have not yet run it against real contributors.
Time is a weaker stake than money against a patient attacker. Someone willing to build reputation for months before cheating can do damage the first version's bonds would have priced. Random seats, one per owner, sampled proofs that a response really came from the site, and open challenges limit it. They do not eliminate it.
The index is below its own bar. 83% against 85% on known sites, and 42 of 100 on new ones. Moving learning to real browsers is meant to raise both. That is a prediction, and the first test is small: post 20 real unanswered requests as paid jobs and compare cost and success with our own cloud on the same list.
Legal review comes before money moves. Paying strangers for delegated work, points, advances and anything resembling an investment each need a lawyer's reading before they go live.
Closing
Google indexed the web for people and kept the value. An index for agents can be kept working by the people who use it, paid by the agents that call it, with half of every call going back to the sites that make it possible. The rule that decides who earns is short enough to fit on one line: you are paid for what you made better, every time someone uses it.
The part of this that runs today is the learning and the replay. You can use it through the remote MCP server at https://unbrowse.ai/mcp; setup is in the docs. For the plain-language version of the network idea, read Mine the Internet.
References
- Tham, L., Garcia, N. M. G., Hahn, J. "Internal APIs Are All You Need." arXiv:2604.00694, April 2026. https://arxiv.org/abs/2604.00694
- Laurie, B., Langley, A., Kasper, E. "Certificate Transparency." RFC 6962, 2013. https://www.rfc-editor.org/rfc/rfc6962
- C2SP, transparency log checkpoint and signed-note formats. https://github.com/C2SP/C2SP
- Your Agent Won't Get an Official API. We Checked 425 Sites. (the 425-site survey, 2026-10-05)