Measured,
not promised.

firecrawl publishes only with its sentence: incomplete — our harness overwrote 7 valid payloads; Firecrawl answered every question on the first fetch.

claude web search is MISSING, not passed. No key in that environment. A row silently absent reads as one that passed, so it stays in the table with no number.

exa is labelled “web search” — the only tool its public endpoint exposes. Its code-retrieval product is unmeasured.

Our own two non-immutable URLs, named: https://github.com/sponsors/wevm (a sponsor link inside wagmi's own docs, which we quoted) and https://github.com/wevm/viem/issues/2484 (the name of a test case in viem's source).

Counted over cached bytes, on 16 questions, and it is one count, not a verdict. The audit issues none and none is computable from it.

The prediction we registered was wrong, and it goes first. We wrote “we expect to be the most expensive arm.” Measured with one tokenizer over full payloads, our default row is 3,071 median tokens — under three of the arms here. The earlier adversarial run’s “most expensive of four” priced a different unit and read our content-only figure against everyone else’s full payload. It licenses no cost claim in either direction.

The two web3ctx rows are ONE finding and are never shown apart. Called the way a client that does not know our argument names calls us, we catch 1 of 6 traps. With intent: "integrate" — the value that reaches the human-validated recipe — 6 of 6. The gap is the argument, and the weaker row is ours to publish.

context7’s 0/0 is its own sentence: its payloads contain no URLs at all. That is a different fact from citations that fail to pin, and collapsing the two would misreport it.

T6 carries a permanent disclosure. Our recipe gained the ERC-4626 rounding matrix on 2026-08-20 after losing this trap in the earlier adversarial run. A benchmark that quietly re-runs a question after fixing its answer is measuring its own patch — so this sentence travels with the row for as long as the row exists.

VOID — 5 row(s), listed: firecrawl-dev/Q06, firecrawl-dev/Q07, firecrawl-dev/Q08, firecrawl-dev/Q09, firecrawl-dev/Q10. All 5 are a real provider quota reached during the run. Excluded from numerator and denominator; that arm publishes as 11 ran / 5 void, never as a fraction of 16 and never as a zero.

claude web search caught 6 of 6 — matching our best row and beating our default one — at 17,163 median tokens against our 3,071, with 6/404 immutable citations against our 316/324. Both compositions belong to that comparison; either number alone tells a different story than the pair does.

⊘ NO VERDICT. n=16 against the registered floor of 393 (D17). Bootstrap CIs are in the result document; a confidence interval does not repair a small n, it states it.

The abstention count rides with the number, always — 20 / 147 in both arms. Precision computed only over answered questions would rise every time the server declined to answer, which would reward silence.

Not comparable to a published code-retrieval precision. CodeGrep and its kin retrieve within one repository: project selection never enters their metric, so under our definition their BM25 would read 1.000 — including the configuration measured at 0.375. The two numbers measure different stages, and ours was pre-registered as an upper bound where a pass is “suggestive, never proof”.

Composition: measured live against the deployed surface, macro-averaged, wildcard-version, both arms on the same questions. The two rows differ by one shipped mechanism and by nothing else — the identical abstention counts are the evidence for that.