Changelog
Index releases, judge versions, weight changes, task-bank rotations, and our own corrections. This page is the record of the measurement instrument itself.
Index releases
v0.1
Agentic Search Index v0.1 · 2026-07-08
First versioned release: 9 providers on a 121-task bank, 3 repeats each. Judge gemini-2.5-pro (pinned), 94.1% agreement with human consensus (kappa 0.85, n=34) against dual human labels; the human ceiling is kappa 0.917. Full calibration detail on the methodology page.
Self-correction ledger
Our own errors get logged here in the open: scoring bugs, judge regressions, retracted results, and definition mistakes, each with what changed and why.
No corrections yet — they'll be listed here, not hidden.