Trust
A score is a measurement, and a measurement has limits. These pages put the limits next to the numbers: how much a score moves when the battery is run again, how often an answer was cut off, which inference backend produced each run, and which trick questions models fell for.
- 5.3points: the median move in a score between 6 repeat runs of 6 models
- 2,958answers cut off at the length limit, across 199 of 228 models
- 2inference backends behind the published runs
- 3paid judge models: openai/gpt-5.6-luna, anthropic/claude-haiku-5.5, deepseek/deepseek-v4.1-flash
Reproducibility
How much a model's score moves when the same battery is run again.
Truncation
Answers that hit the length limit, and what a second chance changes.
Backends
Which inference backend produced each run, and why speeds compare only within one.
Gotcha questions
Every trick question in the battery and which models fell for it.
Errata
What we withheld or corrected, when, and the run that replaced it.
Reproduce an answer
The tag, digest, options and prompt hash behind one stored answer, and what cannot be repeated.
The judges
Where the two judge models agree and where they split.
Methodology
How the battery is built, scored and judged, and what it cannot tell you.