Judged questions
The public questions, with how many answers were marked by two judges and how often the judges were 3 or more points apart. Open a question to read both judges' scores and reasons for every model. Private questions are never listed.
| Question | Category | Starts with | Answers judged twice | Disputed | Mean gap | Mean judge score |
|---|---|---|---|---|---|---|
| instruction-004 | Instruction following | List exactly 3 benefits of drinking water. Format each one as a bullet point starting with "- ". Your whole response must be at least 10 wor… | 228 | 2 | 0.5 | 8.1 |
| instruction-008 | Instruction following | Return a JSON object with the keys "name" and "age" describing a fictional character. Output raw JSON only: no code fences, no commentary be… | 228 | 1 | 0.7 | 7.9 |
| instruction-014 | Instruction following | Write a formal reply in exactly 1 paragraph with at most 3 sentences. Wrap the whole reply in double quotation marks (the first character is… | 228 | 78 | 1.8 | 5.2 |
A dash means no answer to the question was scored by two judges. Scores are on the judges' own 0 to 10 scale; withheld models are left out of every figure.