Judged questions
The public questions, with how many answers were marked by two judges and how often the judges were 3 or more points apart. Open a question to read both judges' scores and reasons for every model. Private questions are never listed.
| Question | Category | Starts with | Answers judged twice | Disputed | Mean gap | Mean judge score |
|---|---|---|---|---|---|---|
| gotchas-001 | Famous gotchas | How many times does the letter 'r' appear in the word 'strawberry'? Give your final answer on the last line in the form: ANSWER: <value> | 0 | — | — | — |
| gotchas-010 | Famous gotchas | You pick door 1 of 3 doors; one hides a car, the other two hide goats. The host, who does NOT know where the car is, opens door 3 at random… | 0 | — | — | — |
| gotchas-013 | Famous gotchas | A doctor gives you 3 pills and tells you to take one every 30 minutes, starting now. How many minutes after taking the first pill do you tak… | 0 | — | — | — |
A dash means no answer to the question was scored by two judges. Scores are on the judges' own 0 to 10 scale; withheld models are left out of every figure.