Trust / Truncation
Truncation
Each question has a limit on how many new tokens a model may write. An answer that runs into the limit is cut off, and what it had written so far is what gets scored. Cut-off answers are asked once more at double the limit, and the result is shown beside the score.
The headline score never changes. The Overall on the leaderboard is always the score at the normal length limit; the second-chance figure below is extra information and is never blended in. How the second chance works is in the methodology.
- 2,958answers cut off at the length limit (questions that count toward Overall)
- 2,863of them got a usable second answer at double the limit
- 199of 228 models had at least one answer cut off
Model by model
| Model | Overall | Cut off | Re-asked | Overall with longer limit | Change |
|---|---|---|---|---|---|
| Magistral Small (2506) | 49.8 | 119 | 89 | 65.8 | +16.0 |
| Qwen3 VL 2B Thinking | 53.7 | 106 | 106 | 59.4 | +5.7 |
| Codegemma 2B Code V1.1 | 15.3 | 92 | 92 | 15.9 | +0.6 |
| Exaone Deep 2.4B | 54.7 | 72 | 71 | 62.6 | +7.9 |
| Deepscaler 1.5B Preview | 52.1 | 60 | 58 | 55.4 | +3.3 |
| Deepseek R1 1.5B Qwen Distill | 48.9 | 60 | 59 | 53.1 | +4.2 |
| Granite4.2 8B | 81.9 | 60 | 58 | 86.9 | +5.0 |
| Olmo-3-7B-Think | 77.7 | 56 | 56 | 88.4 | +10.6 |
| Smollm2 135M Instruct | 20.1 | 55 | 52 | 20.5 | +0.4 |
| Lfm2.5 8B A1B | 75.7 | 52 | 52 | 78.8 | +3.1 |
| Deepcoder 14B Preview | 74.5 | 51 | 51 | 85.0 | +10.5 |
| Smallthinker 3B Preview | 61.1 | 48 | 48 | 65.7 | +4.6 |
| Deepcoder 1.5B Preview | 55.8 | 47 | 47 | 62.9 | +7.1 |
| Granite4.2 3B | 78.2 | 46 | 45 | 83.9 | +5.7 |
| Lfm3 1.7B | 70.2 | 45 | 45 | 77.0 | +6.7 |
| Qwen3-4B-Thinking-2507 | 83.5 | 45 | 45 | 94.7 | +11.2 |
| Qwen 1.8B Chat V1.5 | 24.7 | 43 | 42 | 24.7 | 0 |
| Exaone Deep 7.8B | 65.7 | 42 | 41 | 68.8 | +3.1 |
| DeepSeek-R1-0528-Qwen3-8B | 79.6 | 40 | 40 | 86.6 | +7.0 |
| Phi4 Mini Reasoning 3.8B | 70.9 | 37 | 37 | 74.4 | +3.5 |
| Openthinker 7B V2 | 76.8 | 36 | 36 | 83.0 | +6.2 |
| Qwen3 VL 2B Instruct | 77.4 | 34 | 33 | 80.5 | +3.1 |
| Lfm2.5 Thinking 1.2B | 74.3 | 32 | 32 | 78.1 | +3.8 |
| Medgemma1.5 4B IT | 71.6 | 31 | 28 | 72.7 | +1.2 |
| Stable Code 3B Instruct | 22.8 | 31 | 29 | 22.8 | 0 |
| Qwen3-VL-4B-Instruct | 89.9 | 30 | 30 | 90.9 | +1.0 |
| Gemma3n E2B IT | 79.9 | 29 | 29 | 81.0 | +1.2 |
| Spark-X2.6-4B | 85.0 | 29 | 29 | 89.6 | +4.6 |
| Phi 2.7B Chat V2 | 39.9 | 28 | 28 | 40.3 | +0.4 |
| DeepSeek-R1-Distill-Qwen-7B | 73.6 | 27 | 27 | 77.7 | +4.0 |
| GLM-4.5-Air | 92.8 | 27 | 24 | 94.3 | +1.6 |
| Granite4.2 30B | 92.8 | 27 | 26 | 96.9 | +4.1 |
| YI Coder 1.5B Chat | 26.5 | 27 | 27 | 26.4 | -0.1 |
| Smollm2 360M Instruct | 28.1 | 26 | 23 | 28.2 | +0.1 |
| Medgemma 27B IT | 90.2 | 25 | 25 | 92.8 | +2.6 |
| Flint 40B A2B | 89.0 | 24 | 24 | 92.5 | +3.5 |
| Opencoder 1.5B Instruct | 31.9 | 24 | 22 | 31.9 | 0 |
| gemma-3-12b-it | 92.7 | 23 | 20 | 93.1 | +0.3 |
| gemma-3-27b-it | 95.8 | 23 | 21 | 95.6 | -0.1 |
| gemma-3-4b-it | 82.1 | 23 | 19 | 83.2 | +1.1 |
| Gemma3 1B IT | 56.8 | 22 | 22 | 56.5 | -0.3 |
| Granite3.1 Moe 1B Instruct | 43.6 | 22 | 22 | 44.3 | +0.6 |
| Qwen2 0.5B Instruct | 36.0 | 22 | 21 | 37.0 | +0.9 |
| Dolphin Phi 2.7B V2.6 | 44.0 | 21 | 20 | 44.0 | -0.0 |
| Qwen2 Math 1.5B Instruct | 33.1 | 21 | 21 | 33.0 | -0.1 |
| Qwen3-4B-Instruct-2507 | 90.3 | 21 | 21 | 92.6 | +2.3 |
| Qwen3.6-35B-A3B | 94.6 | 21 | 21 | 97.8 | +3.2 |
| Qwen3.8-27B | 95.4 | 21 | 21 | 98.3 | +2.9 |
| Gemma 3n E4B IT | 86.1 | 20 | 20 | 86.8 | +0.6 |
| Nanbeige_Nanbeige4.3-33B | 90.4 | 20 | 19 | 90.9 | +0.5 |
| Qwen3.5-27B | 96.2 | 20 | 20 | 98.5 | +2.3 |
| Qwen3.5 2B | 75.1 | 19 | 19 | 76.2 | +1.1 |
| Qwen3.6-27B | 95.1 | 19 | 19 | 98.2 | +3.1 |
| medgemma-4b-it | 74.3 | 18 | 18 | 74.4 | +0.1 |
| Qwen3.5-35B-A3B | 94.8 | 18 | 18 | 96.8 | +2.0 |
| Translategemma 12B IT | 89.6 | 18 | 18 | 90.3 | +0.6 |
| Ministral 3 8B Instruct 2512 | 88.7 | 17 | 17 | 88.9 | +0.1 |
| Nemotron Cascade 2 30B A3B | 85.6 | 17 | 16 | 85.5 | -0.1 |
| Qwen3.5-9B | 92.0 | 17 | 17 | 95.2 | +3.2 |
| QwQ 32B | 92.6 | 17 | 14 | 97.7 | +5.0 |
| Sage-A1.5-Q4_K_M | 94.5 | 17 | 17 | 97.4 | +2.9 |
| Falcon3 1B Instruct | 44.3 | 16 | 15 | 44.6 | +0.3 |
| Gemma4 E4B IT | 95.5 | 16 | 16 | 97.5 | +2.0 |
| Ministral 3 14B Instruct (2512) | 92.7 | 16 | 16 | 94.0 | +1.3 |
| Qwen2.5 Coder 0.5B Instruct | 32.8 | 16 | 14 | 33.1 | +0.2 |
| Qwen3-Coder-Next | 96.4 | 16 | 16 | 98.2 | +1.8 |
| Qwen3-VL-30B-A3B-Instruct | 92.9 | 16 | 16 | 95.7 | +2.8 |
| Qwen3.5-4B | 88.7 | 16 | 16 | 91.4 | +2.6 |
| Deepseek R1 8B Llama Distill | 81.7 | 15 | 15 | 84.4 | +2.7 |
| DeepSeek-R1-Distill-Qwen-14B | 85.3 | 15 | 15 | 88.9 | +3.6 |
| DeepSeek-R1-Distill-Qwen-32B | 89.1 | 15 | 9 | 92.2 | +3.1 |
| Gemma4 E2B IT | 93.8 | 15 | 15 | 95.0 | +1.2 |
| NVIDIA Nemotron 3 Nano 30B-A3B | 83.9 | 14 | 14 | 83.9 | 0 |
| Qwen2 1.5B Instruct | 49.7 | 14 | 14 | 49.4 | -0.3 |
| Qwen3-30B-A3B-Instruct-2507 | 95.0 | 14 | 14 | 96.4 | +1.4 |
| Sailor2 1B Chat | 23.5 | 14 | 12 | 23.6 | +0.2 |
| Phi-3-mini-4k-instruct | 63.4 | 13 | 13 | 63.6 | +0.2 |
| Qwen3-VL-8B-Instruct | 94.9 | 13 | 13 | 95.7 | +0.8 |
| GLM-4.7-Flash | 88.9 | 12 | 12 | 90.6 | +1.7 |
| Ministral 3 3B Instruct 2512 | 80.9 | 12 | 12 | 81.2 | +0.3 |
| Qwen2.5 0.5B Instruct | 43.3 | 12 | 12 | 43.2 | -0.1 |
| Qwen3-VL-32B-Instruct | 94.7 | 12 | 12 | 96.5 | +1.8 |
| Codellama 13B Instruct | 52.5 | 11 | 11 | 52.5 | 0 |
| gpt-oss-20b | 95.0 | 11 | 11 | 97.0 | +1.9 |
| Nemotron 3.5 Bolt E4B | 90.3 | 11 | 11 | 93.7 | +3.4 |
| Olmo 3.1 32B Instruct | 89.6 | 11 | 11 | 93.1 | +3.5 |
| Olmo-3-7B-Instruct | 82.9 | 11 | 11 | 83.9 | +1.1 |
| Granite Code 3B Instruct | 38.0 | 10 | 10 | 37.6 | -0.4 |
| Muse Glimmer 30B | 97.9 | 10 | 10 | 99.3 | +1.4 |
| Phi-3.5-mini-instruct | 67.1 | 10 | 10 | 66.9 | -0.1 |
| Phi3 3.8B Mini 128K Instruct | 62.9 | 10 | 9 | 63.1 | +0.2 |
| Qwen 0.5B Chat V1.5 | 16.1 | 10 | 8 | 16.1 | 0 |
| Qwen2.5-VL-32B-Instruct | 89.7 | 10 | 10 | 89.7 | 0 |
| Qwen3-14B | 91.7 | 10 | 10 | 92.3 | +0.6 |
| Devstral Small 2 24B Instruct (2512) | 92.1 | 9 | 9 | 93.3 | +1.2 |
| Falcon2 11B | — | 9 | 9 | — | — |
| Granite3 Moe 1B Instruct | 35.5 | 9 | 9 | 35.3 | -0.2 |
| Nemotron 3.5 Lightning 30B A3B | 92.8 | 9 | 9 | 93.6 | +0.8 |
| Qwen2.5 Coder 1.5B Instruct | 60.5 | 9 | 9 | 60.7 | +0.2 |
| Qwen3-32B | 93.6 | 9 | 9 | 94.9 | +1.3 |
| Qwen3-4B | 89.2 | 9 | 9 | 89.8 | +0.6 |
| Qwen3-8B | 91.5 | 9 | 6 | 91.9 | +0.3 |
| YI 6B Chat V1.5 | 58.0 | 9 | 9 | 57.9 | -0.1 |
| Gemma 2B Instruct V1.1 | 41.9 | 8 | 8 | 41.9 | 0 |
| Llama-3.1-8B-Instruct | 81.0 | 8 | 8 | 81.0 | 0 |
| Qwen2.5-VL-3B-Instruct | 72.3 | 8 | 8 | 72.6 | +0.2 |
| Codellama 7B Instruct | 53.0 | 7 | 7 | 52.8 | -0.2 |
| Codeqwen 7B Chat V1.5 | 49.4 | 7 | 7 | 49.4 | 0 |
| Deepseek Coder 1.3B Instruct | 22.6 | 7 | 7 | 22.3 | -0.3 |
| Granite3 Moe 3B Instruct | 47.7 | 7 | 7 | 47.4 | -0.3 |
| Internlm2 7B Chat V2.5 | 67.4 | 7 | 7 | 67.7 | +0.4 |
| Nemotron3 33B | 86.2 | 7 | 7 | 86.2 | 0 |
| Qwen 4B Chat V1.5 | 30.2 | 7 | 7 | 30.1 | -0.1 |
| Qwen2-VL-7B-Instruct | 72.2 | 7 | 7 | 72.2 | 0 |
| Qwen3 0.6B | 44.0 | 7 | 7 | 43.7 | -0.3 |
| Granite3.1 Moe 3B Instruct | 59.8 | 6 | 6 | 60.4 | +0.6 |
| Lfm3 VL 2B | 72.2 | 6 | 6 | 72.3 | +0.1 |
| Phi3 14B Medium 128K Instruct | 78.0 | 6 | 6 | 78.4 | +0.5 |
| Qwen2.5 1.5B Instruct | 68.5 | 6 | 6 | 68.5 | 0 |
| Qwen3 1.7B | 76.0 | 6 | 6 | 75.8 | -0.2 |
| Qwen3-30B-A3B | 92.8 | 6 | 6 | 93.4 | +0.6 |
| Smollm2 1.7B Instruct | 47.1 | 6 | 6 | 47.1 | 0 |
| Falcon3 3B Instruct | 64.8 | 5 | 5 | 65.2 | +0.3 |
| gemma-4-26B-A4B-it | 97.8 | 5 | 5 | 97.8 | 0 |
| Granite3.2 2B Instruct | 63.8 | 5 | 5 | 63.8 | 0 |
| Internlm2 1.8B Chat V2.5 | 51.9 | 5 | 5 | 52.3 | +0.4 |
| Ornith 9B | 90.0 | 5 | 5 | 91.1 | +1.1 |
| SmolLM3-3B | 78.0 | 5 | 5 | 78.0 | 0 |
| Stablelm2 1.6B Chat | 24.4 | 5 | 4 | 24.2 | -0.2 |
| Apertus-8B-Instruct-2509 | 63.6 | 4 | 4 | 63.6 | 0 |
| Codegeex4 9B All | 68.7 | 4 | 4 | 68.7 | 0 |
| Deepseek V2 16B Lite Chat | 58.2 | 4 | 4 | 58.7 | +0.6 |
| Gemma 4 12B IT | 98.6 | 4 | 4 | 99.2 | +0.6 |
| Granite3 Dense 8B Instruct | 67.6 | 4 | 4 | 67.8 | +0.3 |
| Granite3.1 Dense 2B Instruct | 60.6 | 4 | 4 | 60.6 | 0 |
| Llama2 13B Chat | 45.3 | 4 | 4 | 45.3 | 0 |
| Llama3.2 1B Instruct | 58.0 | 4 | 3 | 58.0 | 0 |
| Mathstral 7B V0.1 | 68.0 | 4 | 2 | 68.0 | 0 |
| Mistral Small 3.2 24B Instruct (2506) | 92.9 | 4 | 4 | 94.3 | +1.4 |
| Nemotron Mini 4B Instruct | 52.0 | 4 | 4 | 52.0 | 0 |
| Ornith-1.5-9B | 89.0 | 4 | 4 | 89.6 | +0.6 |
| Qwen 7B Chat V1.5 | 44.1 | 4 | 4 | 44.1 | 0 |
| Command R7B Arabic 7B 02 2025 | 81.9 | 3 | 3 | 82.4 | +0.4 |
| gemma-4-31B-it | 99.3 | 3 | 3 | 99.6 | +0.3 |
| Granite3 Dense 2B Instruct | 58.3 | 3 | 3 | 58.3 | 0 |
| Granite3.3 2B | 68.6 | 3 | 3 | 68.5 | -0.2 |
| Granite4.1 30B | 92.3 | 3 | 3 | 92.6 | +0.3 |
| Llama-3.2-3B-Instruct | 75.7 | 3 | 3 | 75.7 | 0 |
| Mistral 7B Instruct V0.3 | 59.8 | 3 | 3 | 59.6 | -0.2 |
| Qwen2 Math 7B Instruct | 66.0 | 3 | 3 | 66.1 | +0.2 |
| Qwen3-Coder-30B-A3B-Instruct | 95.5 | 3 | 3 | 95.5 | 0 |
| YI Coder 9B Chat | 59.0 | 3 | 3 | 59.0 | 0 |
| Aya 8B 23 | 55.9 | 2 | 2 | 56.0 | +0.1 |
| Cogito 8B V1 Preview Llama | 83.9 | 2 | 2 | 84.0 | +0.1 |
| Deepseek Coder 6.7B Instruct | 34.7 | 2 | 2 | 34.7 | 0 |
| Deepseek Llm 7B Chat | 54.3 | 2 | 2 | 54.3 | 0 |
| Devstral 24B Small 2505 | 88.9 | 2 | 2 | 88.9 | 0 |
| dolphin-2.9.1-yi-1.5-34b | 73.0 | 2 | 2 | 73.0 | 0 |
| Granite Code 8B Instruct | 40.7 | 2 | 2 | 40.9 | +0.2 |
| Granite3.2 8B Instruct | 74.1 | 2 | 2 | 74.1 | 0 |
| Granite4 1B H | 66.8 | 2 | 2 | 67.0 | +0.2 |
| Hermes3 3B Llama3.2 | 48.5 | 2 | 1 | 48.5 | 0 |
| Lfm2 24B | 87.3 | 2 | 2 | 87.3 | 0 |
| Llama-2-7b-chat-hf | 43.8 | 2 | 2 | 43.8 | 0 |
| Mistral-7B-Instruct-v0.2 | 59.2 | 2 | 2 | 59.4 | +0.2 |
| Phi-4-mini-instruct | 81.6 | 2 | 2 | 81.6 | 0 |
| Qwen2.5-3B-Instruct | 76.8 | 2 | 2 | 77.0 | +0.2 |
| Qwen2.5-Coder-32B-Instruct | 90.5 | 2 | 2 | 90.5 | 0 |
| YI 9B Chat V1.5 | 70.5 | 2 | 2 | 70.5 | 0 |
| Codegemma 7B Instruct V1.1 | 59.5 | 1 | 1 | 59.5 | 0 |
| Codellama 34B Instruct | 61.2 | 1 | 1 | 61.2 | 0 |
| Codestral 22B V0.1 | 74.8 | 1 | 1 | 74.8 | 0 |
| Cogito 3B V1 Preview Llama | 70.4 | 1 | 1 | 70.4 | 0 |
| Command R 35B 08 2024 | 75.2 | 1 | 1 | 75.2 | 0 |
| Command R7B (12-2024) | 74.8 | 1 | 1 | 74.8 | 0 |
| Deepseek Coder 33B Instruct | 33.4 | 1 | 1 | 33.4 | 0 |
| DeepSeek-Coder-V2-Lite-Instruct | 79.0 | 1 | 1 | 79.0 | 0 |
| Exaone3.5 2.4B Instruct | 76.3 | 1 | 1 | 76.3 | 0 |
| Exaone3.5 32B Instruct | 85.0 | 1 | 1 | 85.0 | 0 |
| Falcon3 10B Instruct | 81.3 | 1 | 1 | 81.3 | 0 |
| Falcon3 7B Instruct | 76.2 | 1 | 1 | 76.2 | 0 |
| Glm4 9B Chat | 73.3 | 1 | 1 | 73.3 | 0 |
| gpt-oss-120b | 98.6 | 1 | 1 | 98.6 | 0 |
| Granite3.3 8B | 79.4 | 1 | 1 | 79.4 | 0 |
| Granite4 Micro H | 76.6 | 1 | 1 | 76.6 | 0 |
| Granite4 Tiny H | 79.1 | 1 | 1 | 79.1 | 0 |
| Hermes-3-Llama-3.1-8B | 67.6 | 1 | 1 | 68.0 | +0.4 |
| Laguna XS 2.1 | 91.2 | 1 | 1 | 91.2 | 0 |
| Llama-3.3-70B-Instruct | 94.3 | 1 | 1 | 94.3 | 0 |
| Marco O1 7B | 70.5 | 1 | 1 | 70.5 | 0 |
| Meta-Llama-3-8B-Instruct | 74.7 | 1 | 1 | 74.7 | 0 |
| Olmo2 7B 1124 Instruct | 65.4 | 1 | 1 | 65.4 | 0 |
| phi-4 | 85.9 | 1 | 1 | 86.3 | +0.4 |
| Qwen 14B Chat V1.5 | 54.8 | 1 | 1 | 54.8 | 0 |
| Qwen2.5-Coder-14B-Instruct | 87.7 | 1 | 1 | 87.7 | 0 |
| Qwen2.5-Coder-3B-Instruct | 72.5 | 1 | 1 | 72.5 | 0 |
| Qwen2.5-Coder-7B-Instruct | 84.5 | 1 | 1 | 84.5 | 0 |
| Qwen2.5-VL-7B-Instruct | 82.0 | 1 | 1 | 82.3 | +0.3 |
| YI 34B Chat V1.5 | 75.2 | 1 | 1 | 75.2 | 0 |
By category
| Category | Answers | Cut off | Share | Counts toward Overall |
|---|---|---|---|---|
| Reasoning | 7,980 | 540 | 6.8% | yes |
| Math | 5,700 | 354 | 6.2% | yes |
| Coding | 6,840 | 541 | 7.9% | yes |
| Instruction following | 6,840 | 175 | 2.6% | yes |
| Knowledge & honesty | 6,840 | 73 | 1.1% | yes |
| Long context | 4,104 | 231 | 5.6% | yes |
| Multilingual | 4,560 | 255 | 5.6% | yes |
| Tool use | 4,560 | 54 | 1.2% | yes |
| Safety | 5,472 | 735 | 13.4% | yes |
| Pushback resistance | 2,736 | 43 | 1.6% | no, shown separately |
| Famous gotchas | 3,420 | 162 | 4.7% | no, shown separately |
| Writing | 1,824 | 105 | 5.8% | no, shown separately |
| Self-description | 684 | 5 | 0.7% | no, shown separately |
Withheld
These models are left out of every figure on this page. Why each was withheld is on the leaderboard.