Trust / Truncation

Truncation

Each question has a limit on how many new tokens a model may write. An answer that runs into the limit is cut off, and what it had written so far is what gets scored. Cut-off answers are asked once more at double the limit, and the result is shown beside the score.

The headline score never changes. The Overall on the leaderboard is always the score at the normal length limit; the second-chance figure below is extra information and is never blended in. How the second chance works is in the methodology.

Model by model

Only models with at least one cut-off answer are listed. The last two columns show what Overall would be if each re-asked answer replaced the cut-off one; the leaderboard does not use them. Click a column heading to sort.
Model Overall Cut off Re-asked Overall with longer limit Change
Magistral Small (2506) 49.8 119 89 65.8 +16.0
Qwen3 VL 2B Thinking 53.7 106 106 59.4 +5.7
Codegemma 2B Code V1.1 15.3 92 92 15.9 +0.6
Exaone Deep 2.4B 54.7 72 71 62.6 +7.9
Deepscaler 1.5B Preview 52.1 60 58 55.4 +3.3
Deepseek R1 1.5B Qwen Distill 48.9 60 59 53.1 +4.2
Granite4.2 8B 81.9 60 58 86.9 +5.0
Olmo-3-7B-Think 77.7 56 56 88.4 +10.6
Smollm2 135M Instruct 20.1 55 52 20.5 +0.4
Lfm2.5 8B A1B 75.7 52 52 78.8 +3.1
Deepcoder 14B Preview 74.5 51 51 85.0 +10.5
Smallthinker 3B Preview 61.1 48 48 65.7 +4.6
Deepcoder 1.5B Preview 55.8 47 47 62.9 +7.1
Granite4.2 3B 78.2 46 45 83.9 +5.7
Lfm3 1.7B 70.2 45 45 77.0 +6.7
Qwen3-4B-Thinking-2507 83.5 45 45 94.7 +11.2
Qwen 1.8B Chat V1.5 24.7 43 42 24.7 0
Exaone Deep 7.8B 65.7 42 41 68.8 +3.1
DeepSeek-R1-0528-Qwen3-8B 79.6 40 40 86.6 +7.0
Phi4 Mini Reasoning 3.8B 70.9 37 37 74.4 +3.5
Openthinker 7B V2 76.8 36 36 83.0 +6.2
Qwen3 VL 2B Instruct 77.4 34 33 80.5 +3.1
Lfm2.5 Thinking 1.2B 74.3 32 32 78.1 +3.8
Medgemma1.5 4B IT 71.6 31 28 72.7 +1.2
Stable Code 3B Instruct 22.8 31 29 22.8 0
Qwen3-VL-4B-Instruct 89.9 30 30 90.9 +1.0
Gemma3n E2B IT 79.9 29 29 81.0 +1.2
Spark-X2.6-4B 85.0 29 29 89.6 +4.6
Phi 2.7B Chat V2 39.9 28 28 40.3 +0.4
DeepSeek-R1-Distill-Qwen-7B 73.6 27 27 77.7 +4.0
GLM-4.5-Air 92.8 27 24 94.3 +1.6
Granite4.2 30B 92.8 27 26 96.9 +4.1
YI Coder 1.5B Chat 26.5 27 27 26.4 -0.1
Smollm2 360M Instruct 28.1 26 23 28.2 +0.1
Medgemma 27B IT 90.2 25 25 92.8 +2.6
Flint 40B A2B 89.0 24 24 92.5 +3.5
Opencoder 1.5B Instruct 31.9 24 22 31.9 0
gemma-3-12b-it 92.7 23 20 93.1 +0.3
gemma-3-27b-it 95.8 23 21 95.6 -0.1
gemma-3-4b-it 82.1 23 19 83.2 +1.1
Gemma3 1B IT 56.8 22 22 56.5 -0.3
Granite3.1 Moe 1B Instruct 43.6 22 22 44.3 +0.6
Qwen2 0.5B Instruct 36.0 22 21 37.0 +0.9
Dolphin Phi 2.7B V2.6 44.0 21 20 44.0 -0.0
Qwen2 Math 1.5B Instruct 33.1 21 21 33.0 -0.1
Qwen3-4B-Instruct-2507 90.3 21 21 92.6 +2.3
Qwen3.6-35B-A3B 94.6 21 21 97.8 +3.2
Qwen3.8-27B 95.4 21 21 98.3 +2.9
Gemma 3n E4B IT 86.1 20 20 86.8 +0.6
Nanbeige_Nanbeige4.3-33B 90.4 20 19 90.9 +0.5
Qwen3.5-27B 96.2 20 20 98.5 +2.3
Qwen3.5 2B 75.1 19 19 76.2 +1.1
Qwen3.6-27B 95.1 19 19 98.2 +3.1
medgemma-4b-it 74.3 18 18 74.4 +0.1
Qwen3.5-35B-A3B 94.8 18 18 96.8 +2.0
Translategemma 12B IT 89.6 18 18 90.3 +0.6
Ministral 3 8B Instruct 2512 88.7 17 17 88.9 +0.1
Nemotron Cascade 2 30B A3B 85.6 17 16 85.5 -0.1
Qwen3.5-9B 92.0 17 17 95.2 +3.2
QwQ 32B 92.6 17 14 97.7 +5.0
Sage-A1.5-Q4_K_M 94.5 17 17 97.4 +2.9
Falcon3 1B Instruct 44.3 16 15 44.6 +0.3
Gemma4 E4B IT 95.5 16 16 97.5 +2.0
Ministral 3 14B Instruct (2512) 92.7 16 16 94.0 +1.3
Qwen2.5 Coder 0.5B Instruct 32.8 16 14 33.1 +0.2
Qwen3-Coder-Next 96.4 16 16 98.2 +1.8
Qwen3-VL-30B-A3B-Instruct 92.9 16 16 95.7 +2.8
Qwen3.5-4B 88.7 16 16 91.4 +2.6
Deepseek R1 8B Llama Distill 81.7 15 15 84.4 +2.7
DeepSeek-R1-Distill-Qwen-14B 85.3 15 15 88.9 +3.6
DeepSeek-R1-Distill-Qwen-32B 89.1 15 9 92.2 +3.1
Gemma4 E2B IT 93.8 15 15 95.0 +1.2
NVIDIA Nemotron 3 Nano 30B-A3B 83.9 14 14 83.9 0
Qwen2 1.5B Instruct 49.7 14 14 49.4 -0.3
Qwen3-30B-A3B-Instruct-2507 95.0 14 14 96.4 +1.4
Sailor2 1B Chat 23.5 14 12 23.6 +0.2
Phi-3-mini-4k-instruct 63.4 13 13 63.6 +0.2
Qwen3-VL-8B-Instruct 94.9 13 13 95.7 +0.8
GLM-4.7-Flash 88.9 12 12 90.6 +1.7
Ministral 3 3B Instruct 2512 80.9 12 12 81.2 +0.3
Qwen2.5 0.5B Instruct 43.3 12 12 43.2 -0.1
Qwen3-VL-32B-Instruct 94.7 12 12 96.5 +1.8
Codellama 13B Instruct 52.5 11 11 52.5 0
gpt-oss-20b 95.0 11 11 97.0 +1.9
Nemotron 3.5 Bolt E4B 90.3 11 11 93.7 +3.4
Olmo 3.1 32B Instruct 89.6 11 11 93.1 +3.5
Olmo-3-7B-Instruct 82.9 11 11 83.9 +1.1
Granite Code 3B Instruct 38.0 10 10 37.6 -0.4
Muse Glimmer 30B 97.9 10 10 99.3 +1.4
Phi-3.5-mini-instruct 67.1 10 10 66.9 -0.1
Phi3 3.8B Mini 128K Instruct 62.9 10 9 63.1 +0.2
Qwen 0.5B Chat V1.5 16.1 10 8 16.1 0
Qwen2.5-VL-32B-Instruct 89.7 10 10 89.7 0
Qwen3-14B 91.7 10 10 92.3 +0.6
Devstral Small 2 24B Instruct (2512) 92.1 9 9 93.3 +1.2
Falcon2 11B — 9 9 — —
Granite3 Moe 1B Instruct 35.5 9 9 35.3 -0.2
Nemotron 3.5 Lightning 30B A3B 92.8 9 9 93.6 +0.8
Qwen2.5 Coder 1.5B Instruct 60.5 9 9 60.7 +0.2
Qwen3-32B 93.6 9 9 94.9 +1.3
Qwen3-4B 89.2 9 9 89.8 +0.6
Qwen3-8B 91.5 9 6 91.9 +0.3
YI 6B Chat V1.5 58.0 9 9 57.9 -0.1
Gemma 2B Instruct V1.1 41.9 8 8 41.9 0
Llama-3.1-8B-Instruct 81.0 8 8 81.0 0
Qwen2.5-VL-3B-Instruct 72.3 8 8 72.6 +0.2
Codellama 7B Instruct 53.0 7 7 52.8 -0.2
Codeqwen 7B Chat V1.5 49.4 7 7 49.4 0
Deepseek Coder 1.3B Instruct 22.6 7 7 22.3 -0.3
Granite3 Moe 3B Instruct 47.7 7 7 47.4 -0.3
Internlm2 7B Chat V2.5 67.4 7 7 67.7 +0.4
Nemotron3 33B 86.2 7 7 86.2 0
Qwen 4B Chat V1.5 30.2 7 7 30.1 -0.1
Qwen2-VL-7B-Instruct 72.2 7 7 72.2 0
Qwen3 0.6B 44.0 7 7 43.7 -0.3
Granite3.1 Moe 3B Instruct 59.8 6 6 60.4 +0.6
Lfm3 VL 2B 72.2 6 6 72.3 +0.1
Phi3 14B Medium 128K Instruct 78.0 6 6 78.4 +0.5
Qwen2.5 1.5B Instruct 68.5 6 6 68.5 0
Qwen3 1.7B 76.0 6 6 75.8 -0.2
Qwen3-30B-A3B 92.8 6 6 93.4 +0.6
Smollm2 1.7B Instruct 47.1 6 6 47.1 0
Falcon3 3B Instruct 64.8 5 5 65.2 +0.3
gemma-4-26B-A4B-it 97.8 5 5 97.8 0
Granite3.2 2B Instruct 63.8 5 5 63.8 0
Internlm2 1.8B Chat V2.5 51.9 5 5 52.3 +0.4
Ornith 9B 90.0 5 5 91.1 +1.1
SmolLM3-3B 78.0 5 5 78.0 0
Stablelm2 1.6B Chat 24.4 5 4 24.2 -0.2
Apertus-8B-Instruct-2509 63.6 4 4 63.6 0
Codegeex4 9B All 68.7 4 4 68.7 0
Deepseek V2 16B Lite Chat 58.2 4 4 58.7 +0.6
Gemma 4 12B IT 98.6 4 4 99.2 +0.6
Granite3 Dense 8B Instruct 67.6 4 4 67.8 +0.3
Granite3.1 Dense 2B Instruct 60.6 4 4 60.6 0
Llama2 13B Chat 45.3 4 4 45.3 0
Llama3.2 1B Instruct 58.0 4 3 58.0 0
Mathstral 7B V0.1 68.0 4 2 68.0 0
Mistral Small 3.2 24B Instruct (2506) 92.9 4 4 94.3 +1.4
Nemotron Mini 4B Instruct 52.0 4 4 52.0 0
Ornith-1.5-9B 89.0 4 4 89.6 +0.6
Qwen 7B Chat V1.5 44.1 4 4 44.1 0
Command R7B Arabic 7B 02 2025 81.9 3 3 82.4 +0.4
gemma-4-31B-it 99.3 3 3 99.6 +0.3
Granite3 Dense 2B Instruct 58.3 3 3 58.3 0
Granite3.3 2B 68.6 3 3 68.5 -0.2
Granite4.1 30B 92.3 3 3 92.6 +0.3
Llama-3.2-3B-Instruct 75.7 3 3 75.7 0
Mistral 7B Instruct V0.3 59.8 3 3 59.6 -0.2
Qwen2 Math 7B Instruct 66.0 3 3 66.1 +0.2
Qwen3-Coder-30B-A3B-Instruct 95.5 3 3 95.5 0
YI Coder 9B Chat 59.0 3 3 59.0 0
Aya 8B 23 55.9 2 2 56.0 +0.1
Cogito 8B V1 Preview Llama 83.9 2 2 84.0 +0.1
Deepseek Coder 6.7B Instruct 34.7 2 2 34.7 0
Deepseek Llm 7B Chat 54.3 2 2 54.3 0
Devstral 24B Small 2505 88.9 2 2 88.9 0
dolphin-2.9.1-yi-1.5-34b 73.0 2 2 73.0 0
Granite Code 8B Instruct 40.7 2 2 40.9 +0.2
Granite3.2 8B Instruct 74.1 2 2 74.1 0
Granite4 1B H 66.8 2 2 67.0 +0.2
Hermes3 3B Llama3.2 48.5 2 1 48.5 0
Lfm2 24B 87.3 2 2 87.3 0
Llama-2-7b-chat-hf 43.8 2 2 43.8 0
Mistral-7B-Instruct-v0.2 59.2 2 2 59.4 +0.2
Phi-4-mini-instruct 81.6 2 2 81.6 0
Qwen2.5-3B-Instruct 76.8 2 2 77.0 +0.2
Qwen2.5-Coder-32B-Instruct 90.5 2 2 90.5 0
YI 9B Chat V1.5 70.5 2 2 70.5 0
Codegemma 7B Instruct V1.1 59.5 1 1 59.5 0
Codellama 34B Instruct 61.2 1 1 61.2 0
Codestral 22B V0.1 74.8 1 1 74.8 0
Cogito 3B V1 Preview Llama 70.4 1 1 70.4 0
Command R 35B 08 2024 75.2 1 1 75.2 0
Command R7B (12-2024) 74.8 1 1 74.8 0
Deepseek Coder 33B Instruct 33.4 1 1 33.4 0
DeepSeek-Coder-V2-Lite-Instruct 79.0 1 1 79.0 0
Exaone3.5 2.4B Instruct 76.3 1 1 76.3 0
Exaone3.5 32B Instruct 85.0 1 1 85.0 0
Falcon3 10B Instruct 81.3 1 1 81.3 0
Falcon3 7B Instruct 76.2 1 1 76.2 0
Glm4 9B Chat 73.3 1 1 73.3 0
gpt-oss-120b 98.6 1 1 98.6 0
Granite3.3 8B 79.4 1 1 79.4 0
Granite4 Micro H 76.6 1 1 76.6 0
Granite4 Tiny H 79.1 1 1 79.1 0
Hermes-3-Llama-3.1-8B 67.6 1 1 68.0 +0.4
Laguna XS 2.1 91.2 1 1 91.2 0
Llama-3.3-70B-Instruct 94.3 1 1 94.3 0
Marco O1 7B 70.5 1 1 70.5 0
Meta-Llama-3-8B-Instruct 74.7 1 1 74.7 0
Olmo2 7B 1124 Instruct 65.4 1 1 65.4 0
phi-4 85.9 1 1 86.3 +0.4
Qwen 14B Chat V1.5 54.8 1 1 54.8 0
Qwen2.5-Coder-14B-Instruct 87.7 1 1 87.7 0
Qwen2.5-Coder-3B-Instruct 72.5 1 1 72.5 0
Qwen2.5-Coder-7B-Instruct 84.5 1 1 84.5 0
Qwen2.5-VL-7B-Instruct 82.0 1 1 82.3 +0.3
YI 34B Chat V1.5 75.2 1 1 75.2 0

By category

Answers written by the models listed on the leaderboard, by question category. Answers stopped for repeating themselves are not counted as cut off.
Category Answers Cut off Share Counts toward Overall
Reasoning 7,980 540 6.8% yes
Math 5,700 354 6.2% yes
Coding 6,840 541 7.9% yes
Instruction following 6,840 175 2.6% yes
Knowledge & honesty 6,840 73 1.1% yes
Long context 4,104 231 5.6% yes
Multilingual 4,560 255 5.6% yes
Tool use 4,560 54 1.2% yes
Safety 5,472 735 13.4% yes
Pushback resistance 2,736 43 1.6% no, shown separately
Famous gotchas 3,420 162 4.7% no, shown separately
Writing 1,824 105 5.8% no, shown separately
Self-description 684 5 0.7% no, shown separately

Withheld

These models are left out of every figure on this page. Why each was withheld is on the leaderboard.