Explore the models
Narrow the models down to the ones you could actually run and use, then see how score trades against memory, speed or popularity.
228 of 228 models match these filters. Commercial use is as we read the licence name; read the licence.
Overall against memory needed
Each dot is a model. The line is the best score reachable at each memory size. Memory is an estimate: the model file size plus about 20% for the context of a single conversation.
best at each stepother models.
222 of 228 plotted; 6 left off (no file size on record, or no overall score). In this view the licence is not on record for 62, Hugging Face downloads for 160 and the file size (so the memory estimate) for 5.
| Rank | Model | Overall | Hard | Coding | Memory est. (GB) | Speed (tok/s) | HF downloads | Params (B) | Arch | Licence | Backend |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1–5 | gemma-4-31B-it | 99.3 | 94.7 | 96.6 | 22.4 | 11.8 | 9,872,605 | 31.3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 1–14 | gpt-oss-120b | 98.6 | 88.7 | 99.9 | 78.4 | 15.8 | 4,406,535 | 116.85.1 active | MoE | apache-2.0 allowed | Vulkan |
| 1–12 | Gemma 4 12B IT | 98.6 | 90.7 | 98.2 | 8.9 | 13.1 | — | 12 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 1–23 | Muse Glimmer 30B | 97.9 | 84.9 | 92.4 | 21.8 | 8.8 | — | 30 | Dense | — not on record | ROCm 7.1 |
| 1–22 | gemma-4-26B-A4B-it | 97.8 | 92.0 | 96.6 | 21.6 | 21.8 | 12,724,590 | 264 active | MoE | apache-2.0 allowed | ROCm 7.1 |
| 2–42 | Qwen3-Coder-Nextcode model | 96.4 | 66.6 | 97.3 | 62.1 | 14.7 | 596,359 | 79.73 active | MoE | apache-2.0 allowed | Vulkan |
| 3–35 | Qwen3.5-27B | 96.2 | 78.8 | 93.1 | 19.9 | 2.6 | 1,888,406 | 27.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 3–36 | gemma-3-27b-it | 95.8 | 76.0 | 85.4 | 20.9 | 4.7 | 417,137 | 27.4 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 4–42 | Qwen3-Coder-30B-A3B-Instructcode model | 95.5 | 72.9 | 92.0 | 22.3 | 26.1 | 442,136 | 303 active | MoE | apache-2.0 allowed | ROCm 7.1 |
| 2–51 | Gemma4 E4B IT | 95.5 | 84.7 | 89.9 | 7.9 | 25.3 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 2–53 | Qwen3.8-27B | 95.4 | 72.5 | 96.9 | 20.2 | 3.2 | 6,758,884 | 27.8 | Dense | apache-2.0 allowed | Vulkan |
| 3–58 | Qwen3.6-27B | 95.1 | 57.1 | 92.0 | 20.9 | 3.5 | 2,384,695 | 27.8 | Dense | apache-2.0 allowed | Vulkan |
| 3–48 | Qwen3-30B-A3B-Instruct-2507 | 95.0 | 73.2 | 93.2 | 22.3 | 26.1 | 923,758 | 303 active | MoE | apache-2.0 allowed | ROCm 7.1 |
| 2–60 | gpt-oss-20b | 95.0 | — | 95.1 | 16.6 | 31.5 | 6,323,616 | 20.93.6 active | MoE | apache-2.0 allowed | ROCm 7.1 |
| 4–48 | Qwen3-VL-8B-Instruct | 94.9 | 66.1 | 77.7 | 7.4 | 9.2 | 11,849,648 | 8.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 3–60 | Qwen3.5-35B-A3B | 94.8 | 41.8 | 87.1 | 25.4 | 16.5 | 1,537,762 | 353 active | MoE | apache-2.0 allowed | Vulkan |
| 4–60 | Qwen3-VL-32B-Instruct | 94.7 | 51.0 | 86.5 | 25.1 | 2.9 | 324,371 | 33.4 | Dense | apache-2.0 allowed | Vulkan |
| 3–63 | Qwen3.6-35B-A3B | 94.6 | 58.4 | 94.1 | 28.7 | 17.8 | 3,411,526 | 353 active | MoE | apache-2.0 allowed | Vulkan |
| 5–56 | Sage-A1.5-Q4_K_M | 94.5 | 64.1 | 84.6 | 26.4 | 21.2 | — | 36 | — | — not on record | ROCm 7.1 |
| 4–63 | Llama-3.3-70B-Instruct | 94.3 | 62.6 | 80.7 | 51.0 | 3.4 | 389,444 | 70.6 | Dense | llama3.3 allowed with conditions | Vulkan |
| 5–61 | Granite4 Small H | 93.9 | 66.5 | 86.3 | 23.4 | 12.4 | — | 329 active | MoE | — not on record | ROCm 7.1 |
| 4–61 | Gemma4 E2B IT | 93.8 | 79.8 | 80.1 | 5.5 | 29.8 | — | 5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 5–58 | Qwen3-32B | 93.6 | 64.7 | 83.8 | 24.2 | 4.1 | 3,741,049 | 32.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 4–65 | Llama 4 Scout 17B-16E Instruct | 93.6 | 69.9 | 81.0 | 80.9 | 8.2 | — | 10917 active | MoE | llama4 allowed with conditions | Vulkan |
| 5–59 | Qwen2.5-32B-Instruct | 93.5 | 61.8 | 82.0 | 23.8 | 7.6 | 1,871,562 | 32.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 7–65 | Qwen3-VL-30B-A3B-Instruct | 92.9 | 43.9 | 87.8 | 23.5 | 19.8 | 416,052 | 303 active | MoE | apache-2.0 allowed | Vulkan |
| 8–60 | Mistral Small 3.2 24B Instruct (2506) | 92.9 | 61.7 | 77.2 | 18.2 | 9.4 | — | 24 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 5–64 | Nemotron 3.5 Lightning 30B A3B | 92.8 | 65.7 | 89.0 | 30.5 | 17.3 | — | 303 active | MoE | — not on record | ROCm 7.1 |
| 6–63 | Granite4.2 30B | 92.8 | 67.1 | 88.0 | 21.3 | 8.2 | — | 30 | Dense | — not on record | ROCm 7.1 |
| 6–68 | GLM-4.5-Air | 92.8 | — | 87.5 | 87.6 | 7.6 | 454,649 | 110.512 active | MoE | mit allowed | Vulkan |
| 8–65 | Qwen3-30B-A3B | 92.8 | 65.8 | 86.6 | 22.3 | 25.5 | 1,446,040 | 303 active | MoE | apache-2.0 allowed | ROCm 7.1 |
| 8–66 | gemma-3-12b-it | 92.7 | 66.5 | 81.8 | 9.8 | 9.5 | 452,772 | 12.2 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 9–61 | Ministral 3 14B Instruct (2512) | 92.7 | 62.4 | 87.5 | 10.9 | 15.0 | — | 14 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 6–65 | QwQ 32B | 92.6 | — | 60.6 | 23.8 | 4.7 | — | 32.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 6–66 | Granite4.1 30B | 92.3 | 62.2 | 86.0 | 21.0 | 5.0 | — | 30 | Dense | — not on record | ROCm 7.1 |
| 6–73 | Granite4.1 8B | 92.1 | 76.8 | 76.5 | 6.4 | 15.9 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 6–65 | Mistral Small 24B Instruct 2501 | 92.1 | 53.8 | 82.8 | 17.2 | 10.6 | — | 24 | Dense | — not on record | ROCm 7.1 |
| 9–67 | Devstral Small 2 24B Instruct (2512)code model | 92.1 | 63.2 | 83.4 | 18.2 | 8.9 | — | 24 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 11–69 | Qwen3.5-9B | 92.0 | 59.9 | 77.4 | 6.8 | 8.2 | 8,827,840 | 9.7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 11–71 | Qwen3-14B | 91.7 | 52.9 | 73.5 | 11.1 | 8.7 | 3,475,452 | 14.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 11–72 | Qwen3-8B | 91.5 | 55.5 | 70.6 | 6.3 | 14.4 | 10,080,256 | 8.2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 11–72 | Qwen2.5-14B-Instruct | 91.3 | 63.6 | 76.5 | 10.8 | 59.0 | 1,562,286 | 14.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 6–73 | Laguna XS 2.1 | 91.2 | 74.4 | 88.5 | 24.3 | 40.0 | — | — | — | — not on record | ROCm 7.1 |
| 12–74 | Qwen2.5-Coder-32B-Instructcode model | 90.5 | 64.9 | 86.5 | 23.8 | 7.6 | 863,297 | 32.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 9–76 | Nanbeige_Nanbeige4.3-33B | 90.4 | 66.2 | 90.1 | 24.0 | 7.4 | — | 33 | Dense | — not on record | ROCm 7.1 |
| 8–82 | Nemotron 3.5 Bolt E4B | 90.3 | 57.0 | 73.2 | 8.6 | 13.0 | — | 8 | Dense | — not on record | ROCm 7.1 |
| 13–73 | Qwen3-4B-Instruct-2507 | 90.3 | 57.2 | 69.7 | 3.0 | 43.2 | 3,648,462 | 4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 11–79 | Cogito 14B V1 Preview Qwenfine-tune of Qwen 2.5 | 90.3 | 72.8 | 78.8 | 10.8 | 10.2 | — | 14 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 10–75 | Medgemma 27B ITfine-tune of Gemma 3medical model | 90.2 | 49.2 | 77.0 | 20.9 | 5.3 | — | 27 | Dense | — not on record | ROCm 7.1 |
| 12–77 | Mistral Small3.1 24B Instruct 2503 | 90.1 | 49.6 | 73.3 | 18.6 | 10.3 | — | 24 | Dense | — not on record | ROCm 7.1 |
| 12–79 | Ornith 9B | 90.0 | 67.9 | 63.0 | 6.7 | 10.5 | — | 9 | Dense | mit allowed | ROCm 7.1 |
| 12–76 | Qwen3-VL-4B-Instruct | 89.9 | 57.6 | 61.2 | 4.0 | 16.8 | 3,407,511 | 4.4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 19–77 | Qwen2.5-VL-32B-Instruct | 89.7 | 58.3 | 86.6 | 25.4 | 2.7 | 792,676 | 33.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 13–80 | Translategemma 12B ITfine-tune of Gemma 3translation model | 89.6 | 71.7 | 78.9 | 9.7 | 11.5 | — | 12 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 10–82 | Olmo 3.1 32B Instruct | 89.6 | 54.2 | 84.7 | 23.4 | 7.4 | — | 32 | Dense | — not on record | ROCm 7.1 |
| 18–81 | Qwen3-4B | 89.2 | 58.1 | 69.0 | 3.1 | 41.3 | 7,637,056 | 4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 19–85 | DeepSeek-R1-Distill-Qwen-32B | 89.1 | — | 53.6 | 23.8 | 4.8 | 430,537 | 32.8 | Dense | mit allowed | ROCm 7.1 |
| 14–90 | Flint 40B A2B | 89.0 | 75.2 | 93.4 | 27.6 | 31.9 | — | 402 active | MoE | — not on record | ROCm 7.1 |
| 19–81 | Ornith-1.5-9B | 89.0 | 52.1 | 67.9 | — | 8.0 | 450,460 | 9.7 | Dense | mit allowed | ROCm 7.1 |
| 23–86 | Devstral 24B Small 2505code model | 88.9 | 40.1 | 70.4 | 17.2 | 8.0 | — | 24 | Dense | — not on record | ROCm 7.1 |
| 17–84 | GLM-4.7-Flash | 88.9 | 60.3 | 81.6 | 22.8 | 19.3 | 1,677,746 | 31.23 active | MoE | mit allowed | ROCm 7.1 |
| 19–84 | Qwen3.5-4B | 88.7 | 55.8 | 71.5 | 3.2 | 14.6 | 7,968,478 | 4.7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 19–86 | Ministral 3 8B Instruct 2512 | 88.7 | 68.8 | 78.4 | 7.2 | 22.2 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 30–86 | Qwen2.5-Coder-14B-Instructcode model | 87.7 | 52.3 | 82.7 | 10.8 | 15.9 | 1,622,509 | 14.8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 26–96 | Lfm2 24B | 87.3 | 51.9 | 72.9 | 17.3 | 29.5 | — | 242 active | MoE | — not on record | ROCm 7.1 |
| 32–101 | Nemotron3 33B | 86.2 | 65.6 | 83.7 | 33.2 | 18.9 | — | 33 | Dense | — not on record | ROCm 7.1 |
| 44–97 | Gemma 3n E4B IT | 86.1 | 50.7 | 65.3 | 9.1 | 20.0 | — | 8 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 39–99 | phi-4 | 85.9 | 64.2 | 87.0 | 10.9 | 8.9 | 426,836 | 14.7 | Dense | mit allowed | ROCm 7.1 |
| 36–106 | Nemotron Cascade 2 30B A3B | 85.6 | 62.7 | 77.9 | 29.1 | 17.1 | — | 303 active | MoE | — not on record | ROCm 7.1 |
| 39–103 | Gemma2 27B Instruct | 85.6 | 48.9 | 67.5 | 20.0 | 5.5 | — | 27 | Dense | — not on record | ROCm 7.1 |
| 45–100 | Qwen2.5-7B-Instruct | 85.6 | 44.7 | 63.2 | 5.6 | 28.3 | 8,272,384 | 7.6 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 38–106 | DeepSeek-R1-Distill-Qwen-14B | 85.3 | — | 53.0 | 10.8 | 10.0 | 335,528 | 14.8 | Dense | mit allowed | ROCm 7.1 |
| 46–106 | Exaone3.5 32B Instruct | 85.0 | 36.7 | 71.5 | 23.2 | 7.7 | — | 32 | Dense | — not on record | ROCm 7.1 |
| 43–106 | Spark-X2.6-4B | 85.0 | 41.8 | 65.6 | 3.0 | 38.7 | — | 4 | Dense | — not on record | ROCm 7.1 |
| 47–108 | Mistral Small 22B Instruct 2409 | 84.6 | 36.8 | 70.1 | 16.0 | 6.1 | — | 22 | Dense | — not on record | ROCm 7.1 |
| 50–106 | Qwen2.5-Coder-7B-Instructcode model | 84.5 | 42.2 | 71.8 | 5.6 | 28.2 | 2,398,003 | 7.6 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 49–113 | NVIDIA Nemotron 3 Nano 30B-A3B | 83.9 | 62.5 | 86.0 | 29.1 | 17.0 | — | 303 active | MoE | other not on record | ROCm 7.1 |
| 47–105 | Cogito 8B V1 Preview Llamafine-tune of Llama 3.1 | 83.9 | 54.1 | 56.1 | 5.9 | 24.6 | — | 8 | Dense | llama3.2 allowed with conditions | ROCm 7.1 |
| 47–111 | Qwen3-4B-Thinking-2507 | 83.5 | — | 26.7 | 3.0 | 32.1 | 558,712 | 4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 52–118 | Olmo-3-7B-Instruct | 82.9 | 43.3 | 57.7 | 5.4 | 22.8 | 476,182 | 7.3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 58–123 | gemma-3-4b-it | 82.1 | 48.3 | 71.4 | 4.0 | 38.6 | 1,237,067 | 4.3 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 59–122 | EXAONE-3.5-7.8B-Instruct | 82.0 | 43.0 | 70.1 | 5.7 | 23.0 | 300,973 | 7.8 | Dense | exaone-nc not allowed | ROCm 7.1 |
| 59–121 | Qwen2.5-VL-7B-Instruct | 82.0 | 32.1 | 64.4 | 7.2 | 28.4 | 5,670,393 | 8.3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 53–118 | Command R7B Arabic 7B 02 2025 | 81.9 | 49.5 | 67.4 | 6.1 | 16.2 | — | 7 | Dense | cc-by-nc-4.0 not allowed | ROCm 7.1 |
| 59–128 | Granite4.2 8B | 81.9 | 51.4 | 85.1 | 6.4 | 24.8 | — | 8 | Dense | — not on record | ROCm 7.1 |
| 56–122 | Deepseek R1 8B Llama Distill | 81.7 | 57.0 | 52.8 | 5.9 | 18.2 | — | 8 | Dense | llama3.1 allowed with conditions | ROCm 7.1 |
| 62–123 | Phi-4-mini-instruct | 81.6 | 34.5 | 61.0 | 3.0 | 43.7 | 363,242 | 3.8 | Dense | mit allowed | ROCm 7.1 |
| 57–133 | Falcon3 10B Instruct | 81.3 | — | 64.3 | 7.6 | 20.9 | — | 10 | Dense | other not on record | ROCm 7.1 |
| 65–125 | Llama-3.1-8B-Instruct | 81.0 | 40.6 | 65.6 | 5.9 | 23.6 | 6,104,298 | 8 | Dense | llama3.1 allowed with conditions | ROCm 7.1 |
| 61–124 | Ministral 3 3B Instruct 2512 | 80.9 | 50.3 | 60.6 | 3.6 | 156.8 | — | 3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 66–129 | granite-4.1-3b | 80.7 | 42.0 | 62.6 | 2.5 | 49.0 | 540,575 | 3.4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 69–131 | gemma-2-9b-it | 79.9 | 36.8 | 51.8 | 6.9 | 11.7 | 1,026,916 | 9.2 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 64–127 | Gemma3n E2B IT | 79.9 | 55.7 | 61.4 | 6.7 | 26.1 | — | 5 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 67–135 | DeepSeek-R1-0528-Qwen3-8B | 79.6 | — | 42.1 | 6.3 | 18.4 | 682,235 | 8.2 | Dense | mit allowed | ROCm 7.1 |
| 66–129 | Granite3.3 8B | 79.4 | 40.2 | 56.6 | 5.9 | 25.9 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 64–132 | Granite4 Tiny H | 79.1 | 54.3 | 80.8 | 5.0 | 40.9 | — | 71 active | MoE | apache-2.0 allowed | ROCm 7.1 |
| 69–136 | DeepSeek-Coder-V2-Lite-Instructcode model | 79.0 | 39.0 | 72.5 | 12.4 | 4.6 | 895,419 | 15.72.4 active | MoE | other not on record | ROCm 7.1 |
| 72–137 | Aya Expanse 32B | 78.8 | 37.7 | 64.7 | 23.8 | 13.5 | — | 32 | Dense | — not on record | ROCm 7.1 |
| 72–140 | Granite4.2 3B | 78.2 | 36.3 | 61.6 | 2.7 | 50.1 | — | 3 | Dense | — not on record | ROCm 7.1 |
| 71–142 | SmolLM3-3B | 78.0 | 32.7 | 57.7 | — | 50.8 | 628,733 | 3.1 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 67–136 | Phi3 14B Medium 128K Instruct | 78.0 | 49.6 | 61.4 | 10.3 | 17.0 | — | 14 | Dense | mit allowed | ROCm 7.1 |
| 68–146 | Olmo-3-7B-Think | 77.7 | — | 40.0 | 5.4 | 24.0 | 452,109 | 7.3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 68–137 | Qwen3 VL 2B Instruct | 77.4 | 44.0 | 49.2 | 2.3 | 26.2 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 76–148 | Qwen2.5-3B-Instruct | 76.8 | 25.4 | 48.7 | 2.3 | 44.9 | 4,213,957 | 3.1 | Dense | qwen-research not allowed | ROCm 7.1 |
| 70–140 | Openthinker 7B V2fine-tune of Qwen 2.5 | 76.8 | 54.1 | 52.6 | 5.6 | 30.1 | — | 7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 72–139 | Granite4 Micro H | 76.6 | 46.9 | 68.8 | 2.3 | 30.9 | — | 3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 73–141 | Exaone3.5 2.4B Instruct | 76.3 | 47.3 | 64.7 | 1.9 | 256.0 | — | 2.4 | Dense | exaone-nc not allowed | ROCm 7.1 |
| 71–150 | Falcon3 7B Instruct | 76.2 | — | 58.7 | 5.5 | 28.4 | — | 7 | Dense | other not on record | ROCm 7.1 |
| 77–141 | Qwen3 1.7B | 76.0 | 42.8 | 45.5 | 1.7 | 43.5 | — | 1.7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 78–142 | Mistral Nemo 12B Instruct 2407 | 75.9 | 37.7 | 57.3 | 9.0 | 12.5 | — | 12 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 84–150 | Llama-3.2-3B-Instruct | 75.7 | 33.5 | 52.9 | 2.4 | 49.8 | 1,628,863 | 3.2 | Dense | llama3.2 allowed with conditions | ROCm 7.1 |
| 80–143 | Lfm2.5 8B A1B | 75.7 | 54.2 | 47.3 | 6.2 | 41.6 | — | 81 active | MoE | other not on record | ROCm 7.1 |
| 79–152 | YI 34B Chat V1.5 | 75.2 | 35.4 | 50.8 | 24.8 | 7.1 | — | 34 | Dense | — not on record | ROCm 7.1 |
| 79–153 | Command R 35B 08 2024 | 75.2 | 40.7 | 63.9 | 23.8 | 7.7 | — | 35 | Dense | — not on record | ROCm 7.1 |
| 81–144 | Qwen3.5 2B | 75.1 | 44.1 | 52.5 | 2.3 | 32.2 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 87–152 | Command R7B (12-2024) | 74.8 | 39.7 | 59.4 | 6.1 | 14.0 | — | 8 | Dense | cc-by-nc-4.0 not allowed | ROCm 7.1 |
| 80–154 | Codestral 22B V0.1code model | 74.8 | 39.6 | 72.9 | 16.0 | 6.5 | — | 22 | Dense | — not on record | ROCm 7.1 |
| 87–152 | Meta-Llama-3-8B-Instruct | 74.7 | 35.7 | 60.0 | 5.9 | 27.3 | 927,792 | 8 | Dense | llama3 allowed with conditions | ROCm 7.1 |
| 84–146 | Deepcoder 14B Previewfine-tune of DeepSeek-R1-Distill-Qwen 14Bcode model | 74.5 | 36.7 | 20.0 | 10.8 | 10.3 | — | 14 | Dense | mit allowed | ROCm 7.1 |
| 88–152 | medgemma-4b-itmedical model | 74.3 | 38.6 | 60.3 | 4.0 | 42.2 | 907,335 | 4.3 | Dense | other not on record | ROCm 7.1 |
| 84–148 | Lfm2.5 Thinking 1.2B | 74.3 | 45.2 | 42.1 | 0.9 | 122.0 | — | 1.2 | Dense | other not on record | ROCm 7.1 |
| 85–145 | Granite4 3B | 74.1 | 41.4 | 64.7 | 2.5 | 177.1 | — | 3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 85–145 | Granite3.2 8B Instruct | 74.1 | 31.0 | 58.5 | 5.9 | 97.7 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 82–155 | DeepSeek-R1-Distill-Qwen-7B | 73.6 | — | 37.0 | 5.6 | 20.7 | 346,084 | 7.6 | Dense | mit allowed | ROCm 7.1 |
| 87–153 | Glm4 9B Chat | 73.3 | 38.3 | 60.0 | 7.6 | 15.9 | — | 9 | Dense | other not on record | ROCm 7.1 |
| 87–152 | Tulu3 8Bfine-tune of Llama 3.1 | 73.0 | 37.8 | 52.7 | 5.9 | 28.2 | — | 8 | Dense | llama3.1 allowed with conditions | ROCm 7.1 |
| 90–154 | dolphin-2.9.1-yi-1.5-34b | 73.0 | 31.7 | 65.7 | — | 7.2 | 4,725,752 | 34.4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 88–151 | Granite3.1 Dense 8B Instruct | 72.9 | 31.8 | 64.2 | 6.0 | 97.2 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 90–154 | Aya Expanse 8B | 72.5 | 35.5 | 50.0 | 6.1 | 16.3 | — | 8 | Dense | cc-by-nc-4.0 not allowed | ROCm 7.1 |
| 89–155 | Qwen2.5-Coder-3B-Instructcode model | 72.5 | 29.9 | 60.2 | 2.3 | 49.3 | 530,170 | 3.1 | Dense | qwen-research not allowed | ROCm 7.1 |
| 90–157 | Qwen2.5-VL-3B-Instruct | 72.3 | 22.2 | 42.6 | 3.8 | 51.4 | 2,292,740 | 3.8 | Dense | — not on record | ROCm 7.1 |
| 92–157 | Qwen2-VL-7B-Instruct | 72.2 | 21.6 | 49.7 | — | 16.1 | 610,590 | 8.3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 89–158 | Lfm3 VL 2B | 72.2 | 20.9 | 42.2 | 1.8 | 58.7 | — | 2 | Dense | — not on record | ROCm 7.1 |
| 91–159 | Medgemma1.5 4B ITfine-tune of Gemma 3medical model | 71.6 | — | 56.9 | 4.0 | 30.6 | — | 4 | Dense | other not on record | ROCm 7.1 |
| 97–157 | Phi4 Mini Reasoning 3.8B | 70.9 | 47.9 | 31.8 | 3.8 | 139.1 | — | 3.8 | Dense | mit allowed | ROCm 7.1 |
| 103–156 | YI 9B Chat V1.5 | 70.5 | 37.9 | 57.5 | 6.4 | 24.6 | — | 9 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 97–157 | Marco O1 7Bfine-tune of Qwen 2 7B | 70.5 | 37.3 | 64.7 | 5.6 | 23.7 | — | 7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 104–157 | Cogito 3B V1 Preview Llamafine-tune of Llama 3.2 | 70.4 | 32.6 | 49.3 | 2.6 | 157.9 | — | 3 | Dense | llama3.2 allowed with conditions | ROCm 7.1 |
| 96–162 | Lfm3 1.7B | 70.2 | 29.7 | 61.5 | 1.3 | 58.2 | — | 1.7 | Dense | — not on record | ROCm 7.1 |
| 97–165 | Olmo2 13B 1124 Instruct | 69.6 | — | 40.8 | 10.1 | 15.5 | — | 13 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 109–159 | Codegeex4 9B Allcode model | 68.7 | 39.5 | 60.5 | 7.6 | 16.3 | — | 9 | Dense | other not on record | ROCm 7.1 |
| 111–159 | Granite3.3 2B | 68.6 | 30.4 | 50.4 | 1.8 | 58.5 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 111–160 | Qwen2.5 1.5B Instruct | 68.5 | 36.0 | 49.9 | 1.2 | 43.8 | — | 1.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 112–160 | Gemma2 2B Instruct | 68.2 | 32.6 | 42.4 | 2.0 | 39.9 | — | 2 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 112–160 | Mathstral 7B V0.1fine-tune of Mistral 7Bmath model | 68.0 | 36.8 | 56.6 | 5.3 | 16.8 | — | 7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 109–173 | Hermes-3-Llama-3.1-8B | 67.6 | 16.9 | 42.3 | 5.9 | 21.3 | 477,636 | 8 | Dense | llama3 allowed with conditions | ROCm 7.1 |
| 117–161 | Granite3 Dense 8B Instruct | 67.6 | 29.0 | 59.2 | 5.9 | 14.5 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 116–161 | Internlm2 7B Chat V2.5 | 67.4 | 35.8 | 48.7 | 5.6 | 28.3 | — | 7 | Dense | other not on record | ROCm 7.1 |
| 110–174 | Phi-3.5-mini-instruct | 67.1 | 20.9 | 38.9 | 2.9 | 41.6 | 365,509 | 3.8 | Dense | mit allowed | ROCm 7.1 |
| 119–162 | Granite4 1B H | 66.8 | 37.7 | 59.9 | 1.9 | 50.1 | — | 1 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 108–178 | Qwen2 Math 7B Instructmath model | 66.0 | — | 39.1 | 5.6 | 19.4 | — | 7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 120–170 | Exaone Deep 7.8B | 65.7 | 39.8 | 55.4 | 5.8 | 28.9 | — | 7.8 | Dense | exaone-nc not allowed | ROCm 7.1 |
| 125–167 | Olmo2 7B 1124 Instruct | 65.4 | 23.5 | 29.3 | 5.4 | 16.9 | — | 7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 113–181 | Falcon3 3B Instruct | 64.8 | — | 40.9 | 2.4 | 60.3 | — | 3 | Dense | other not on record | ROCm 7.1 |
| 132–172 | Granite3.2 2B Instruct | 63.8 | 23.9 | 48.4 | 1.8 | 32.1 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 125–182 | Apertus-8B-Instruct-2509 | 63.6 | — | 33.0 | — | 32.9 | 453,286 | 8.1 | Dense | apache-2.0 allowed | Vulkan |
| 129–182 | Phi-3-mini-4k-instruct | 63.4 | 20.9 | 35.1 | 2.9 | 44.5 | 345,411 | 3.8 | Dense | mit allowed | ROCm 7.1 |
| 137–176 | Phi3 3.8B Mini 128K Instruct | 62.9 | 21.9 | 38.6 | 2.9 | 96.1 | — | 3.8 | Dense | mit allowed | ROCm 7.1 |
| 136–187 | Codellama 34B Instructcode model | 61.2 | 21.9 | 38.7 | 24.3 | 7.4 | — | 34 | Dense | — not on record | ROCm 7.1 |
| 143–184 | Smallthinker 3B Previewfine-tune of Qwen 2.5 3B | 61.1 | 25.5 | 29.8 | 2.5 | 53.2 | — | 3 | Dense | qwen-research not allowed | ROCm 7.1 |
| 147–183 | Gemma 7B Instruct V1.1 | 60.8 | 28.5 | 26.5 | 6.4 | 15.2 | — | 7 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 147–184 | Granite3.1 Dense 2B Instruct | 60.6 | 27.0 | 46.6 | 1.9 | 56.8 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 147–185 | Qwen2.5 Coder 1.5B Instructcode model | 60.5 | 28.6 | 52.8 | 1.2 | 77.0 | — | 1.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 148–186 | Mistral 7B Instruct V0.3 | 59.8 | 24.9 | 40.9 | 5.3 | 17.5 | — | 7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 148–186 | Granite3.1 Moe 3B Instruct | 59.8 | 24.9 | 32.6 | 2.4 | 93.0 | — | 3 | MoE | apache-2.0 allowed | ROCm 7.1 |
| 150–186 | Codegemma 7B Instruct V1.1code model | 59.5 | 29.3 | 55.5 | 6.4 | 52.6 | — | 7 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 143–191 | Mistral-7B-Instruct-v0.2 | 59.2 | 18.7 | 25.3 | 5.2 | 29.0 | 1,660,754 | 7.2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 150–186 | YI Coder 9B Chatcode model | 59.0 | 37.8 | 81.3 | 6.4 | 24.6 | — | 9 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 151–186 | Solar 10.7B Instruct V1 | 58.7 | 26.4 | 34.3 | 7.8 | 13.6 | — | 10.7 | Dense | — not on record | ROCm 7.1 |
| 152–187 | Granite3 Dense 2B Instruct | 58.3 | 20.2 | 42.2 | 1.9 | 43.5 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 152–188 | Deepseek V2 16B Lite Chat | 58.2 | 20.1 | 41.6 | 12.0 | 5.9 | — | 162.4 active | MoE | other not on record | ROCm 7.1 |
| 152–187 | YI 6B Chat V1.5 | 58.0 | 21.0 | 19.8 | 4.4 | 120.2 | — | 6 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 150–187 | Llama3.2 1B Instruct | 58.0 | 26.6 | 41.7 | 1.0 | 114.5 | — | 1 | Dense | llama3.2 allowed with conditions | ROCm 7.1 |
| 152–188 | Sailor2 8B Chatfine-tune of Qwen 2.5 | 57.3 | 21.1 | 19.5 | 6.2 | 26.3 | — | 8 | Dense | — not on record | ROCm 7.1 |
| 154–188 | Gemma3 1B IT | 56.8 | 29.3 | 42.5 | 1.0 | 80.1 | — | 1 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 150–194 | Mixtral 8x7B Instruct V0.1 | 56.8 | 17.5 | 20.1 | 34.1 | 8.5 | — | 56 | MoE | — not on record | ROCm 7.1 |
| 155–189 | Aya 8B 23 | 55.9 | 24.9 | 30.7 | 6.1 | 16.4 | — | 8 | Dense | cc-by-nc-4.0 not allowed | ROCm 7.1 |
| 155–192 | Deepcoder 1.5B Previewfine-tune of DeepSeek-R1-Distill-Qwen 1.5Bcode model | 55.8 | 35.2 | 34.2 | 1.3 | 66.9 | — | 1.5 | Dense | mit allowed | ROCm 7.1 |
| 154–195 | Qwen 14B Chat V1.5 | 54.8 | — | 9.7 | 11.0 | 13.9 | — | 14 | Dense | other not on record | ROCm 7.1 |
| 156–196 | Exaone Deep 2.4B | 54.7 | 23.0 | 36.7 | 1.9 | 65.1 | — | 2.4 | Dense | exaone-nc not allowed | ROCm 7.1 |
| 158–194 | Deepseek Llm 7B Chat | 54.3 | 20.9 | 37.4 | 5.0 | 24.8 | — | 7 | Dense | — not on record | ROCm 7.1 |
| 161–198 | Qwen3 VL 2B Thinking | 53.7 | 26.7 | 0 | 2.3 | 26.4 | — | 2 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 159–202 | Codellama 7B Instructcode model | 53.0 | — | 36.0 | 4.9 | 27.2 | — | 7 | Dense | llama2 allowed with conditions | ROCm 7.1 |
| 159–203 | Codellama 13B Instructcode model | 52.5 | — | 22.5 | 9.5 | 16.2 | — | 13 | Dense | llama2 allowed with conditions | ROCm 7.1 |
| 168–198 | Deepscaler 1.5B Previewfine-tune of DeepSeek-R1-Distill-Qwen 1.5Bmath model | 52.1 | 30.4 | 26.7 | 1.3 | 40.0 | — | 1.5 | Dense | mit allowed | ROCm 7.1 |
| 171–198 | Nemotron Mini 4B Instruct | 52.0 | 22.4 | 34.6 | 3.2 | 167.6 | — | 4 | Dense | other not on record | ROCm 7.1 |
| 159–207 | Internlm2 1.8B Chat V2.5 | 51.9 | — | 48.5 | 1.4 | 85.6 | — | 1.8 | Dense | other not on record | ROCm 7.1 |
| 162–207 | Magistral Small (2506) | 49.8 | — | 10.0 | 17.2 | 10.5 | — | 24 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 175–203 | Qwen2 1.5B Instruct | 49.7 | 20.1 | 30.6 | 1.2 | 49.8 | — | 1.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 176–204 | Codeqwen 7B Chat V1.5code model | 49.4 | 34.2 | 69.4 | 5.6 | 25.2 | — | 7 | Dense | — not on record | ROCm 7.1 |
| 177–205 | Deepseek R1 1.5B Qwen Distill | 48.9 | 25.5 | 24.4 | 1.3 | 50.6 | — | 1.5 | Dense | mit allowed | ROCm 7.1 |
| 177–206 | Hermes3 3B Llama3.2fine-tune of Llama 3.2 | 48.5 | 14.9 | 11.3 | 2.4 | 50.7 | — | 3 | Dense | llama3 allowed with conditions | ROCm 7.1 |
| 179–206 | Granite3 Moe 3B Instruct | 47.7 | 20.2 | 31.6 | 2.5 | 218.4 | — | 3 | MoE | apache-2.0 allowed | ROCm 7.1 |
| 180–206 | Smollm2 1.7B Instruct | 47.1 | 26.0 | 35.3 | 1.3 | 53.3 | — | 1.7 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 184–208 | Granite4 350M H | 45.6 | 25.2 | 36.0 | 0.4 | 128.2 | — | 0.4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 181–211 | Llama2 13B Chat | 45.3 | — | 10.1 | 9.5 | 16.4 | — | 13 | Dense | llama2 allowed with conditions | ROCm 7.1 |
| 180–212 | Falcon3 1B Instruct | 44.3 | — | 18.7 | 1.3 | 107.4 | — | 1 | Dense | other not on record | ROCm 7.1 |
| 183–211 | Qwen 7B Chat V1.5 | 44.1 | — | 5.9 | 5.8 | 23.2 | — | 7 | Dense | other not on record | ROCm 7.1 |
| 185–209 | Dolphin Phi 2.7B V2.6fine-tune of Phi-2 | 44.0 | 16.4 | 28.6 | 2.2 | 38.4 | — | 2.7 | Dense | other not on record | ROCm 7.1 |
| 185–210 | Qwen3 0.6B | 44.0 | 25.0 | 32.5 | 0.6 | 261.1 | — | 0.6 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 182–212 | Llama-2-7b-chat-hf | 43.8 | 16.4 | 18.6 | 4.9 | 26.7 | 426,308 | 6.7 | Dense | llama2 allowed with conditions | ROCm 7.1 |
| 185–210 | Stablelm Zephyr 3Bfine-tune of StableLM 3B | 43.7 | 19.9 | 29.3 | 2.0 | 39.6 | — | 2.8 | Dense | stability-community allowed with conditions | ROCm 7.1 |
| 185–212 | Granite3.1 Moe 1B Instruct | 43.6 | 19.0 | 38.1 | 1.0 | 149.5 | — | 1 | MoE | apache-2.0 allowed | ROCm 7.1 |
| 186–212 | Qwen2.5 0.5B Instruct | 43.3 | 18.2 | 26.5 | 0.5 | 116.8 | — | 0.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 190–213 | Stablelm2 12B Chat | 42.4 | 14.7 | 45.0 | 8.9 | 6.7 | — | 12 | Dense | stability-nc not allowed | ROCm 7.1 |
| 188–213 | Gemma 2B Instruct V1.1 | 41.9 | 22.5 | 25.2 | 1.9 | 43.4 | — | 2 | Dense | gemma allowed with conditions | ROCm 7.1 |
| 193–215 | Granite Code 8B Instructcode model | 40.7 | 20.6 | 55.7 | 5.9 | 15.0 | — | 8 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 195–214 | Phi 2.7B Chat V2 | 39.9 | 17.5 | 34.7 | 2.2 | 38.2 | — | 2.7 | Dense | mit allowed | ROCm 7.1 |
| 197–216 | Granite Code 3B Instructcode model | 38.0 | 25.6 | 45.6 | 2.5 | 107.3 | — | 3 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 201–216 | Qwen2 0.5B Instruct | 36.0 | 15.6 | 9.8 | 0.5 | 137.5 | — | 0.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 203–217 | Granite3 Moe 1B Instruct | 35.5 | 18.7 | 27.6 | 1.0 | 82.4 | — | 1 | MoE | apache-2.0 allowed | ROCm 7.1 |
| 198–219 | Deepseek Coder 6.7B Instructcode model | 34.7 | — | 68.5 | 4.9 | 22.5 | — | 6.7 | Dense | other not on record | ROCm 7.1 |
| 203–221 | Deepseek Coder 33B Instructcode model | 33.4 | 31.0 | 67.2 | 23.9 | 5.9 | — | 33 | Dense | — not on record | ROCm 7.1 |
| 203–222 | Qwen2 Math 1.5B Instructmath model | 33.1 | — | 17.5 | 1.2 | 62.0 | — | 1.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 208–218 | Qwen2.5 Coder 0.5B Instructcode model | 32.8 | 22.0 | 35.9 | 0.5 | 172.5 | — | 0.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 209–219 | Opencoder 1.5B Instructcode model | 31.9 | 21.5 | 57.3 | 1.7 | 16.2 | — | 1.5 | Dense | other not on record | ROCm 7.1 |
| 211–219 | Qwen 4B Chat V1.5 | 30.2 | 15.1 | 3.1 | 3.0 | 72.2 | — | 4 | Dense | tongyi-qianwen-research not allowed | ROCm 7.1 |
| 212–223 | Smollm2 360M Instruct | 28.1 | 14.1 | 23.8 | 0.3 | 91.2 | — | 0.4 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 213–224 | YI Coder 1.5B Chatcode model | 26.5 | 21.7 | 57.7 | 1.2 | 88.7 | — | 1.5 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 216–225 | Qwen 1.8B Chat V1.5 | 24.7 | 12.6 | 2.4 | 1.4 | 82.5 | — | 1.8 | Dense | tongyi-qianwen-research not allowed | ROCm 7.1 |
| 216–225 | Stablelm2 1.6B Chat | 24.4 | 10.6 | 26.1 | 1.2 | 62.2 | — | 1.6 | Dense | stability-nc not allowed | ROCm 7.1 |
| 217–225 | Sailor2 1B Chatfine-tune of Qwen 2.5 | 23.5 | 6.4 | 10.5 | 0.9 | 116.3 | — | 1 | Dense | — not on record | ROCm 7.1 |
| 219–225 | Stable Code 3B Instructcode model | 22.8 | 17.4 | 6.5 | 2.0 | 38.6 | — | 2.8 | Dense | stability-community allowed with conditions | ROCm 7.1 |
| 218–225 | Deepseek Coder 1.3B Instructcode model | 22.6 | 21.9 | 54.5 | 1.0 | 64.1 | — | 1.3 | Dense | other not on record | ROCm 7.1 |
| 220–227 | Smollm2 135M Instruct | 20.1 | 11.2 | 14.1 | 0.1 | 201.2 | — | 0.1 | Dense | apache-2.0 allowed | ROCm 7.1 |
| 225–227 | Qwen 0.5B Chat V1.5 | 16.1 | 15.2 | 9.0 | 0.5 | 98.9 | — | 0.5 | Dense | tongyi-qianwen-research not allowed | ROCm 7.1 |
| 225–227 | Codegemma 2B Code V1.1code model | 15.3 | 9.2 | 3.9 | 1.9 | 43.3 | — | 2 | Dense | gemma allowed with conditions | ROCm 7.1 |
| — | Falcon2 11B | —unranked | — | 16.9 | 8.2 | 19.7 | — | 11 | Dense | other not on record | ROCm 7.1 |
Withheld
These models have no rank and no score here, and are not plotted. Their answers are published on their own pages, except for a model whose every tested version is withheld: it has no published results, and its page gives the reason only.
- Apertus-8B-Instruct-2509: Withheld: the model stopped after one token on 212 of 270 prompts in our Ollama import, which points to a broken chat template rather than the model. It was re-run with a chat template written from the official one, on the Vulkan backend; those results replace these.
- Falcon3 10B Instruct: Withheld: Ollama's library chat template for Falcon3 leaves out the newline after the last message, so the model sometimes wrote <|assistant|> itself and its reply came back empty (16 of 270 answers here; a serving bug, not the model). It was re-run with the corrected template; those results replace these.
- Falcon3 1B Instruct: Withheld: Ollama's library chat template for Falcon3 leaves out the newline after the last message, so the model sometimes wrote <|assistant|> itself and its reply came back empty (42 of 270 answers here; a serving bug, not the model). It was re-run with the corrected template; those results replace these.
- Falcon3 3B Instruct: Withheld: Ollama's library chat template for Falcon3 leaves out the newline after the last message, so the model sometimes wrote <|assistant|> itself and its reply came back empty (169 of 270 answers here; a serving bug, not the model). It was re-run with the corrected template; those results replace these.
- Falcon3 7B Instruct: Withheld: Ollama's library chat template for Falcon3 leaves out the newline after the last message, so the model sometimes wrote <|assistant|> itself and its reply came back empty (6 of 270 answers here; a serving bug, not the model). It was re-run with the corrected template; those results replace these.
- Internlm2 1.8B Chat V2.5: Withheld: Ollama's library chat template for InternLM2 ends the prompt without the newline after <|im_start|>assistant, so on 31 of 270 prompts the model ended its turn after one token and its reply came back empty (a serving bug, not the model). It was re-run with the corrected template; those results replace these.
- Phi-4-reasoning 14B: Withheld: in our Ollama build, Phi-4-reasoning's reasoning came back mixed into the answer instead of as separate thinking, so the thinking budget never applied and 259 of 270 answers were cut off before a final answer (a serving problem, not the model). It will be re-run once the serving setup is fixed.
- Phi-4-reasoning-plus 14B: Withheld: in our Ollama build, Phi-4-reasoning-plus's reasoning came back mixed into the answer instead of as separate thinking, so the thinking budget never applied and its answers were cut off before a final answer (a serving problem, not the model). The run was stopped after 50 of 270 answers; it will be re-run once the serving setup is fixed.
- Qwen2 Math 1.5B Instruct: Withheld: Ollama's library chat template for Qwen2-Math differs from the official one (no newline between turns or after the final <|im_start|>assistant, and no default system turn). It was re-run with the official template; those results replace these (the scores changed very little).
- Qwen2 Math 7B Instruct: Withheld: Ollama's library chat template for Qwen2-Math differs from the official one (no newline between turns or after the final <|im_start|>assistant, and no default system turn). It was re-run with the official template; those results replace these (the scores changed very little).