Speed on this laptop
Every model gets a short performance test before its benchmark run. All numbers on this page were measured on one AMD Ryzen AI Max+ 395 laptop through Ollama (GPU label recorded with the runs: AMD Radeon 8060S (Ryzen AI Max+ 395, 128 GB unified, ~78 GB GPU)); backends in use: ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after); ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots); ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots); ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots); Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots); Vulkan (Ollama 0.40.4, flash attention, 16k context; 2 parallel slots); Vulkan (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots). They compare models with each other on this machine, not with other hardware. Speeds are not comparable across backends: the backend each model was measured on is shown in its row.
Speed against quality
- Tiny (under 3B)
- 3–4B
- 4–9B
- 9–16B
- 16–35B
- > 35B
- size unknown
Per-model speed and memory
| Model | Backend | Answers generated | Params (B) | Overall | Decode (tok/s) | Decode x4 (tok/s) | Prefill 4k (tok/s) | Prefill 12k (tok/s) | TTFT (s) | Load (s) | GPU memory on our server (GB) | GPU share |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3 0.6B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 0.6 | 44.0 | 223.0 | 261.1 | 8611 | 5013 | 0.02 | 66.1 | 15.6 | 100% |
| Exaone3.5 2.4B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2.4 | 76.3 | 100.9 | 256.0 | 2890 | 2325 | 0.04 | 74.3 | 11.9 | 100% |
| Granite3 Moe 3B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 47.7 | 147.4 | 218.4 | 3846 | 4340 | 0.04 | 11.4 | 4.3 | 100% |
| Smollm2 135M Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 0.1 | 20.1 | 179.9 | 201.2 | 4628 | 4924 | 0.03 | 16.2 | 1.7 | 100% |
| Granite4 3B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 74.1 | 71.3 | 177.1 | 1586 | 1229 | 0.06 | 48.9 | 13.2 | 100% |
| Qwen2.5 Coder 0.5B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 0.5 | 32.8 | 96.7 | 172.5 | 3052 | 2336 | 0.05 | 13.6 | 2.2 | 100% |
| Nemotron Mini 4B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 4 | 52.0 | 66.8 | 167.6 | 2234 | 2567 | 0.06 | 21.1 | 6.6 | 100% |
| Cogito 3B V1 Preview Llamafine-tune of Llama 3.2 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 70.4 | 73.4 | 157.9 | 2153 | 1601 | 0.05 | 68.8 | 17.4 | 100% |
| Ministral 3 3B Instruct 2512 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 80.9 | 70.9 | 156.8 | 2516 | 1632 | 0.06 | 75.1 | 16.6 | 100% |
| Granite3.1 Moe 1B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | 43.6 | 102.6 | 149.5 | 2095 | 1624 | 0.06 | 44.8 | 7.5 | 100% |
| Phi4 Mini Reasoning 3.8B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3.8 | 70.9 | 48.8 | 139.1 | 1946 | 1411 | 0.07 | 78.0 | 20.7 | 100% |
| Qwen2 0.5B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 0.5 | 36.0 | 93.7 | 137.5 | 3110 | 2336 | 0.05 | 14.1 | 2.2 | 100% |
| Granite4 350M H | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 0.4 | 45.6 | 55.8 | 128.2 | 1739 | 1756 | 0.14 | 8.4 | 1.2 | 100% |
| Lfm2.5 Thinking 1.2B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.2 | 74.3 | 205.0 | 122.0 | 6055 | 6498 | 0.03 | 3.0 | 1.1 | 100% |
| YI 6B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 6 | 58.0 | 43.3 | 120.2 | 1256 | 1353 | 0.11 | 16.9 | 5.9 | 100% |
| Qwen2.5 0.5B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 0.5 | 43.3 | 99.3 | 116.8 | 3069 | 2327 | 0.05 | 13.6 | 2.2 | 100% |
| Sailor2 1B Chatfine-tune of Qwen 2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | 23.5 | 57.1 | 116.3 | 1538 | 1149 | 0.09 | 23.3 | 4.1 | 100% |
| Llama3.2 1B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | 58.0 | 58.5 | 114.5 | 1602 | 1381 | 0.07 | 34.6 | 5.4 | 100% |
| Falcon3 1B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | 44.3 | 51.4 | 107.4 | 1350 | 1381 | 0.08 | 35.7 | 1.1 | 100% |
| Granite Code 3B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 38.0 | 71.8 | 107.3 | 1511 | 1715 | 0.07 | 25.4 | 7.7 | 100% |
| Qwen 0.5B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 0.5 | 16.1 | 99.8 | 98.9 | 1951 | 1104 | 0.05 | 105.1 | 13.4 | 100% |
| Granite3.2 8B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 74.1 | 34.7 | 97.7 | 836 | 714 | 0.13 | 100.4 | 26.9 | 100% |
| Granite3.1 Dense 8B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 72.9 | 34.8 | 97.2 | 818 | 685 | 0.13 | 105.1 | 27.0 | 100% |
| Phi3 3.8B Mini 128K Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3.8 | 62.9 | 63.9 | 96.1 | 1123 | 687 | 0.08 | 243.8 | 54.1 | 100% |
| Granite3.1 Moe 3B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 59.8 | 53.1 | 93.0 | 1079 | 838 | 0.13 | 61.1 | 10.9 | 100% |
| Smollm2 360M Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 0.4 | 28.1 | 275.3 | 91.2 | 9527 | 2732 | 0.02 | 13.6 | 3.1 | 100% |
| YI Coder 1.5B Chatcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.5 | 26.5 | 49.5 | 88.7 | 1051 | 682 | 0.10 | 183.9 | 26.9 | 100% |
| Falcon3 1B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | —unranked | 59.0 | 87.3 | 1301 | 1327 | 0.08 | 36.3 | 6.0 | 100% |
| Internlm2 1.8B Chat V2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.8 | 51.9 | 45.6 | 85.6 | 1086 | 889 | 0.10 | 82.2 | 1.2 | 100% |
| Qwen 1.8B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.8 | 24.7 | 127.4 | 82.5 | 3774 | 2311 | 0.03 | 115.9 | 27.1 | 100% |
| Granite3 Moe 1B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1 | 35.5 | 70.0 | 82.4 | 1882 | 2180 | 0.08 | 16.7 | 2.3 | 90% |
| Gemma3 1B IT | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | 56.8 | 57.0 | 80.1 | 1929 | 1987 | 0.10 | 11.3 | 1.5 | 100% |
| Qwen2.5 Coder 1.5B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.5 | 60.5 | 45.8 | 77.0 | 1221 | 1016 | 0.10 | 26.8 | 5.1 | 100% |
| Internlm2 1.8B Chat V2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.8 | —unranked | 43.0 | 73.2 | 1049 | 822 | 0.10 | 82.0 | 14.3 | 100% |
| Qwen 4B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 4 | 30.2 | 67.5 | 72.2 | 1750 | 1243 | 0.07 | 249.2 | 56.1 | 100% |
| Deepcoder 1.5B Previewfine-tune of DeepSeek-R1-Distill-Qwen 1.5Bcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.5 | 55.8 | 44.9 | 66.9 | 1295 | 1007 | 0.10 | 26.8 | 5.1 | 100% |
| Exaone Deep 2.4B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2.4 | 54.7 | 31.2 | 65.1 | 777 | 615 | 0.14 | 76.5 | 11.9 | 100% |
| Deepseek Coder 1.3B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.3 | 22.6 | 53.6 | 64.1 | 1057 | 688 | 0.10 | 182.5 | 26.9 | 100% |
| Stablelm2 1.6B Chat | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.6 | 24.4 | 47.9 | 62.2 | 794 | 982 | 0.10 | 44.7 | 7.5 | 100% |
| Qwen2 Math 1.5B Instructmath model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.5 | 33.1 | 45.2 | 62.0 | 1217 | 1324 | 0.10 | 11.6 | 1.0 | 100% |
| Qwen2 Math 1.5B Instructmath model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.5 | —unranked | 44.1 | 61.1 | 1220 | 1311 | 0.10 | 9.2 | 2.1 | 100% |
| Falcon3 3B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 64.8 | 27.9 | 60.3 | 674 | 518 | 0.15 | 82.8 | 2.0 | 100% |
| Qwen2.5-14B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 14.8 | 91.3 | 21.7 | 59.0 | 555 | 416 | 0.18 | 141.0 | 34.8 | 100% |
| Lfm3 VL 2B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 72.2 | 41.9 | 58.7 | 1285 | 1259 | 0.15 | 17.3 | 2.0 | 100% |
| Granite3.3 2B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 68.6 | 28.6 | 58.5 | 645 | 546 | 0.19 | 78.0 | 12.6 | 100% |
| Lfm3 1.7B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.7 | 70.2 | 42.1 | 58.2 | 1276 | 1268 | 0.15 | 19.0 | 1.6 | 100% |
| Falcon3 3B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | —unranked | 26.0 | 57.9 | 685 | 522 | 0.15 | 93.8 | 14.0 | 100% |
| Granite3.1 Dense 2B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 60.6 | 28.8 | 56.8 | 635 | 534 | 0.18 | 70.0 | 12.6 | 100% |
| Smollm2 1.7B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1.7 | 47.1 | 41.7 | 53.3 | 671 | 679 | 0.11 | 95.1 | 14.2 | 100% |
| Smallthinker 3B Previewfine-tune of Qwen 2.5 3B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 61.1 | 22.1 | 53.2 | 672 | 587 | 0.21 | 46.7 | 7.2 | 100% |
| Codegemma 7B Instruct V1.1code model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 59.5 | 10.3 | 52.6 | 199 | 356 | 0.50 | 256.7 | 36.1 | 100% |
| Qwen2.5-VL-3B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.8 | 72.3 | 21.9 | 51.4 | 659 | 574 | 0.21 | 24.6 | 5.5 | 100% |
| SmolLM3-3B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 3.1 | 78.0 | 22.2 | 50.8 | 684 | 595 | 0.20 | 75.8 | 12.0 | 100% |
| Hermes3 3B Llama3.2fine-tune of Llama 3.2 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 48.5 | 23.4 | 50.7 | 502 | 376 | 0.19 | 106.9 | 17.4 | 100% |
| Deepseek R1 1.5B Qwen Distill | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.5 | 48.9 | 44.9 | 50.6 | 1287 | 1006 | 0.10 | 27.0 | 5.1 | 100% |
| Granite4.2 3B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 78.2 | 24.3 | 50.1 | 472 | 347 | 0.20 | 84.6 | 13.2 | 100% |
| Granite4 1B H | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 1 | 66.8 | 24.9 | 50.1 | 616 | 631 | 0.35 | 15.8 | 3.4 | 100% |
| Qwen2 1.5B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.5 | 49.7 | 43.1 | 49.8 | 1284 | 1000 | 0.12 | — | — | — |
| Llama-3.2-3B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.2 | 75.7 | 20.7 | 49.8 | 497 | 380 | 0.20 | 67.8 | 9.9 | 100% |
| Qwen2.5-Coder-3B-Instructcode model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.1 | 72.5 | 21.3 | 49.3 | 665 | 579 | 0.21 | 22.8 | 4.7 | 100% |
| granite-4.1-3b | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.4 | 80.7 | 20.1 | 49.0 | 472 | 346 | 0.25 | 48.2 | 7.8 | 100% |
| Qwen2.5-3B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 to 8 at a time | 3.1 | 76.8 | 21.9 | 44.9 | 681 | 591 | 0.20 | 23.4 | 4.7 | 100% |
| Phi-3-mini-4k-instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.8 | 63.4 | 17.6 | 44.5 | 329 | 371 | 0.27 | 57.8 | 8.9 | 100% |
| Qwen2.5 1.5B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.5 | 68.5 | 45.5 | 43.8 | 1222 | 1003 | 0.10 | 27.0 | 5.1 | 100% |
| Phi-4-mini-instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.8 | 81.6 | 17.1 | 43.7 | 514 | 373 | 0.23 | 73.3 | 11.4 | 100% |
| Granite3 Dense 2B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 58.3 | 29.7 | 43.5 | 626 | 652 | 0.18 | 20.1 | 4.4 | 100% |
| Qwen3 1.7B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.7 | 76.0 | 38.5 | 43.5 | 1057 | 779 | 0.13 | 95.5 | 16.5 | 100% |
| Gemma 2B Instruct V1.1 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 41.9 | 29.3 | 43.4 | 861 | 859 | 0.16 | 13.5 | 3.3 | 100% |
| Codegemma 2B Code V1.1code model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 15.3 | 29.1 | 43.3 | 858 | 856 | 0.14 | 10.8 | 3.3 | 100% |
| Qwen3-4B-Instruct-2507 | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 4 | 90.3 | 17.1 | 43.2 | 412 | 308 | 0.26 | 85.7 | 12.5 | 100% |
| medgemma-4b-itmedical model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 4.3 | 74.3 | 17.1 | 42.2 | 490 | 491 | 0.31 | 26.6 | 4.5 | 100% |
| Lfm2.5 8B A1B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 8 | 75.7 | 43.4 | 41.6 | 942 | 910 | 0.23 | 9.5 | 5.5 | 100% |
| Phi-3.5-mini-instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 3.8 | 67.1 | 17.6 | 41.6 | 276 | 156 | 0.28 | 219.2 | 28.4 | 100% |
| Qwen3-4B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 4 | 89.2 | 17.0 | 41.3 | 484 | 343 | 0.27 | 66.5 | 12.7 | 100% |
| Granite4 Tiny H | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 79.1 | 24.2 | 40.9 | 442 | 476 | 0.42 | 19.4 | 6.1 | 100% |
| Deepscaler 1.5B Previewfine-tune of DeepSeek-R1-Distill-Qwen 1.5Bmath model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.5 | 52.1 | 45.5 | 40.0 | 1287 | 1006 | 0.10 | 29.1 | 5.1 | 100% |
| Laguna XS 2.1 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | — | 91.2 | 23.1 | 40.0 | 323 | 289 | 0.64 | 68.4 | 23.2 | 100% |
| Gemma2 2B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2 | 68.2 | 27.6 | 39.9 | 719 | 716 | 0.19 | 45.9 | 4.7 | 100% |
| Stablelm Zephyr 3Bfine-tune of StableLM 3B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2.8 | 43.7 | 26.5 | 39.6 | 407 | 524 | 0.21 | 72.5 | 12.5 | 100% |
| Spark-X2.6-4B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 4 | 85.0 | 19.7 | 38.7 | 469 | 438 | 0.38 | 58.2 | 5.3 | 100% |
| gemma-3-4b-it | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 4.3 | 82.1 | 16.9 | 38.6 | 493 | 490 | 0.31 | 27.1 | 4.5 | 100% |
| Stable Code 3B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2.8 | 22.8 | 26.4 | 38.6 | 404 | 224 | 0.21 | 316.1 | 44.9 | 100% |
| Dolphin Phi 2.7B V2.6fine-tune of Phi-2 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2.7 | 44.0 | 25.0 | 38.4 | 503 | 515 | 0.17 | 38.8 | 7.2 | 100% |
| Phi 2.7B Chat V2 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 2.7 | 39.9 | 26.0 | 38.2 | 508 | 529 | 0.17 | 37.3 | 7.2 | 100% |
| Apertus-8B-Instruct-2509 | Vulkan (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8.1 | 63.6 | 13.5 | 32.9 | 188 | 115 | 0.38 | 10.6 | 22.3 | 100% |
| Qwen3.5 2B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 2 | 75.1 | 33.3 | 32.2 | 961 | 939 | 0.20 | 8.3 | 1.9 | 100% |
| Granite3.2 2B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 2 | 63.8 | 30.1 | 32.1 | 635 | 538 | 0.18 | 71.5 | 12.6 | 100% |
| Qwen3-4B-Thinking-2507 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 4 | 83.5 | 21.5 | 32.1 | 511 | 352 | 0.24 | 126.5 | 2.5 | 100% |
| Flint 40B A2B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 40 | 89.0 | 25.9 | 31.9 | 342 | 323 | 0.66 | 60.3 | 24.3 | 100% |
| gpt-oss-20b | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 20.9 | 95.0 | 14.9 | 31.5 | 437 | 429 | 0.62 | 40.6 | 13.5 | 100% |
| Granite4 Micro H | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 3 | 76.6 | 13.4 | 30.9 | 364 | 370 | 0.60 | 23.9 | 4.0 | 100% |
| Medgemma1.5 4B ITfine-tune of Gemma 3medical model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 4 | 71.6 | 20.2 | 30.6 | 508 | 508 | 0.28 | 39.4 | 5.9 | 100% |
| Openthinker 7B V2fine-tune of Qwen 2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 76.8 | 12.6 | 30.1 | 308 | 280 | 0.36 | 59.6 | 12.5 | 100% |
| Gemma4 E2B IT | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 5 | 93.8 | 17.2 | 29.8 | 673 | 593 | 0.39 | 18.3 | 0.6 | 100% |
| Lfm2 24B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 1 at a time | 24 | 87.3 | 32.5 | 29.5 | 583 | 591 | 0.51 | 16.1 | 15.0 | 100% |
| Mistral-7B-Instruct-v0.2 | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 7.2 | 59.2 | 10.2 | 29.0 | 274 | 225 | 0.43 | 131.9 | 22.0 | 100% |
| Exaone Deep 7.8B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7.8 | 65.7 | 12.2 | 28.9 | 276 | 230 | 0.37 | 131.4 | 22.3 | 100% |
| Falcon3 7B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 76.2 | 12.5 | 28.4 | 290 | 250 | 0.37 | 103.5 | 4.6 | 100% |
| Qwen2.5-VL-7B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 8.3 | 82.0 | 10.2 | 28.4 | 306 | 278 | 0.40 | 64.0 | 13.2 | 100% |
| Qwen2.5-7B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 7.6 | 85.6 | 10.1 | 28.3 | 306 | 279 | 0.39 | 34.1 | 8.7 | 100% |
| Internlm2 7B Chat V2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 67.4 | 12.0 | 28.3 | 252 | 217 | 0.41 | 135.3 | 22.2 | 100% |
| Tulu3 8Bfine-tune of Llama 3.1 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 73.0 | 12.0 | 28.2 | 274 | 231 | 0.37 | 126.5 | 22.4 | 100% |
| Qwen2.5-Coder-7B-Instructcode model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 7.6 | 84.5 | 9.8 | 28.2 | 308 | 279 | 0.40 | 62.7 | 12.5 | 100% |
| Meta-Llama-3-8B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 8 | 74.7 | 9.6 | 27.3 | 272 | 271 | 0.41 | 38.2 | 9.2 | 100% |
| Codellama 7B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 7 | 53.0 | 13.4 | 27.2 | 252 | 181 | 0.40 | 248.4 | 38.8 | 100% |
| Llama-2-7b-chat-hf | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 to 8 at a time | 6.7 | 43.8 | 11.0 | 26.7 | 266 | 291 | 0.43 | 68.4 | 12.8 | 100% |
| Qwen3 VL 2B Thinking | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 2 | 53.7 | 27.2 | 26.4 | 950 | 806 | 0.14 | 19.0 | 3.6 | 100% |
| Sailor2 8B Chatfine-tune of Qwen 2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 57.3 | 10.6 | 26.3 | 223 | 204 | 0.42 | 86.3 | 14.2 | 100% |
| Qwen3 VL 2B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 2 | 77.4 | 26.5 | 26.2 | 933 | 788 | 0.15 | 16.5 | 3.6 | 100% |
| Qwen3-30B-A3B-Instruct-2507 | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 30 | 95.0 | 20.4 | 26.1 | 384 | 304 | 0.49 | 100.3 | 31.7 | 100% |
| Gemma3n E2B IT | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 5 | 79.9 | 13.8 | 26.1 | 484 | 486 | 0.35 | 19.6 | 3.4 | 100% |
| Qwen3-Coder-30B-A3B-Instructcode model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 30 | 95.5 | 20.1 | 26.1 | 382 | 303 | 0.49 | 100.5 | 31.7 | 100% |
| Granite3.3 8B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 79.4 | 11.3 | 25.9 | 254 | 203 | 0.48 | 154.5 | 26.9 | 100% |
| Qwen3-30B-A3B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 30 | 92.8 | 19.7 | 25.5 | 380 | 301 | 0.52 | 101.4 | 31.7 | 100% |
| Gemma4 E4B IT | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 95.5 | 13.2 | 25.3 | 416 | 391 | 0.41 | 30.5 | 1.2 | 100% |
| Codeqwen 7B Chat V1.5code model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 49.4 | 11.8 | 25.2 | 316 | 279 | 0.38 | 68.8 | 13.6 | 100% |
| Granite4.2 8B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 81.9 | 11.1 | 24.8 | 244 | 197 | 0.42 | 169.2 | 27.1 | 100% |
| Deepseek Llm 7B Chat | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 54.3 | 12.2 | 24.8 | 278 | 299 | 0.37 | 118.1 | 20.3 | 100% |
| Falcon3 7B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | —unranked | 11.6 | 24.7 | 279 | 240 | 0.38 | 112.2 | 20.1 | 100% |
| Cogito 8B V1 Preview Llamafine-tune of Llama 3.1 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 83.9 | 11.2 | 24.6 | 275 | 226 | 0.37 | 128.9 | 22.4 | 100% |
| YI Coder 9B Chatcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 9 | 59.0 | 10.5 | 24.6 | 219 | 189 | 0.54 | 98.7 | 18.5 | 100% |
| YI 9B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 9 | 70.5 | 10.0 | 24.6 | 229 | 242 | 0.55 | 33.0 | 8.6 | 100% |
| Olmo-3-7B-Think | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7.3 | 77.7 | 13.5 | 24.0 | 257 | 229 | 0.61 | 209.9 | 4.5 | 100% |
| Marco O1 7Bfine-tune of Qwen 2 7B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 70.5 | 11.8 | 23.7 | 316 | 276 | 0.37 | 58.4 | 12.5 | 100% |
| Llama-3.1-8B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 8 | 81.0 | 9.6 | 23.6 | 278 | 229 | 0.41 | 70.1 | 13.7 | 100% |
| Qwen 7B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 7 | 44.1 | 12.3 | 23.2 | 249 | 184 | 0.42 | 256.8 | 39.2 | 100% |
| EXAONE-3.5-7.8B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 7.8 | 82.0 | 9.5 | 23.0 | 278 | 229 | 0.40 | 70.1 | 13.6 | 100% |
| Olmo-3-7B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 to 8 at a time | 7.3 | 82.9 | 10.2 | 22.8 | 235 | 195 | 0.65 | 141.4 | 14.4 | 100% |
| Deepseek Coder 6.7B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 6.7 | 34.7 | 13.6 | 22.5 | 277 | 197 | 0.41 | 262.0 | 38.8 | 100% |
| Ministral 3 8B Instruct 2512 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 88.7 | 11.0 | 22.2 | 271 | 201 | 0.41 | 145.3 | 23.8 | 100% |
| gemma-4-26B-A4B-it | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 26 | 97.8 | 15.7 | 21.8 | 366 | 330 | 0.61 | 70.1 | 22.6 | 100% |
| Hermes-3-Llama-3.1-8B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 to 8 at a time | 8 | 67.6 | 9.6 | 21.3 | 276 | 229 | 0.45 | 73.0 | 13.7 | 100% |
| Sage-A1.5-Q4_K_M | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 1 at a time | 36 | 94.5 | 22.6 | 21.2 | 395 | 375 | 0.57 | 25.2 | 21.8 | 100% |
| Falcon3 10B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 10 | —unranked | 9.2 | 21.2 | 177 | 154 | 0.52 | 142.2 | 28.3 | 100% |
| Falcon3 10B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 10 | 81.3 | 9.0 | 20.9 | 200 | 174 | 0.53 | 146.1 | 6.3 | 100% |
| DeepSeek-R1-Distill-Qwen-7B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7.6 | 73.6 | 13.8 | 20.7 | 342 | 301 | 0.31 | 54.2 | 4.7 | 100% |
| Gemma 3n E4B IT | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 8 | 86.1 | 12.1 | 20.0 | 282 | 277 | 0.54 | 22.4 | 4.2 | 100% |
| Qwen3-VL-30B-A3B-Instruct | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 30 | 92.9 | 26.9 | 19.8 | 298 | 167 | 0.74 | 12.6 | 20.6 | 100% |
| Falcon2 11B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 11 | —unranked | 8.1 | 19.7 | 184 | 187 | 0.57 | 31.8 | 10.9 | 100% |
| Qwen2 Math 7B Instructmath model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 66.0 | 12.2 | 19.4 | 307 | 320 | 0.36 | 18.3 | 4.7 | 100% |
| GLM-4.7-Flash | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 31.2 | 88.9 | 14.1 | 19.3 | 187 | 104 | 0.58 | 63.9 | 26.0 | 100% |
| Nemotron3 33B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 1 at a time | 33 | 86.2 | 20.0 | 18.9 | 372 | 367 | 0.83 | 24.0 | 25.0 | 100% |
| DeepSeek-R1-0528-Qwen3-8B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8.2 | 79.6 | 12.2 | 18.4 | 289 | 230 | 0.39 | 135.4 | 5.2 | 100% |
| Deepseek R1 8B Llama Distill | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 81.7 | 11.9 | 18.2 | 277 | 231 | 0.37 | 136.3 | 22.4 | 100% |
| Qwen3.6-35B-A3B | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 35 | 94.6 | 22.1 | 17.8 | 276 | 284 | 1.63 | 15.2 | 23.0 | 100% |
| Mistral 7B Instruct V0.3 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 59.8 | 11.6 | 17.5 | 272 | 223 | 0.44 | 125.9 | 22.0 | 100% |
| Qwen2 Math 7B Instructmath model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | —unranked | 11.2 | 17.4 | 309 | 313 | 0.37 | 21.9 | 6.5 | 100% |
| Nemotron 3.5 Lightning 30B A3B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 1 at a time | 30 | 92.8 | 17.7 | 17.3 | 352 | 352 | 0.85 | 22.6 | 25.4 | 100% |
| Nemotron Cascade 2 30B A3B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 1 at a time | 30 | 85.6 | 20.5 | 17.1 | 372 | 368 | 0.82 | 22.4 | 24.4 | 100% |
| NVIDIA Nemotron 3 Nano 30B-A3B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 1 at a time | 30 | 83.9 | 18.4 | 17.0 | 349 | 346 | 0.80 | 22.2 | 24.4 | 100% |
| Phi3 14B Medium 128K Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 14 | 78.0 | — | 17.0 | — | 126 | — | — | — | — |
| Olmo2 7B 1124 Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 65.4 | 11.8 | 16.9 | 261 | 289 | 0.37 | 112.3 | 21.6 | 100% |
| Mathstral 7B V0.1fine-tune of Mistral 7Bmath model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 68.0 | 12.0 | 16.8 | 272 | 224 | 0.42 | 126.8 | 22.0 | 100% |
| Qwen3-VL-4B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 4.4 | 89.9 | 17.3 | 16.8 | 455 | 364 | 0.25 | 23.8 | 5.5 | 100% |
| Qwen3.5-35B-A3B | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 35 | 94.8 | 23.2 | 16.5 | 235 | 227 | 1.41 | 15.3 | 21.7 | 100% |
| Aya 8B 23 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 55.9 | 10.9 | 16.4 | 243 | 246 | 0.40 | 84.5 | 14.2 | 100% |
| Llama2 13B Chat | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 13 | 45.3 | 7.1 | 16.4 | 148 | 160 | 0.69 | 109.3 | 21.5 | 100% |
| Aya Expanse 8B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 72.5 | 10.7 | 16.3 | 244 | 243 | 0.40 | 82.2 | 14.2 | 100% |
| Codegeex4 9B Allcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 9 | 68.7 | 10.1 | 16.3 | 262 | 229 | 0.41 | 40.3 | 11.8 | 100% |
| Opencoder 1.5B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 1.5 | 31.9 | 17.4 | 16.2 | 361 | 496 | 0.22 | 47.4 | 8.4 | 100% |
| Codellama 13B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 13 | 52.5 | 6.9 | 16.2 | 122 | 98 | 0.70 | 444.4 | 61.7 | 100% |
| Command R7B Arabic 7B 02 2025 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 81.9 | 11.0 | 16.2 | 248 | 229 | 0.51 | 92.4 | 10.5 | 100% |
| Qwen2-VL-7B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 8.3 | 72.2 | 10.2 | 16.1 | 331 | 294 | 0.43 | 55.4 | 12.8 | 100% |
| Granite4.1 8B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 92.1 | 10.4 | 15.9 | 225 | 184 | 0.50 | 189.7 | 27.1 | 100% |
| Glm4 9B Chat | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 9 | 73.3 | 9.8 | 15.9 | 261 | 228 | 0.41 | 40.6 | 11.8 | 100% |
| Qwen2.5-Coder-14B-Instructcode model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 14.8 | 87.7 | 5.2 | 15.9 | 112 | 86 | 0.74 | 203.4 | 34.8 | 100% |
| gpt-oss-120b | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 116.8 | 98.6 | 11.0 | 15.8 | 158 | 140 | 2.73 | 104.8 | 64.6 | 100% |
| Olmo2 13B 1124 Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 13 | 69.6 | 6.8 | 15.5 | 149 | 161 | 0.63 | 109.5 | 21.7 | 100% |
| Gemma 7B Instruct V1.1 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 7 | 60.8 | 10.2 | 15.2 | 198 | 200 | 0.50 | 226.1 | 36.1 | 100% |
| Granite Code 8B Instructcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 40.7 | 10.5 | 15.0 | 222 | 227 | 0.47 | 41.4 | 10.0 | 100% |
| Ministral 3 14B Instruct (2512) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 14 | 92.7 | 5.6 | 15.0 | 178 | 135 | 0.73 | 173.0 | 30.0 | 100% |
| Qwen3-Coder-Nextcode model | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 79.7 | 96.4 | 18.1 | 14.7 | 177 | 180 | 1.93 | 75.7 | 52.2 | 100% |
| Qwen3.5-4B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 4.7 | 88.7 | 15.1 | 14.6 | 360 | 348 | 0.42 | 11.1 | 3.6 | 100% |
| Granite3 Dense 8B Instruct | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 67.6 | 10.5 | 14.5 | 255 | 257 | 0.49 | 46.6 | 10.5 | 100% |
| Qwen3-8B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 8.2 | 91.5 | 9.3 | 14.4 | 249 | 203 | 0.50 | 148.6 | 24.7 | 100% |
| Command R7B (12-2024) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 8 | 74.8 | 8.7 | 14.0 | 265 | 266 | 0.56 | 60.2 | 10.4 | 100% |
| Qwen 14B Chat V1.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 14 | 54.8 | 6.3 | 13.9 | 136 | 105 | 0.73 | 462.5 | 62.6 | 100% |
| Solar 10.7B Instruct V1 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 10.7 | 58.7 | 8.7 | 13.6 | 183 | 191 | 0.61 | 47.3 | 13.1 | 100% |
| Aya Expanse 32B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 32 | 78.8 | 10.0 | 13.5 | 281 | 88 | 0.42 | 89.2 | 31.4 | 100% |
| Gemma 4 12B IT | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 12 | 98.6 | 8.1 | 13.1 | 157 | 148 | 0.94 | 89.0 | 8.9 | 100% |
| Nemotron 3.5 Bolt E4B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 8 | 90.3 | 9.8 | 13.0 | 384 | 365 | 0.39 | 41.7 | 5.7 | 100% |
| Mistral Nemo 12B Instruct 2407 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 12 | 75.9 | 8.2 | 12.5 | 177 | 153 | 0.54 | 143.5 | 29.2 | 100% |
| Granite4 Small H | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 32 | 93.9 | 8.4 | 12.4 | 133 | 135 | 1.44 | 37.9 | 23.9 | 100% |
| gemma-4-31B-it | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 4 at a time | 31.3 | 99.3 | 3.8 | 11.8 | 64 | 58 | 2.37 | 122.7 | 11.2 | 100% |
| gemma-2-9b-it | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 9.2 | 79.9 | 7.6 | 11.7 | 172 | 171 | 0.69 | 154.4 | 14.7 | 100% |
| Translategemma 12B ITfine-tune of Gemma 3translation model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 12 | 89.6 | 7.6 | 11.5 | 177 | 176 | 0.77 | 97.3 | 13.1 | 100% |
| Mistral Small 24B Instruct 2501 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 24 | 92.1 | 4.2 | 10.6 | 98 | 88 | 1.03 | 204.3 | 36.5 | 100% |
| Ornith 9B | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 9 | 90.0 | 10.8 | 10.5 | 262 | 256 | 0.80 | 12.3 | 5.9 | 100% |
| Magistral Small (2506) | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 24 | 49.8 | 4.5 | 10.5 | 107 | 95 | 1.04 | 153.8 | 14.3 | 100% |
| Deepcoder 14B Previewfine-tune of DeepSeek-R1-Distill-Qwen 14Bcode model | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 14 | 74.5 | 6.7 | 10.3 | 146 | 108 | 0.67 | 199.0 | 34.9 | 100% |
| Mistral Small3.1 24B Instruct 2503 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 24 | 90.1 | 4.3 | 10.3 | 79 | 75 | 1.34 | 154.5 | 36.6 | 100% |
| Cogito 14B V1 Preview Qwenfine-tune of Qwen 2.5 | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 14 | 90.3 | 6.6 | 10.2 | 140 | 106 | 0.75 | 200.3 | 34.9 | 100% |
| DeepSeek-R1-Distill-Qwen-14B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 14.8 | 85.3 | 7.1 | 10.0 | 147 | 104 | 0.62 | 173.4 | 9.0 | 100% |
| gemma-3-12b-it | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 12.2 | 92.7 | 6.1 | 9.5 | 139 | 138 | 0.84 | 115.4 | 13.1 | 100% |
| Mistral Small 3.2 24B Instruct (2506) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 24 | 92.9 | 3.1 | 9.4 | 99 | 80 | 1.72 | 176.9 | 36.5 | 100% |
| Qwen3-VL-8B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 8.8 | 94.9 | 9.4 | 9.2 | 261 | 229 | 0.43 | 25.9 | 7.8 | 100% |
| phi-4 | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 14.7 | 85.9 | 5.0 | 8.9 | 135 | 122 | 0.75 | 210.6 | 36.2 | 100% |
| Devstral Small 2 24B Instruct (2512)code model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 24 | 92.1 | 3.1 | 8.9 | 80 | 65 | 1.54 | 177.6 | 36.5 | 100% |
| Muse Glimmer 30B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 30 | 97.9 | 3.6 | 8.8 | 96 | 96 | 2.12 | 42.3 | 18.3 | 100% |
| Apertus-8B-Instruct-2509 | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 1 at a time | 8.1 | —unranked | 9.4 | 8.7 | 294 | 243 | 0.42 | 115.3 | 22.3 | 100% |
| Qwen3-14B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 14.8 | 91.7 | 5.0 | 8.7 | 153 | 113 | 0.81 | 171.5 | 30.9 | 100% |
| Mixtral 8x7B Instruct V0.1 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 1 at a time | 56 | 56.8 | 7.6 | 8.5 | 137 | 127 | 1.32 | 140.0 | 46.5 | 100% |
| Qwen3.5-9B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 9.7 | 92.0 | 8.4 | 8.2 | 225 | 222 | 0.69 | 13.5 | 6.1 | 100% |
| Llama 4 Scout 17B-16E Instruct | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 109 | 93.6 | 17.0 | 8.2 | 147 | 163 | 2.53 | 81.4 | 70.4 | 100% |
| Granite4.2 30B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 30 | 92.8 | 3.4 | 8.2 | 62 | 60 | 1.37 | 243.3 | 52.3 | 100% |
| Devstral 24B Small 2505code model | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 24 | 88.9 | 4.2 | 8.0 | 122 | 86 | 1.54 | 165.4 | 36.5 | 100% |
| Ornith-1.5-9B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 1 at a time | 9.7 | 89.0 | 8.1 | 8.0 | 220 | 215 | 0.71 | 11.7 | 6.4 | 100% |
| Command R 35B 08 2024 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 35 | 75.2 | 3.2 | 7.7 | 61 | 55 | 1.34 | 159.0 | 42.2 | 100% |
| Exaone3.5 32B Instruct | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 32 | 85.0 | 3.1 | 7.7 | 66 | 57 | 1.31 | 256.6 | 53.9 | 100% |
| Qwen2.5-Coder-32B-Instructcode model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 32.8 | 90.5 | 2.3 | 7.6 | 52 | 46 | 1.61 | 244.5 | 54.2 | 100% |
| Qwen2.5-32B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 32.8 | 93.5 | 2.3 | 7.6 | 53 | 46 | 1.61 | 251.7 | 54.2 | 100% |
| GLM-4.5-Air | Vulkan (Ollama 0.40.4, flash attention, 16k context; 2 parallel slots) | 2 at a time | 110.5 | 92.8 | 6.0 | 7.6 | 53 | 29 | 3.15 | 110.5 | 76.8 | 100% |
| Nanbeige_Nanbeige4.3-33B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 33 | 90.4 | 3.1 | 7.4 | 71 | 58 | 1.54 | 200.9 | 48.9 | 100% |
| Codellama 34B Instructcode model | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 34 | 61.2 | 3.1 | 7.4 | 58 | 51 | 1.62 | 190.2 | 46.7 | 100% |
| Olmo 3.1 32B Instruct | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 32 | 89.6 | 3.1 | 7.4 | 56 | 54 | 2.61 | 128.4 | 27.7 | 100% |
| dolphin-2.9.1-yi-1.5-34b | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 34.4 | 73.0 | 2.4 | 7.2 | 51 | 51 | 1.94 | 123.1 | 37.2 | 100% |
| YI 34B Chat V1.5 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 34 | 75.2 | 3.0 | 7.1 | 57 | 59 | 1.62 | 71.7 | 28.8 | 100% |
| Stablelm2 12B Chat | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 8 at a time | 12 | 42.4 | 4.7 | 6.7 | 84 | 103 | 0.85 | 50.2 | 14.0 | 100% |
| Codestral 22B V0.1code model | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 22 | 74.8 | 4.4 | 6.5 | 76 | 64 | 1.17 | 242.0 | 44.0 | 100% |
| Mistral Small 22B Instruct 2409 | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 22 | 84.6 | 4.2 | 6.1 | 76 | 65 | 1.19 | 244.6 | 44.0 | 100% |
| Deepseek V2 16B Lite Chat | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | 1 at a time | 16 | 58.2 | 12.2 | 5.9 | 244 | 109 | 0.49 | 243.4 | 46.8 | 100% |
| Deepseek Coder 33B Instructcode model | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 33 | 33.4 | 3.0 | 5.9 | 64 | 54 | 1.62 | 239.9 | 53.5 | 100% |
| Gemma2 27B Instruct | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 27 | 85.6 | 3.7 | 5.5 | 67 | 68 | 1.52 | 157.6 | 27.0 | 100% |
| Medgemma 27B ITfine-tune of Gemma 3medical model | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 27 | 90.2 | 3.5 | 5.3 | 88 | 89 | 1.64 | 126.6 | 22.1 | 100% |
| Granite4.1 30B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 30 | 92.3 | 3.4 | 5.0 | 62 | 59 | 1.49 | 242.1 | 52.3 | 100% |
| DeepSeek-R1-Distill-Qwen-32B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 32.8 | 89.1 | 3.1 | 4.8 | 56 | 49 | 1.36 | 281.5 | 19.9 | 100% |
| QwQ 32B | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) | 8 at a time | 32.8 | 92.6 | 3.2 | 4.7 | 72 | 60 | 1.46 | 271.7 | 54.2 | 100% |
| gemma-3-27b-it | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 27.4 | 95.8 | 2.7 | 4.7 | 71 | 72 | 1.91 | 124.9 | 22.1 | 100% |
| DeepSeek-Coder-V2-Lite-Instructcode model | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 15.7 | 79.0 | 11.1 | 4.6 | 160 | 110 | 0.50 | 283.9 | 46.8 | 100% |
| Qwen3-32B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 32.8 | 93.6 | 2.3 | 4.1 | 54 | 46 | 1.76 | 246.2 | 55.0 | 100% |
| Qwen3.6-27B | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 27.8 | 95.1 | — | 3.5 | 55 | 55 | — | 11.8 | 17.3 | 100% |
| Llama-3.3-70B-Instruct | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 4 at a time | 70.6 | 94.3 | 1.4 | 3.4 | 16 | 11 | 4.30 | 51.1 | 63.8 | 100% |
| Qwen3.8-27B | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 27.8 | 95.4 | 3.5 | 3.2 | 52 | 51 | 4.13 | 12.2 | 17.6 | 100% |
| Qwen3-VL-32B-Instruct | Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) | 1 at a time | 33.4 | 94.7 | 3.2 | 2.9 | 33 | 22 | 1.49 | 15.6 | 24.5 | 100% |
| Qwen2.5-VL-32B-Instruct | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 1 at a time | 33.5 | 89.7 | 2.3 | 2.7 | 52 | — | 1.61 | 248.4 | 54.9 | 100% |
| Qwen3.5-27B | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | 8 at a time | 27.8 | 96.2 | 2.7 | 2.6 | 71 | 69 | 2.06 | 27.7 | 17.6 | 100% |
A dash means the metric was not measured, which is different from 0. GPU memory on our server (1 to 8 answers at a time) is what Ollama reported for the loaded model with this server's settings (parallel slots and context length), so it is not what one user needs, in either direction; Answers generated is how many benchmark answers the runner kept in flight for the model, and a range such as “4 to 8 at a time” means the setting changed during its run (see the methodology). Load time is the wall time to load the model after a fresh unload; the operating system's file cache can make it faster than a cold disk read. Decode speed uses up to 256 generated tokens; a model that stops earlier is still measured. Raw numbers: speed.json. To see which models fit the memory you have, use the hardware picker. The method is described in the methodology.