Speed on this laptop

Every model gets a short performance test before its benchmark run. All numbers on this page were measured on one AMD Ryzen AI Max+ 395 laptop through Ollama (GPU label recorded with the runs: AMD Radeon 8060S (Ryzen AI Max+ 395, 128 GB unified, ~78 GB GPU)); backends in use: ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after); ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots); ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots); ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots); Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots); Vulkan (Ollama 0.40.4, flash attention, 16k context; 2 parallel slots); Vulkan (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots). They compare models with each other on this machine, not with other hardware. Speeds are not comparable across backends: the backend each model was measured on is shown in its row.

Speed against quality

Overall score against decode speed with 4 parallel streams One dot per ranked model. Horizontal axis: total tokens per second with 4 parallel streams, logarithmic scale. Vertical axis: Overall score. Dot size and colour show the size class; the dots no other model beats on both axes are labelled with the model name, and every dot has a tooltip and a link to its model page. 10 30 50 70 90 2 5 10 20 50 100 200 500 Decode speed, 4 parallel streams (tokens per second, log scale) Overall (0 to 100) gemma-4-31B-it: Overall 99.3, 11.8 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)gemma-4-31B-it gpt-oss-120b: Overall 98.6, 15.8 tok/s with 4 parallel streams, > 35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots)gpt-oss-120b Gemma 4 12B IT: Overall 98.6, 13.1 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Muse Glimmer 30B: Overall 97.9, 8.8 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) gemma-4-26B-A4B-it: Overall 97.8, 21.8 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)gemma-4-26B-A4B-it Qwen3-Coder-Next: Overall 96.4, 14.7 tok/s with 4 parallel streams, > 35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Qwen3.5-27B: Overall 96.2, 2.6 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) gemma-3-27b-it: Overall 95.8, 4.7 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3-Coder-30B-A3B-Instruct: Overall 95.5, 26.1 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)Qwen3-Coder-30B-A3B-Instruct Gemma4 E4B IT: Overall 95.5, 25.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3.8-27B: Overall 95.4, 3.2 tok/s with 4 parallel streams, 16–35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Qwen3.6-27B: Overall 95.1, 3.5 tok/s with 4 parallel streams, 16–35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Qwen3-30B-A3B-Instruct-2507: Overall 95.0, 26.1 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)Qwen3-30B-A3B-Instruct-2507 gpt-oss-20b: Overall 95.0, 31.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots)gpt-oss-20b Qwen3-VL-8B-Instruct: Overall 94.9, 9.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3.5-35B-A3B: Overall 94.8, 16.5 tok/s with 4 parallel streams, 16–35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Qwen3-VL-32B-Instruct: Overall 94.7, 2.9 tok/s with 4 parallel streams, 16–35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Qwen3.6-35B-A3B: Overall 94.6, 17.8 tok/s with 4 parallel streams, 16–35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Sage-A1.5-Q4_K_M: Overall 94.5, 21.2 tok/s with 4 parallel streams, > 35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Llama-3.3-70B-Instruct: Overall 94.3, 3.4 tok/s with 4 parallel streams, > 35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Granite4 Small H: Overall 93.9, 12.4 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Gemma4 E2B IT: Overall 93.8, 29.8 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3-32B: Overall 93.6, 4.1 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Llama 4 Scout 17B-16E Instruct: Overall 93.6, 8.2 tok/s with 4 parallel streams, > 35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Qwen2.5-32B-Instruct: Overall 93.5, 7.6 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3-VL-30B-A3B-Instruct: Overall 92.9, 19.8 tok/s with 4 parallel streams, 16–35B, Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) Mistral Small 3.2 24B Instruct (2506): Overall 92.9, 9.4 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Nemotron 3.5 Lightning 30B A3B: Overall 92.8, 17.3 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Granite4.2 30B: Overall 92.8, 8.2 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) GLM-4.5-Air: Overall 92.8, 7.6 tok/s with 4 parallel streams, > 35B, Vulkan (Ollama 0.40.4, flash attention, 16k context; 2 parallel slots) Qwen3-30B-A3B: Overall 92.8, 25.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) gemma-3-12b-it: Overall 92.7, 9.5 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Ministral 3 14B Instruct (2512): Overall 92.7, 15.0 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) QwQ 32B: Overall 92.6, 4.7 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Granite4.1 30B: Overall 92.3, 5.0 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Granite4.1 8B: Overall 92.1, 15.9 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Mistral Small 24B Instruct 2501: Overall 92.1, 10.6 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Devstral Small 2 24B Instruct (2512): Overall 92.1, 8.9 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3.5-9B: Overall 92.0, 8.2 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3-14B: Overall 91.7, 8.7 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3-8B: Overall 91.5, 14.4 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen2.5-14B-Instruct: Overall 91.3, 59.0 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)Qwen2.5-14B-Instruct Laguna XS 2.1: Overall 91.2, 40.0 tok/s with 4 parallel streams, size unknown, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen2.5-Coder-32B-Instruct: Overall 90.5, 7.6 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Nanbeige_Nanbeige4.3-33B: Overall 90.4, 7.4 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Nemotron 3.5 Bolt E4B: Overall 90.3, 13.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen3-4B-Instruct-2507: Overall 90.3, 43.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Cogito 14B V1 Preview Qwen: Overall 90.3, 10.2 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Medgemma 27B IT: Overall 90.2, 5.3 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Mistral Small3.1 24B Instruct 2503: Overall 90.1, 10.3 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Ornith 9B: Overall 90.0, 10.5 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3-VL-4B-Instruct: Overall 89.9, 16.8 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen2.5-VL-32B-Instruct: Overall 89.7, 2.7 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Translategemma 12B IT: Overall 89.6, 11.5 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Olmo 3.1 32B Instruct: Overall 89.6, 7.4 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen3-4B: Overall 89.2, 41.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) DeepSeek-R1-Distill-Qwen-32B: Overall 89.1, 4.8 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Flint 40B A2B: Overall 89.0, 31.9 tok/s with 4 parallel streams, > 35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Ornith-1.5-9B: Overall 89.0, 8.0 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Devstral 24B Small 2505: Overall 88.9, 8.0 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) GLM-4.7-Flash: Overall 88.9, 19.3 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen3.5-4B: Overall 88.7, 14.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Ministral 3 8B Instruct 2512: Overall 88.7, 22.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5-Coder-14B-Instruct: Overall 87.7, 15.9 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Lfm2 24B: Overall 87.3, 29.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Nemotron3 33B: Overall 86.2, 18.9 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Gemma 3n E4B IT: Overall 86.1, 20.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) phi-4: Overall 85.9, 8.9 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Nemotron Cascade 2 30B A3B: Overall 85.6, 17.1 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Gemma2 27B Instruct: Overall 85.6, 5.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen2.5-7B-Instruct: Overall 85.6, 28.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) DeepSeek-R1-Distill-Qwen-14B: Overall 85.3, 10.0 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Exaone3.5 32B Instruct: Overall 85.0, 7.7 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Spark-X2.6-4B: Overall 85.0, 38.7 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Mistral Small 22B Instruct 2409: Overall 84.6, 6.1 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen2.5-Coder-7B-Instruct: Overall 84.5, 28.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) NVIDIA Nemotron 3 Nano 30B-A3B: Overall 83.9, 17.0 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Cogito 8B V1 Preview Llama: Overall 83.9, 24.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3-4B-Thinking-2507: Overall 83.5, 32.1 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Olmo-3-7B-Instruct: Overall 82.9, 22.8 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) gemma-3-4b-it: Overall 82.1, 38.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) EXAONE-3.5-7.8B-Instruct: Overall 82.0, 23.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen2.5-VL-7B-Instruct: Overall 82.0, 28.4 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Command R7B Arabic 7B 02 2025: Overall 81.9, 16.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite4.2 8B: Overall 81.9, 24.8 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Deepseek R1 8B Llama Distill: Overall 81.7, 18.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Phi-4-mini-instruct: Overall 81.6, 43.7 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Falcon3 10B Instruct: Overall 81.3, 20.9 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Llama-3.1-8B-Instruct: Overall 81.0, 23.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Ministral 3 3B Instruct 2512: Overall 80.9, 156.8 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots)Ministral 3 3B Instruct 2512 granite-4.1-3b: Overall 80.7, 49.0 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) gemma-2-9b-it: Overall 79.9, 11.7 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Gemma3n E2B IT: Overall 79.9, 26.1 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) DeepSeek-R1-0528-Qwen3-8B: Overall 79.6, 18.4 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Granite3.3 8B: Overall 79.4, 25.9 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite4 Tiny H: Overall 79.1, 40.9 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) DeepSeek-Coder-V2-Lite-Instruct: Overall 79.0, 4.6 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Aya Expanse 32B: Overall 78.8, 13.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Granite4.2 3B: Overall 78.2, 50.1 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) SmolLM3-3B: Overall 78.0, 50.8 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Phi3 14B Medium 128K Instruct: Overall 78.0, 17.0 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Olmo-3-7B-Think: Overall 77.7, 24.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen3 VL 2B Instruct: Overall 77.4, 26.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5-3B-Instruct: Overall 76.8, 44.9 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Openthinker 7B V2: Overall 76.8, 30.1 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite4 Micro H: Overall 76.6, 30.9 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Exaone3.5 2.4B Instruct: Overall 76.3, 256.0 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots)Exaone3.5 2.4B Instruct Falcon3 7B Instruct: Overall 76.2, 28.4 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3 1.7B: Overall 76.0, 43.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Mistral Nemo 12B Instruct 2407: Overall 75.9, 12.5 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Llama-3.2-3B-Instruct: Overall 75.7, 49.8 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Lfm2.5 8B A1B: Overall 75.7, 41.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) YI 34B Chat V1.5: Overall 75.2, 7.1 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Command R 35B 08 2024: Overall 75.2, 7.7 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen3.5 2B: Overall 75.1, 32.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Command R7B (12-2024): Overall 74.8, 14.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Codestral 22B V0.1: Overall 74.8, 6.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Meta-Llama-3-8B-Instruct: Overall 74.7, 27.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Deepcoder 14B Preview: Overall 74.5, 10.3 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) medgemma-4b-it: Overall 74.3, 42.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Lfm2.5 Thinking 1.2B: Overall 74.3, 122.0 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite4 3B: Overall 74.1, 177.1 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3.2 8B Instruct: Overall 74.1, 97.7 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) DeepSeek-R1-Distill-Qwen-7B: Overall 73.6, 20.7 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Glm4 9B Chat: Overall 73.3, 15.9 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Tulu3 8B: Overall 73.0, 28.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) dolphin-2.9.1-yi-1.5-34b: Overall 73.0, 7.2 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Granite3.1 Dense 8B Instruct: Overall 72.9, 97.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Aya Expanse 8B: Overall 72.5, 16.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5-Coder-3B-Instruct: Overall 72.5, 49.3 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen2.5-VL-3B-Instruct: Overall 72.3, 51.4 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Qwen2-VL-7B-Instruct: Overall 72.2, 16.1 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Lfm3 VL 2B: Overall 72.2, 58.7 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Medgemma1.5 4B IT: Overall 71.6, 30.6 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Phi4 Mini Reasoning 3.8B: Overall 70.9, 139.1 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) YI 9B Chat V1.5: Overall 70.5, 24.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Marco O1 7B: Overall 70.5, 23.7 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Cogito 3B V1 Preview Llama: Overall 70.4, 157.9 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Lfm3 1.7B: Overall 70.2, 58.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Olmo2 13B 1124 Instruct: Overall 69.6, 15.5 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Codegeex4 9B All: Overall 68.7, 16.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3.3 2B: Overall 68.6, 58.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5 1.5B Instruct: Overall 68.5, 43.8 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Gemma2 2B Instruct: Overall 68.2, 39.9 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Mathstral 7B V0.1: Overall 68.0, 16.8 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Hermes-3-Llama-3.1-8B: Overall 67.6, 21.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Granite3 Dense 8B Instruct: Overall 67.6, 14.5 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Internlm2 7B Chat V2.5: Overall 67.4, 28.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Phi-3.5-mini-instruct: Overall 67.1, 41.6 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Granite4 1B H: Overall 66.8, 50.1 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2 Math 7B Instruct: Overall 66.0, 19.4 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Exaone Deep 7.8B: Overall 65.7, 28.9 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Olmo2 7B 1124 Instruct: Overall 65.4, 16.9 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Falcon3 3B Instruct: Overall 64.8, 60.3 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3.2 2B Instruct: Overall 63.8, 32.1 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Apertus-8B-Instruct-2509: Overall 63.6, 32.9 tok/s with 4 parallel streams, 4–9B, Vulkan (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Phi-3-mini-4k-instruct: Overall 63.4, 44.5 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Phi3 3.8B Mini 128K Instruct: Overall 62.9, 96.1 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Codellama 34B Instruct: Overall 61.2, 7.4 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Smallthinker 3B Preview: Overall 61.1, 53.2 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Gemma 7B Instruct V1.1: Overall 60.8, 15.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3.1 Dense 2B Instruct: Overall 60.6, 56.8 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5 Coder 1.5B Instruct: Overall 60.5, 77.0 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Mistral 7B Instruct V0.3: Overall 59.8, 17.5 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3.1 Moe 3B Instruct: Overall 59.8, 93.0 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Codegemma 7B Instruct V1.1: Overall 59.5, 52.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Mistral-7B-Instruct-v0.2: Overall 59.2, 29.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) YI Coder 9B Chat: Overall 59.0, 24.6 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Solar 10.7B Instruct V1: Overall 58.7, 13.6 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3 Dense 2B Instruct: Overall 58.3, 43.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Deepseek V2 16B Lite Chat: Overall 58.2, 5.9 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) YI 6B Chat V1.5: Overall 58.0, 120.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Llama3.2 1B Instruct: Overall 58.0, 114.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Sailor2 8B Chat: Overall 57.3, 26.3 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Gemma3 1B IT: Overall 56.8, 80.1 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Mixtral 8x7B Instruct V0.1: Overall 56.8, 8.5 tok/s with 4 parallel streams, > 35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Aya 8B 23: Overall 55.9, 16.4 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Deepcoder 1.5B Preview: Overall 55.8, 66.9 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen 14B Chat V1.5: Overall 54.8, 13.9 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Exaone Deep 2.4B: Overall 54.7, 65.1 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Deepseek Llm 7B Chat: Overall 54.3, 24.8 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3 VL 2B Thinking: Overall 53.7, 26.4 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Codellama 7B Instruct: Overall 53.0, 27.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Codellama 13B Instruct: Overall 52.5, 16.2 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Deepscaler 1.5B Preview: Overall 52.1, 40.0 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Nemotron Mini 4B Instruct: Overall 52.0, 167.6 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Internlm2 1.8B Chat V2.5: Overall 51.9, 85.6 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Magistral Small (2506): Overall 49.8, 10.5 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen2 1.5B Instruct: Overall 49.7, 49.8 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Codeqwen 7B Chat V1.5: Overall 49.4, 25.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Deepseek R1 1.5B Qwen Distill: Overall 48.9, 50.6 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Hermes3 3B Llama3.2: Overall 48.5, 50.7 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3 Moe 3B Instruct: Overall 47.7, 218.4 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Smollm2 1.7B Instruct: Overall 47.1, 53.3 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite4 350M H: Overall 45.6, 128.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Llama2 13B Chat: Overall 45.3, 16.4 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Falcon3 1B Instruct: Overall 44.3, 107.4 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen 7B Chat V1.5: Overall 44.1, 23.2 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Dolphin Phi 2.7B V2.6: Overall 44.0, 38.4 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen3 0.6B: Overall 44.0, 261.1 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots)Qwen3 0.6B Llama-2-7b-chat-hf: Overall 43.8, 26.7 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) Stablelm Zephyr 3B: Overall 43.7, 39.6 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3.1 Moe 1B Instruct: Overall 43.6, 149.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5 0.5B Instruct: Overall 43.3, 116.8 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Stablelm2 12B Chat: Overall 42.4, 6.7 tok/s with 4 parallel streams, 9–16B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Gemma 2B Instruct V1.1: Overall 41.9, 43.4 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite Code 8B Instruct: Overall 40.7, 15.0 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Phi 2.7B Chat V2: Overall 39.9, 38.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite Code 3B Instruct: Overall 38.0, 107.3 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2 0.5B Instruct: Overall 36.0, 137.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Granite3 Moe 1B Instruct: Overall 35.5, 82.4 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Deepseek Coder 6.7B Instruct: Overall 34.7, 22.5 tok/s with 4 parallel streams, 4–9B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) Deepseek Coder 33B Instruct: Overall 33.4, 5.9 tok/s with 4 parallel streams, 16–35B, ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) Qwen2 Math 1.5B Instruct: Overall 33.1, 62.0 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen2.5 Coder 0.5B Instruct: Overall 32.8, 172.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Opencoder 1.5B Instruct: Overall 31.9, 16.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen 4B Chat V1.5: Overall 30.2, 72.2 tok/s with 4 parallel streams, 3–4B, ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Smollm2 360M Instruct: Overall 28.1, 91.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) YI Coder 1.5B Chat: Overall 26.5, 88.7 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen 1.8B Chat V1.5: Overall 24.7, 82.5 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Stablelm2 1.6B Chat: Overall 24.4, 62.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Sailor2 1B Chat: Overall 23.5, 116.3 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Stable Code 3B Instruct: Overall 22.8, 38.6 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Deepseek Coder 1.3B Instruct: Overall 22.6, 64.1 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Smollm2 135M Instruct: Overall 20.1, 201.2 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Qwen 0.5B Chat V1.5: Overall 16.1, 98.9 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) Codegemma 2B Code V1.1: Overall 15.3, 43.3 tok/s with 4 parallel streams, Tiny (under 3B), ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots)
  • Tiny (under 3B)
  • 3–4B
  • 4–9B
  • 9–16B
  • 16–35B
  • > 35B
  • size unknown
Up and to the right is better: higher quality and faster. Size class is shown by dot size and colour and the model name is printed beside the dots that no other model beats on both axes (faster and higher Overall); hover or focus any dot for its name, and every dot links to its model page. Only ranked models with a measured 4-stream speed are plotted; not plotted: Apertus-8B-Instruct-2509 (unranked), Falcon2 11B (unranked), Falcon3 10B Instruct (unranked), Falcon3 1B Instruct (unranked), Falcon3 3B Instruct (unranked), Falcon3 7B Instruct (unranked), Internlm2 1.8B Chat V2.5 (unranked), Qwen2 Math 1.5B Instruct (unranked), Qwen2 Math 7B Instruct (unranked).

Per-model speed and memory

Measured on one laptop. Decode speed is generation speed; prefill is prompt-processing speed at about 4,000 and 12,000 prompt tokens; time to first token excludes load time. Click a column heading to sort.
Model Backend Answers generated Params (B) Overall Decode (tok/s) Decode x4 (tok/s) Prefill 4k (tok/s) Prefill 12k (tok/s) TTFT (s) Load (s) GPU memory on our server (GB) GPU share
Qwen3 0.6B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 0.6 44.0 223.0 261.1 8611 5013 0.02 66.1 15.6 100%
Exaone3.5 2.4B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2.4 76.3 100.9 256.0 2890 2325 0.04 74.3 11.9 100%
Granite3 Moe 3B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 47.7 147.4 218.4 3846 4340 0.04 11.4 4.3 100%
Smollm2 135M Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 0.1 20.1 179.9 201.2 4628 4924 0.03 16.2 1.7 100%
Granite4 3B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 74.1 71.3 177.1 1586 1229 0.06 48.9 13.2 100%
Qwen2.5 Coder 0.5B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 0.5 32.8 96.7 172.5 3052 2336 0.05 13.6 2.2 100%
Nemotron Mini 4B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 4 52.0 66.8 167.6 2234 2567 0.06 21.1 6.6 100%
Cogito 3B V1 Preview Llamafine-tune of Llama 3.2 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 70.4 73.4 157.9 2153 1601 0.05 68.8 17.4 100%
Ministral 3 3B Instruct 2512 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 80.9 70.9 156.8 2516 1632 0.06 75.1 16.6 100%
Granite3.1 Moe 1B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 43.6 102.6 149.5 2095 1624 0.06 44.8 7.5 100%
Phi4 Mini Reasoning 3.8B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3.8 70.9 48.8 139.1 1946 1411 0.07 78.0 20.7 100%
Qwen2 0.5B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 0.5 36.0 93.7 137.5 3110 2336 0.05 14.1 2.2 100%
Granite4 350M H ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 0.4 45.6 55.8 128.2 1739 1756 0.14 8.4 1.2 100%
Lfm2.5 Thinking 1.2B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.2 74.3 205.0 122.0 6055 6498 0.03 3.0 1.1 100%
YI 6B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 6 58.0 43.3 120.2 1256 1353 0.11 16.9 5.9 100%
Qwen2.5 0.5B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 0.5 43.3 99.3 116.8 3069 2327 0.05 13.6 2.2 100%
Sailor2 1B Chatfine-tune of Qwen 2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 23.5 57.1 116.3 1538 1149 0.09 23.3 4.1 100%
Llama3.2 1B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 58.0 58.5 114.5 1602 1381 0.07 34.6 5.4 100%
Falcon3 1B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 44.3 51.4 107.4 1350 1381 0.08 35.7 1.1 100%
Granite Code 3B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 38.0 71.8 107.3 1511 1715 0.07 25.4 7.7 100%
Qwen 0.5B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 0.5 16.1 99.8 98.9 1951 1104 0.05 105.1 13.4 100%
Granite3.2 8B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 74.1 34.7 97.7 836 714 0.13 100.4 26.9 100%
Granite3.1 Dense 8B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 72.9 34.8 97.2 818 685 0.13 105.1 27.0 100%
Phi3 3.8B Mini 128K Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3.8 62.9 63.9 96.1 1123 687 0.08 243.8 54.1 100%
Granite3.1 Moe 3B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 59.8 53.1 93.0 1079 838 0.13 61.1 10.9 100%
Smollm2 360M Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 0.4 28.1 275.3 91.2 9527 2732 0.02 13.6 3.1 100%
YI Coder 1.5B Chatcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.5 26.5 49.5 88.7 1051 682 0.10 183.9 26.9 100%
Falcon3 1B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 —unranked 59.0 87.3 1301 1327 0.08 36.3 6.0 100%
Internlm2 1.8B Chat V2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.8 51.9 45.6 85.6 1086 889 0.10 82.2 1.2 100%
Qwen 1.8B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.8 24.7 127.4 82.5 3774 2311 0.03 115.9 27.1 100%
Granite3 Moe 1B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1 35.5 70.0 82.4 1882 2180 0.08 16.7 2.3 90%
Gemma3 1B IT ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 56.8 57.0 80.1 1929 1987 0.10 11.3 1.5 100%
Qwen2.5 Coder 1.5B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.5 60.5 45.8 77.0 1221 1016 0.10 26.8 5.1 100%
Internlm2 1.8B Chat V2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.8 —unranked 43.0 73.2 1049 822 0.10 82.0 14.3 100%
Qwen 4B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 4 30.2 67.5 72.2 1750 1243 0.07 249.2 56.1 100%
Deepcoder 1.5B Previewfine-tune of DeepSeek-R1-Distill-Qwen 1.5Bcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.5 55.8 44.9 66.9 1295 1007 0.10 26.8 5.1 100%
Exaone Deep 2.4B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2.4 54.7 31.2 65.1 777 615 0.14 76.5 11.9 100%
Deepseek Coder 1.3B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.3 22.6 53.6 64.1 1057 688 0.10 182.5 26.9 100%
Stablelm2 1.6B Chat ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.6 24.4 47.9 62.2 794 982 0.10 44.7 7.5 100%
Qwen2 Math 1.5B Instructmath model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.5 33.1 45.2 62.0 1217 1324 0.10 11.6 1.0 100%
Qwen2 Math 1.5B Instructmath model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.5 —unranked 44.1 61.1 1220 1311 0.10 9.2 2.1 100%
Falcon3 3B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 64.8 27.9 60.3 674 518 0.15 82.8 2.0 100%
Qwen2.5-14B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 14.8 91.3 21.7 59.0 555 416 0.18 141.0 34.8 100%
Lfm3 VL 2B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 2 72.2 41.9 58.7 1285 1259 0.15 17.3 2.0 100%
Granite3.3 2B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2 68.6 28.6 58.5 645 546 0.19 78.0 12.6 100%
Lfm3 1.7B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 1.7 70.2 42.1 58.2 1276 1268 0.15 19.0 1.6 100%
Falcon3 3B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 —unranked 26.0 57.9 685 522 0.15 93.8 14.0 100%
Granite3.1 Dense 2B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2 60.6 28.8 56.8 635 534 0.18 70.0 12.6 100%
Smollm2 1.7B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1.7 47.1 41.7 53.3 671 679 0.11 95.1 14.2 100%
Smallthinker 3B Previewfine-tune of Qwen 2.5 3B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 61.1 22.1 53.2 672 587 0.21 46.7 7.2 100%
Codegemma 7B Instruct V1.1code model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 59.5 10.3 52.6 199 356 0.50 256.7 36.1 100%
Qwen2.5-VL-3B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.8 72.3 21.9 51.4 659 574 0.21 24.6 5.5 100%
SmolLM3-3B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 3.1 78.0 22.2 50.8 684 595 0.20 75.8 12.0 100%
Hermes3 3B Llama3.2fine-tune of Llama 3.2 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 48.5 23.4 50.7 502 376 0.19 106.9 17.4 100%
Deepseek R1 1.5B Qwen Distill ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.5 48.9 44.9 50.6 1287 1006 0.10 27.0 5.1 100%
Granite4.2 3B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 3 78.2 24.3 50.1 472 347 0.20 84.6 13.2 100%
Granite4 1B H ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 1 66.8 24.9 50.1 616 631 0.35 15.8 3.4 100%
Qwen2 1.5B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.5 49.7 43.1 49.8 1284 1000 0.12 — — —
Llama-3.2-3B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.2 75.7 20.7 49.8 497 380 0.20 67.8 9.9 100%
Qwen2.5-Coder-3B-Instructcode model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.1 72.5 21.3 49.3 665 579 0.21 22.8 4.7 100%
granite-4.1-3b ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.4 80.7 20.1 49.0 472 346 0.25 48.2 7.8 100%
Qwen2.5-3B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 to 8 at a time 3.1 76.8 21.9 44.9 681 591 0.20 23.4 4.7 100%
Phi-3-mini-4k-instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.8 63.4 17.6 44.5 329 371 0.27 57.8 8.9 100%
Qwen2.5 1.5B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.5 68.5 45.5 43.8 1222 1003 0.10 27.0 5.1 100%
Phi-4-mini-instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.8 81.6 17.1 43.7 514 373 0.23 73.3 11.4 100%
Granite3 Dense 2B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2 58.3 29.7 43.5 626 652 0.18 20.1 4.4 100%
Qwen3 1.7B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.7 76.0 38.5 43.5 1057 779 0.13 95.5 16.5 100%
Gemma 2B Instruct V1.1 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2 41.9 29.3 43.4 861 859 0.16 13.5 3.3 100%
Codegemma 2B Code V1.1code model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2 15.3 29.1 43.3 858 856 0.14 10.8 3.3 100%
Qwen3-4B-Instruct-2507 ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 4 90.3 17.1 43.2 412 308 0.26 85.7 12.5 100%
medgemma-4b-itmedical model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 4.3 74.3 17.1 42.2 490 491 0.31 26.6 4.5 100%
Lfm2.5 8B A1B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 8 75.7 43.4 41.6 942 910 0.23 9.5 5.5 100%
Phi-3.5-mini-instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 3.8 67.1 17.6 41.6 276 156 0.28 219.2 28.4 100%
Qwen3-4B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 4 89.2 17.0 41.3 484 343 0.27 66.5 12.7 100%
Granite4 Tiny H ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 79.1 24.2 40.9 442 476 0.42 19.4 6.1 100%
Deepscaler 1.5B Previewfine-tune of DeepSeek-R1-Distill-Qwen 1.5Bmath model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.5 52.1 45.5 40.0 1287 1006 0.10 29.1 5.1 100%
Laguna XS 2.1 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time — 91.2 23.1 40.0 323 289 0.64 68.4 23.2 100%
Gemma2 2B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2 68.2 27.6 39.9 719 716 0.19 45.9 4.7 100%
Stablelm Zephyr 3Bfine-tune of StableLM 3B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2.8 43.7 26.5 39.6 407 524 0.21 72.5 12.5 100%
Spark-X2.6-4B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 4 85.0 19.7 38.7 469 438 0.38 58.2 5.3 100%
gemma-3-4b-it ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 4.3 82.1 16.9 38.6 493 490 0.31 27.1 4.5 100%
Stable Code 3B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2.8 22.8 26.4 38.6 404 224 0.21 316.1 44.9 100%
Dolphin Phi 2.7B V2.6fine-tune of Phi-2 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2.7 44.0 25.0 38.4 503 515 0.17 38.8 7.2 100%
Phi 2.7B Chat V2 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 2.7 39.9 26.0 38.2 508 529 0.17 37.3 7.2 100%
Apertus-8B-Instruct-2509 Vulkan (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8.1 63.6 13.5 32.9 188 115 0.38 10.6 22.3 100%
Qwen3.5 2B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 2 75.1 33.3 32.2 961 939 0.20 8.3 1.9 100%
Granite3.2 2B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 2 63.8 30.1 32.1 635 538 0.18 71.5 12.6 100%
Qwen3-4B-Thinking-2507 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 4 83.5 21.5 32.1 511 352 0.24 126.5 2.5 100%
Flint 40B A2B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 40 89.0 25.9 31.9 342 323 0.66 60.3 24.3 100%
gpt-oss-20b ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 20.9 95.0 14.9 31.5 437 429 0.62 40.6 13.5 100%
Granite4 Micro H ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 3 76.6 13.4 30.9 364 370 0.60 23.9 4.0 100%
Medgemma1.5 4B ITfine-tune of Gemma 3medical model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 4 71.6 20.2 30.6 508 508 0.28 39.4 5.9 100%
Openthinker 7B V2fine-tune of Qwen 2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 76.8 12.6 30.1 308 280 0.36 59.6 12.5 100%
Gemma4 E2B IT ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 5 93.8 17.2 29.8 673 593 0.39 18.3 0.6 100%
Lfm2 24B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 1 at a time 24 87.3 32.5 29.5 583 591 0.51 16.1 15.0 100%
Mistral-7B-Instruct-v0.2 ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 7.2 59.2 10.2 29.0 274 225 0.43 131.9 22.0 100%
Exaone Deep 7.8B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7.8 65.7 12.2 28.9 276 230 0.37 131.4 22.3 100%
Falcon3 7B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 76.2 12.5 28.4 290 250 0.37 103.5 4.6 100%
Qwen2.5-VL-7B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 8.3 82.0 10.2 28.4 306 278 0.40 64.0 13.2 100%
Qwen2.5-7B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 7.6 85.6 10.1 28.3 306 279 0.39 34.1 8.7 100%
Internlm2 7B Chat V2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 67.4 12.0 28.3 252 217 0.41 135.3 22.2 100%
Tulu3 8Bfine-tune of Llama 3.1 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 73.0 12.0 28.2 274 231 0.37 126.5 22.4 100%
Qwen2.5-Coder-7B-Instructcode model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 7.6 84.5 9.8 28.2 308 279 0.40 62.7 12.5 100%
Meta-Llama-3-8B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 8 74.7 9.6 27.3 272 271 0.41 38.2 9.2 100%
Codellama 7B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 7 53.0 13.4 27.2 252 181 0.40 248.4 38.8 100%
Llama-2-7b-chat-hf ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 to 8 at a time 6.7 43.8 11.0 26.7 266 291 0.43 68.4 12.8 100%
Qwen3 VL 2B Thinking ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 2 53.7 27.2 26.4 950 806 0.14 19.0 3.6 100%
Sailor2 8B Chatfine-tune of Qwen 2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 57.3 10.6 26.3 223 204 0.42 86.3 14.2 100%
Qwen3 VL 2B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 2 77.4 26.5 26.2 933 788 0.15 16.5 3.6 100%
Qwen3-30B-A3B-Instruct-2507 ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 30 95.0 20.4 26.1 384 304 0.49 100.3 31.7 100%
Gemma3n E2B IT ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 5 79.9 13.8 26.1 484 486 0.35 19.6 3.4 100%
Qwen3-Coder-30B-A3B-Instructcode model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 30 95.5 20.1 26.1 382 303 0.49 100.5 31.7 100%
Granite3.3 8B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 79.4 11.3 25.9 254 203 0.48 154.5 26.9 100%
Qwen3-30B-A3B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 30 92.8 19.7 25.5 380 301 0.52 101.4 31.7 100%
Gemma4 E4B IT ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 95.5 13.2 25.3 416 391 0.41 30.5 1.2 100%
Codeqwen 7B Chat V1.5code model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 49.4 11.8 25.2 316 279 0.38 68.8 13.6 100%
Granite4.2 8B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 8 81.9 11.1 24.8 244 197 0.42 169.2 27.1 100%
Deepseek Llm 7B Chat ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 54.3 12.2 24.8 278 299 0.37 118.1 20.3 100%
Falcon3 7B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 —unranked 11.6 24.7 279 240 0.38 112.2 20.1 100%
Cogito 8B V1 Preview Llamafine-tune of Llama 3.1 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 83.9 11.2 24.6 275 226 0.37 128.9 22.4 100%
YI Coder 9B Chatcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 9 59.0 10.5 24.6 219 189 0.54 98.7 18.5 100%
YI 9B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 9 70.5 10.0 24.6 229 242 0.55 33.0 8.6 100%
Olmo-3-7B-Think ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 7.3 77.7 13.5 24.0 257 229 0.61 209.9 4.5 100%
Marco O1 7Bfine-tune of Qwen 2 7B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 70.5 11.8 23.7 316 276 0.37 58.4 12.5 100%
Llama-3.1-8B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 8 81.0 9.6 23.6 278 229 0.41 70.1 13.7 100%
Qwen 7B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 7 44.1 12.3 23.2 249 184 0.42 256.8 39.2 100%
EXAONE-3.5-7.8B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 7.8 82.0 9.5 23.0 278 229 0.40 70.1 13.6 100%
Olmo-3-7B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 to 8 at a time 7.3 82.9 10.2 22.8 235 195 0.65 141.4 14.4 100%
Deepseek Coder 6.7B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 6.7 34.7 13.6 22.5 277 197 0.41 262.0 38.8 100%
Ministral 3 8B Instruct 2512 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 88.7 11.0 22.2 271 201 0.41 145.3 23.8 100%
gemma-4-26B-A4B-it ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 26 97.8 15.7 21.8 366 330 0.61 70.1 22.6 100%
Hermes-3-Llama-3.1-8B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 to 8 at a time 8 67.6 9.6 21.3 276 229 0.45 73.0 13.7 100%
Sage-A1.5-Q4_K_M ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 1 at a time 36 94.5 22.6 21.2 395 375 0.57 25.2 21.8 100%
Falcon3 10B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 10 —unranked 9.2 21.2 177 154 0.52 142.2 28.3 100%
Falcon3 10B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 10 81.3 9.0 20.9 200 174 0.53 146.1 6.3 100%
DeepSeek-R1-Distill-Qwen-7B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 7.6 73.6 13.8 20.7 342 301 0.31 54.2 4.7 100%
Gemma 3n E4B IT ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 8 86.1 12.1 20.0 282 277 0.54 22.4 4.2 100%
Qwen3-VL-30B-A3B-Instruct Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 30 92.9 26.9 19.8 298 167 0.74 12.6 20.6 100%
Falcon2 11B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 11 —unranked 8.1 19.7 184 187 0.57 31.8 10.9 100%
Qwen2 Math 7B Instructmath model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 66.0 12.2 19.4 307 320 0.36 18.3 4.7 100%
GLM-4.7-Flash ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 31.2 88.9 14.1 19.3 187 104 0.58 63.9 26.0 100%
Nemotron3 33B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 1 at a time 33 86.2 20.0 18.9 372 367 0.83 24.0 25.0 100%
DeepSeek-R1-0528-Qwen3-8B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 8.2 79.6 12.2 18.4 289 230 0.39 135.4 5.2 100%
Deepseek R1 8B Llama Distill ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 81.7 11.9 18.2 277 231 0.37 136.3 22.4 100%
Qwen3.6-35B-A3B Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 35 94.6 22.1 17.8 276 284 1.63 15.2 23.0 100%
Mistral 7B Instruct V0.3 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 59.8 11.6 17.5 272 223 0.44 125.9 22.0 100%
Qwen2 Math 7B Instructmath model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 —unranked 11.2 17.4 309 313 0.37 21.9 6.5 100%
Nemotron 3.5 Lightning 30B A3B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 1 at a time 30 92.8 17.7 17.3 352 352 0.85 22.6 25.4 100%
Nemotron Cascade 2 30B A3B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 1 at a time 30 85.6 20.5 17.1 372 368 0.82 22.4 24.4 100%
NVIDIA Nemotron 3 Nano 30B-A3B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 1 at a time 30 83.9 18.4 17.0 349 346 0.80 22.2 24.4 100%
Phi3 14B Medium 128K Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 14 78.0 — 17.0 — 126 — — — —
Olmo2 7B 1124 Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 65.4 11.8 16.9 261 289 0.37 112.3 21.6 100%
Mathstral 7B V0.1fine-tune of Mistral 7Bmath model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 68.0 12.0 16.8 272 224 0.42 126.8 22.0 100%
Qwen3-VL-4B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 4.4 89.9 17.3 16.8 455 364 0.25 23.8 5.5 100%
Qwen3.5-35B-A3B Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 35 94.8 23.2 16.5 235 227 1.41 15.3 21.7 100%
Aya 8B 23 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 55.9 10.9 16.4 243 246 0.40 84.5 14.2 100%
Llama2 13B Chat ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 13 45.3 7.1 16.4 148 160 0.69 109.3 21.5 100%
Aya Expanse 8B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 72.5 10.7 16.3 244 243 0.40 82.2 14.2 100%
Codegeex4 9B Allcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 9 68.7 10.1 16.3 262 229 0.41 40.3 11.8 100%
Opencoder 1.5B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 1.5 31.9 17.4 16.2 361 496 0.22 47.4 8.4 100%
Codellama 13B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 13 52.5 6.9 16.2 122 98 0.70 444.4 61.7 100%
Command R7B Arabic 7B 02 2025 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 81.9 11.0 16.2 248 229 0.51 92.4 10.5 100%
Qwen2-VL-7B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 8.3 72.2 10.2 16.1 331 294 0.43 55.4 12.8 100%
Granite4.1 8B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 92.1 10.4 15.9 225 184 0.50 189.7 27.1 100%
Glm4 9B Chat ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 9 73.3 9.8 15.9 261 228 0.41 40.6 11.8 100%
Qwen2.5-Coder-14B-Instructcode model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 14.8 87.7 5.2 15.9 112 86 0.74 203.4 34.8 100%
gpt-oss-120b Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 4 at a time 116.8 98.6 11.0 15.8 158 140 2.73 104.8 64.6 100%
Olmo2 13B 1124 Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 13 69.6 6.8 15.5 149 161 0.63 109.5 21.7 100%
Gemma 7B Instruct V1.1 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 7 60.8 10.2 15.2 198 200 0.50 226.1 36.1 100%
Granite Code 8B Instructcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 40.7 10.5 15.0 222 227 0.47 41.4 10.0 100%
Ministral 3 14B Instruct (2512) ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 14 92.7 5.6 15.0 178 135 0.73 173.0 30.0 100%
Qwen3-Coder-Nextcode model Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 79.7 96.4 18.1 14.7 177 180 1.93 75.7 52.2 100%
Qwen3.5-4B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 4.7 88.7 15.1 14.6 360 348 0.42 11.1 3.6 100%
Granite3 Dense 8B Instruct ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 8 67.6 10.5 14.5 255 257 0.49 46.6 10.5 100%
Qwen3-8B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 8.2 91.5 9.3 14.4 249 203 0.50 148.6 24.7 100%
Command R7B (12-2024) ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 8 74.8 8.7 14.0 265 266 0.56 60.2 10.4 100%
Qwen 14B Chat V1.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 4 parallel slots) 4 at a time 14 54.8 6.3 13.9 136 105 0.73 462.5 62.6 100%
Solar 10.7B Instruct V1 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 10.7 58.7 8.7 13.6 183 191 0.61 47.3 13.1 100%
Aya Expanse 32B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 32 78.8 10.0 13.5 281 88 0.42 89.2 31.4 100%
Gemma 4 12B IT ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 12 98.6 8.1 13.1 157 148 0.94 89.0 8.9 100%
Nemotron 3.5 Bolt E4B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 8 90.3 9.8 13.0 384 365 0.39 41.7 5.7 100%
Mistral Nemo 12B Instruct 2407 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 12 75.9 8.2 12.5 177 153 0.54 143.5 29.2 100%
Granite4 Small H ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 32 93.9 8.4 12.4 133 135 1.44 37.9 23.9 100%
gemma-4-31B-it ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 4 at a time 31.3 99.3 3.8 11.8 64 58 2.37 122.7 11.2 100%
gemma-2-9b-it ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 9.2 79.9 7.6 11.7 172 171 0.69 154.4 14.7 100%
Translategemma 12B ITfine-tune of Gemma 3translation model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 12 89.6 7.6 11.5 177 176 0.77 97.3 13.1 100%
Mistral Small 24B Instruct 2501 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 24 92.1 4.2 10.6 98 88 1.03 204.3 36.5 100%
Ornith 9B ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 9 90.0 10.8 10.5 262 256 0.80 12.3 5.9 100%
Magistral Small (2506) ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 24 49.8 4.5 10.5 107 95 1.04 153.8 14.3 100%
Deepcoder 14B Previewfine-tune of DeepSeek-R1-Distill-Qwen 14Bcode model ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 14 74.5 6.7 10.3 146 108 0.67 199.0 34.9 100%
Mistral Small3.1 24B Instruct 2503 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 24 90.1 4.3 10.3 79 75 1.34 154.5 36.6 100%
Cogito 14B V1 Preview Qwenfine-tune of Qwen 2.5 ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 14 90.3 6.6 10.2 140 106 0.75 200.3 34.9 100%
DeepSeek-R1-Distill-Qwen-14B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 14.8 85.3 7.1 10.0 147 104 0.62 173.4 9.0 100%
gemma-3-12b-it ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 12.2 92.7 6.1 9.5 139 138 0.84 115.4 13.1 100%
Mistral Small 3.2 24B Instruct (2506) ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 24 92.9 3.1 9.4 99 80 1.72 176.9 36.5 100%
Qwen3-VL-8B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 8.8 94.9 9.4 9.2 261 229 0.43 25.9 7.8 100%
phi-4 ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 14.7 85.9 5.0 8.9 135 122 0.75 210.6 36.2 100%
Devstral Small 2 24B Instruct (2512)code model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 24 92.1 3.1 8.9 80 65 1.54 177.6 36.5 100%
Muse Glimmer 30B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 30 97.9 3.6 8.8 96 96 2.12 42.3 18.3 100%
Apertus-8B-Instruct-2509 ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 1 at a time 8.1 —unranked 9.4 8.7 294 243 0.42 115.3 22.3 100%
Qwen3-14B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 14.8 91.7 5.0 8.7 153 113 0.81 171.5 30.9 100%
Mixtral 8x7B Instruct V0.1 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 1 at a time 56 56.8 7.6 8.5 137 127 1.32 140.0 46.5 100%
Qwen3.5-9B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 9.7 92.0 8.4 8.2 225 222 0.69 13.5 6.1 100%
Llama 4 Scout 17B-16E Instruct Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 109 93.6 17.0 8.2 147 163 2.53 81.4 70.4 100%
Granite4.2 30B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 30 92.8 3.4 8.2 62 60 1.37 243.3 52.3 100%
Devstral 24B Small 2505code model ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 24 88.9 4.2 8.0 122 86 1.54 165.4 36.5 100%
Ornith-1.5-9B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 1 at a time 9.7 89.0 8.1 8.0 220 215 0.71 11.7 6.4 100%
Command R 35B 08 2024 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 35 75.2 3.2 7.7 61 55 1.34 159.0 42.2 100%
Exaone3.5 32B Instruct ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 32 85.0 3.1 7.7 66 57 1.31 256.6 53.9 100%
Qwen2.5-Coder-32B-Instructcode model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 32.8 90.5 2.3 7.6 52 46 1.61 244.5 54.2 100%
Qwen2.5-32B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 32.8 93.5 2.3 7.6 53 46 1.61 251.7 54.2 100%
GLM-4.5-Air Vulkan (Ollama 0.40.4, flash attention, 16k context; 2 parallel slots) 2 at a time 110.5 92.8 6.0 7.6 53 29 3.15 110.5 76.8 100%
Nanbeige_Nanbeige4.3-33B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 33 90.4 3.1 7.4 71 58 1.54 200.9 48.9 100%
Codellama 34B Instructcode model ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 34 61.2 3.1 7.4 58 51 1.62 190.2 46.7 100%
Olmo 3.1 32B Instruct ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 32 89.6 3.1 7.4 56 54 2.61 128.4 27.7 100%
dolphin-2.9.1-yi-1.5-34b ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 34.4 73.0 2.4 7.2 51 51 1.94 123.1 37.2 100%
YI 34B Chat V1.5 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 34 75.2 3.0 7.1 57 59 1.62 71.7 28.8 100%
Stablelm2 12B Chat ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 8 at a time 12 42.4 4.7 6.7 84 103 0.85 50.2 14.0 100%
Codestral 22B V0.1code model ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 22 74.8 4.4 6.5 76 64 1.17 242.0 44.0 100%
Mistral Small 22B Instruct 2409 ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 22 84.6 4.2 6.1 76 65 1.19 244.6 44.0 100%
Deepseek V2 16B Lite Chat ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) 1 at a time 16 58.2 12.2 5.9 244 109 0.49 243.4 46.8 100%
Deepseek Coder 33B Instructcode model ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 33 33.4 3.0 5.9 64 54 1.62 239.9 53.5 100%
Gemma2 27B Instruct ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 27 85.6 3.7 5.5 67 68 1.52 157.6 27.0 100%
Medgemma 27B ITfine-tune of Gemma 3medical model ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 27 90.2 3.5 5.3 88 89 1.64 126.6 22.1 100%
Granite4.1 30B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 30 92.3 3.4 5.0 62 59 1.49 242.1 52.3 100%
DeepSeek-R1-Distill-Qwen-32B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 32.8 89.1 3.1 4.8 56 49 1.36 281.5 19.9 100%
QwQ 32B ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) 8 at a time 32.8 92.6 3.2 4.7 72 60 1.46 271.7 54.2 100%
gemma-3-27b-it ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 27.4 95.8 2.7 4.7 71 72 1.91 124.9 22.1 100%
DeepSeek-Coder-V2-Lite-Instructcode model ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 15.7 79.0 11.1 4.6 160 110 0.50 283.9 46.8 100%
Qwen3-32B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 32.8 93.6 2.3 4.1 54 46 1.76 246.2 55.0 100%
Qwen3.6-27B Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 27.8 95.1 — 3.5 55 55 — 11.8 17.3 100%
Llama-3.3-70B-Instruct Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 4 at a time 70.6 94.3 1.4 3.4 16 11 4.30 51.1 63.8 100%
Qwen3.8-27B Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 27.8 95.4 3.5 3.2 52 51 4.13 12.2 17.6 100%
Qwen3-VL-32B-Instruct Vulkan (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots) 1 at a time 33.4 94.7 3.2 2.9 33 22 1.49 15.6 24.5 100%
Qwen2.5-VL-32B-Instruct ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 1 at a time 33.5 89.7 2.3 2.7 52 — 1.61 248.4 54.9 100%
Qwen3.5-27B ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) 8 at a time 27.8 96.2 2.7 2.6 71 69 2.06 27.7 17.6 100%

A dash means the metric was not measured, which is different from 0. GPU memory on our server (1 to 8 answers at a time) is what Ollama reported for the loaded model with this server's settings (parallel slots and context length), so it is not what one user needs, in either direction; Answers generated is how many benchmark answers the runner kept in flight for the model, and a range such as “4 to 8 at a time” means the setting changed during its run (see the methodology). Load time is the wall time to load the model after a fresh unload; the operating system's file cache can make it faster than a cold disk read. Decode speed uses up to 256 generated tokens; a model that stops earlier is still measured. Raw numbers: speed.json. To see which models fit the memory you have, use the hardware picker. The method is described in the methodology.