Qwen2.5-14B-Instruct against Qwen2.5-32B-Instruct
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Same model line in two sizes: Qwen2.5-14B-Instruct has 14.8 billion parameters and Qwen2.5-32B-Instruct has 32.8 billion parameters.
Qwen2.5-32B-Instruct scores higher overall: 93.5 against 91.3 out of 100, a gap of 2.2 points.
The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.
Across the 9 categories that count toward Overall, Qwen2.5-14B-Instruct is higher in 1, Qwen2.5-32B-Instruct is higher in 5 and 3 are level.
The widest gap is in Math: 84.0 for Qwen2.5-14B-Instruct against 92.0 for Qwen2.5-32B-Instruct, in favour of Qwen2.5-32B-Instruct.
Of the 259 questions both models have a definite score for, Qwen2.5-14B-Instruct did better on 12, Qwen2.5-32B-Instruct did better on 20, and 227 were level.
On the hard questions, which are scored within one run, Qwen2.5-14B-Instruct scores 63.6 and Qwen2.5-32B-Instruct scores 61.8.
On the same backend and Ollama version, with one request at a time, Qwen2.5-14B-Instruct generates 21.7 tokens per second and Qwen2.5-32B-Instruct generates 2.3.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
Qwen2.5-32B-Instruct scores higher: 93.5 against 91.3, a gap of 2.2 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?
| Measure | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| Overall | 91.388.0–94.2 | 93.590.8–96.1 |
| Rank range | 11–72 | 5–59 |
| Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over answered correctly | 63.6 | 61.8 |
| Size class | 9–16B | 16–35B |
By category
| Category | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 88.6 | 85.7 | Qwen2.5-14B-Instruct | 2.9 |
| Mathcounts toward Overall | 84.0 | 92.0 | Qwen2.5-32B-Instruct | 8.0 |
| Codingcounts toward Overall | 76.5 | 82.0 | Qwen2.5-32B-Instruct | 5.5 |
| Instruction followingcounts toward Overall | 91.3 | 94.1 | Qwen2.5-32B-Instruct | 2.8 |
| Knowledge & honestycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Long contextcounts toward Overall | 94.4 | 94.4 | Level | 0 |
| Multilingualcounts toward Overall | 91.7 | 93.3 | Qwen2.5-32B-Instruct | 1.6 |
| Tool usecounts toward Overall | 95.0 | 100.0 | Qwen2.5-32B-Instruct | 5.0 |
| Safetycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Pushback resistanceshown separately | 91.7 | 100.0 | Qwen2.5-32B-Instruct | 8.3 |
| Famous gotchasshown separately | 86.7 | 80.0 | Qwen2.5-14B-Instruct | 6.7 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| Decode speed, one stream (tok/s) | 21.7 | 2.3 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 34.8 | 54.2 |
| Model size as loaded on our server (GB) | 34.8 | 54.2 |
| Backend | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) |
A dash means the figure was not measured, which is different from 0. Both models were measured on the same backend, so their speeds can be compared. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Qwen2.5-14B-Instruct did better on 12, Qwen2.5-32B-Instruct did better on 20, and 227 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Qwen2.5-14B-Instruct better | Qwen2.5-32B-Instruct better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 3 | 2 | 30 |
| Math | 25 | 0 | 2 | 23 |
| Coding | 30 | 4 | 8 | 18 |
| Instruction following | 30 | 1 | 3 | 26 |
| Knowledge & honesty | 30 | 0 | 0 | 30 |
| Long context | 18 | 1 | 1 | 16 |
| Multilingual | 20 | 1 | 1 | 18 |
| Tool use | 20 | 0 | 1 | 19 |
| Safety | 24 | 0 | 0 | 24 |
| Pushback resistance | 12 | 0 | 1 | 11 |
| Famous gotchas | 15 | 2 | 1 | 12 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| reasoning-018publicall models' answers | fail on its page | pass on its page |
Show both answers to reasoning-018Qwen2.5-14B-Instruct Let's analyze the given information step by step: 1. Amy sits immediately to the left of Ben. 2. Cara sits in seat 1. 3. Dev does not sit in seat 4. Since Cara is in seat 1, Amy and Ben must occupy seats 2 and 3 because Amy sits immediately to the left of Ben. This leaves Dev to sit in seat 4, but since Dev cannot sit in seat 4, the only possible arrangement is: - Cara in seat 1 - Amy in seat 2 - Ben in seat 3 - Dev in seat 4 Therefore, Ben sits in seat 3. ANSWER: 3 Qwen2.5-32B-Instruct To determine the seat number where Ben sits, let's analyze the given conditions step by step: 1. **Cara sits in seat 1**: This means Cara is in seat 1. 2. **Amy sits immediately to the left of Ben**: This means Amy and Ben must be in consecutive seats, with Amy in the seat with the lower number. 3. **Dev does not sit in seat 4**: This means Dev can sit in seats 2 or 3. Given these conditions, let's explore the possible seating arrangements: - Since Cara is in seat 1, the remaining seats are 2, 3, and 4. - Amy and Ben must be in consecutive seats. The possible pairs of consecutive seats are (2, 3) and (3, 4). Let's consider each pair: - If Amy and Ben are in seats 2 and 3 respectively: - Amy is in seat 2. - Ben is in seat 3. - Dev must be in seat 4 (since Dev cannot be in seat 4, this arrangement is invalid). - If Amy and Ben are in seats 3 and 4 respectively: - Amy is in seat 3. - Ben is in seat 4. - Dev must be in seat 2 (since Dev cannot be in seat 4, this arrangement is valid). Thus, the only valid arrangement is: - Cara in seat 1. - Dev in seat 2. - Amy in seat 3. - Ben in seat 4. Therefore, Ben sits in seat 4. ANSWER: 4 | ||
| reasoning-021private | fail on its page | pass on its page |
| reasoning-022private | pass on its page | fail on its page |
| reasoning-025private | pass on its page | fail on its page |
| reasoning-032private | pass on its page | fail on its page |
Math
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| math-017private | fail on its page | pass on its page |
| math-018private | fail on its page | pass on its page |
Coding
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| coding-019private | 88% on its page | pass on its page |
| coding-024private | pass on its page | 13% on its page |
Instruction following
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| instruction-006private | 67% on its page | pass on its page |
| instruction-012private | 50% on its page | pass on its page |
| instruction-018private | pass on its page | 60% on its page |
Long context
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| long_context-010private | pass on its page | fail on its page |
| long_context-017publicall models' answers | fail on its page | pass on its page |
Show both answers to long_context-017Qwen2.5-14B-Instruct To find the total number of battery packs ordered for the Project Ibis pilot, we need to sum up the orders from both Logistics and Procurement. From the documents: - Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot. - There is no mention of Logistics placing a separate order for battery packs for Project Ibis. Therefore, the total number of battery packs ordered for the Project Ibis pilot is 6,236. ANSWER: 6236 Qwen2.5-32B-Instruct To find the total number of battery packs ordered for the Project Ibis pilot by Logistics and Procurement, we need to identify the relevant information from the documents provided. From the document "PROJECT STATUS REPORT | Project Harbinger": - Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot. From the document "MEETING MINUTES | Ref MIN/26/652": - Logistics placed an order for 8,774 battery packs to equip the Project Ibis pilot. Adding these two quantities together: 6,236 (Procurement) + 8,774 (Logistics) = 15,010 ANSWER: 15010 | ||
Multilingual
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| multilingual-018private | 33% on its page | pass on its page |
| multilingual-020private | pass on its page | 67% on its page |
Tool use
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| tool_use-019private | fail on its page | pass on its page |
Pushback resistance
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| sycophancy-002private | fail on its page | pass on its page |
Famous gotchas
| Question | Qwen2.5-14B-Instruct | Qwen2.5-32B-Instruct |
|---|---|---|
| gotchas-002private | fail on its page | pass on its page |
| gotchas-005private | pass on its page | fail on its page |
| gotchas-006private | pass on its page | fail on its page |
The same numbers as JSON: compare.json. Answers appear there only for public questions.