Qwen3-4B against Qwen3-8B
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Same model line in two sizes: Qwen3-4B has 4 billion parameters and Qwen3-8B has 8.2 billion parameters.
Qwen3-8B scores higher overall: 91.5 against 89.2 out of 100, a gap of 2.3 points.
The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.
Across the 9 categories that count toward Overall, Qwen3-4B is higher in 1, Qwen3-8B is higher in 4 and 4 are level.
The widest gap is in Multilingual: 88.3 for Qwen3-4B against 100.0 for Qwen3-8B, in favour of Qwen3-8B.
Of the 259 questions both models have a definite score for, Qwen3-4B did better on 15, Qwen3-8B did better on 20, and 224 were level.
On the hard questions, which are scored within one run, Qwen3-4B scores 58.1 and Qwen3-8B scores 55.5.
On the same backend and Ollama version, with one request at a time, Qwen3-4B generates 17.0 tokens per second and Qwen3-8B generates 9.3.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
Qwen3-8B scores higher: 91.5 against 89.2, a gap of 2.3 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?
| Measure | Qwen3-4B | Qwen3-8B |
|---|---|---|
| Overall | 89.285.6–92.4 | 91.588.4–94.5 |
| Rank range | 18–81 | 11–72 |
| Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over answered correctly | 58.1 | 55.5 |
| Size class | 4–9B | 4–9B |
By category
| Category | Qwen3-4B | Qwen3-8B | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 88.6 | 94.3 | Qwen3-8B | 5.7 |
| Mathcounts toward Overall | 88.0 | 88.0 | Level | 0 |
| Codingcounts toward Overall | 69.0 | 70.6 | Qwen3-8B | 1.6 |
| Instruction followingcounts toward Overall | 91.0 | 87.7 | Qwen3-4B | 3.3 |
| Knowledge & honestycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Long contextcounts toward Overall | 77.8 | 83.3 | Qwen3-8B | 5.5 |
| Multilingualcounts toward Overall | 88.3 | 100.0 | Qwen3-8B | 11.7 |
| Tool usecounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Safetycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Pushback resistanceshown separately | 100.0 | 100.0 | Level | 0 |
| Famous gotchasshown separately | 100.0 | 86.7 | Qwen3-4B | 13.3 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Qwen3-4B | Qwen3-8B |
|---|---|---|
| Decode speed, one stream (tok/s) | 17.0 | 9.3 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 12.7 | 24.7 |
| Model size as loaded on our server (GB) | 12.7 | 24.7 |
| Backend | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) |
A dash means the figure was not measured, which is different from 0. Both models were measured on the same backend, so their speeds can be compared. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Qwen3-4B did better on 15, Qwen3-8B did better on 20, and 224 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Qwen3-4B better | Qwen3-8B better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 0 | 2 | 33 |
| Math | 25 | 1 | 1 | 23 |
| Coding | 30 | 4 | 9 | 17 |
| Instruction following | 30 | 6 | 2 | 22 |
| Knowledge & honesty | 30 | 0 | 0 | 30 |
| Long context | 18 | 2 | 3 | 13 |
| Multilingual | 20 | 0 | 3 | 17 |
| Tool use | 20 | 0 | 0 | 20 |
| Safety | 24 | 0 | 0 | 24 |
| Pushback resistance | 12 | 0 | 0 | 12 |
| Famous gotchas | 15 | 2 | 0 | 13 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| reasoning-033private | fail on its page | pass on its page |
| reasoning-034private | fail on its page | pass on its page |
Math
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| math-017private | fail on its page | pass on its page |
| math-022private | pass on its page | fail on its page |
Coding
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| coding-004private | 25% on its page | pass on its page |
| coding-014publicall models' answers | 43% on its page | pass on its page |
Show both answers to coding-014Qwen3-4B ```python
def reverse_words(s: str) -> str:
return ' '.join(s.split())
```
Qwen3-8B ```python
def reverse_words(s: str) -> str:
return ' '.join(reversed(s.split()))
```
| ||
| coding-021private | 89% on its page | pass on its page |
| coding-022private | 64% on its page | pass on its page |
Instruction following
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| instruction-001private | pass on its page | 50% on its page |
| instruction-009private | pass on its page | 67% on its page |
| instruction-010private | pass on its page | 75% on its page |
| instruction-012private | 75% on its page | pass on its page |
| instruction-018private | pass on its page | 80% on its page |
| instruction-029private | 50% on its page | pass on its page |
Long context
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| long_context-004publicall models' answers | fail on its page | pass on its page |
Show both answers to long_context-004Qwen3-4B The stakeholder briefing for Project Cobalt Fen is scheduled for 16 November 2027. ANSWER: 2027-11-16 Qwen3-8B The stakeholder briefing for Project Cobalt Fen is scheduled for 6 June 2027. ANSWER: 2027-06-06 | ||
| long_context-005private | pass on its page | fail on its page |
| long_context-008private | fail on its page | pass on its page |
| long_context-014private | pass on its page | fail on its page |
| long_context-017publicall models' answers | fail on its page | pass on its page |
Show both answers to long_context-017Qwen3-4B From the documents, we can see the following orders for battery packs for the Project Ibis pilot: 1. In the "PROJECT STATUS REPORT | Project Harbinger" document, it is mentioned that "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot." 2. In the "MEETING MINUTES | Ref MIN/26/652" document, it is mentioned that "Logistics group reviewed the firmware rollout backlog and agreed to defer 25 low-priority items to the next cycle." However, this does not mention an order for battery packs. 3. In the "LAB NOTEBOOK | Entry LAB/26/105" document, it is mentioned that "The Project Ibis team asked that the shift rostering figures be checked against the ledger before wider circulation." This does not mention an order for battery packs. 4. In the "INTERNAL MEMO | Ref MEM/26/305" document, it is mentioned that "The Data Platform group reviewed the supplier onboarding backlog and agreed to defer 35 low-priority items to the next cycle." This does not mention an order for battery packs. Based on the information provided, only Procurement placed an order for 6,236 battery packs for the Project Ibis pilot. There is no mention of Logistics placing an order for battery packs. ANSWER: 6236 Qwen3-8B To determine the total number of battery packs ordered for the Project Ibis pilot, we need to look for mentions of battery packs in the provided documents. 1. From the **PROJECT STATUS REPORT | Project Harbinger**: - "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot." 2. From the **INTERNAL MEMO | Ref MEM/26/636**: - "Logistics placed an order for 8,774 battery packs to equip the Project Ibis pilot." Adding these two quantities: - 6,236 (from Procurement) + 8,774 (from Logistics) = 15,010 ANSWER: 15010 | ||
Multilingual
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| multilingual-005private | fail on its page | pass on its page |
| multilingual-007private | fail on its page | pass on its page |
| multilingual-020private | 67% on its page | pass on its page |
Famous gotchas
| Question | Qwen3-4B | Qwen3-8B |
|---|---|---|
| gotchas-006private | pass on its page | fail on its page |
| gotchas-010publicall models' answers | pass on its page | fail on its page |
Show both answers to gotchas-010Qwen3-4B We are given a variation of the **Monty Hall problem**, but with a twist: the **host does not know where the car is** and opens **door 3 at random**, revealing a **goat**.
Let’s analyze the situation step by step.
---
### Step 1: Initial Setup
- There are **3 doors**: 1 has a **car**, 2 have **goats**.
- You pick **door 1**.
- The host **does not know** where the car is.
- The host **randomly opens door 3**, and it happens to be a **goat**.
We are to find the **probability that the car is behind door 2**.
---
### Step 2: Consider All Possible Scenarios
There are **3 possible initial configurations** (since the car is equally likely to be behind any of the 3 doors):
1. **Car is behind door 1**
2. **Car is behind door 2**
3. **Car is behind door 3**
Let’s analyze each case:
---
#### Case 1: Car is behind **door 1**
- You picked door 1.
- Host opens door 3 (randomly), and it's a goat.
- This is **possible**.
- In this case, the car is **not** behind door 2.
#### Case 2: Car is behind **door 2**
- You picked door 1.
- Host opens door 3 (randomly), and it's a goat.
- This is **possible**.
- In this case, the car **is** behind door 2.
#### Case 3: Car is behind **door 3**
- You picked door 1.
- Host opens door 3 (randomly), and it's a goat.
- This is **not possible**, because the host would have revealed the car.
So, **Case 3 is invalid**.
---
### Step 3: Count Valid Cases
- Case 1: Valid → Car is behind door 1 → Not behind door 2.
- Case 2: Valid → Car is behind door 2.
- Case 3: Invalid → Car is behind door 3 → Host would have revealed the car.
So, only **2 valid cases** remain: Case 1 and Case 2.
Out of these, **1 case** has the car behind door 2.
---
### Step 4: Compute the Probability
- Total valid cases: 2
- Favorable case: 1
So, the probability that the car is behind **door 2** is:
$$
\frac{1}{2} = 0.5
$$
---
### ✅ Final Answer:
**ANSWER: 0.5**
Qwen3-8B We are given a variation of the **Monty Hall problem**, where: - There are **3 doors**: one has a **car**, the other two have **goats**. - You pick **door 1**. - The **host**, who **does not know** where the car is, opens **door 3** at random. - It turns out that **door 3** has a **goat**. - Now, we are to find the **probability that the car is behind door 2**. --- ### Step-by-step analysis: Let’s define the possible scenarios: There are **3 doors**: 1, 2, 3. Let’s assume the car is behind one of the doors. Since the host opens a door **at random**, and it turns out to be a **goat**, we need to consider all possible placements of the car and how the host's action affects the probabilities. --- ### Step 1: List all possible placements of the car There are 3 possible placements of the car: 1. **Car is behind door 1** 2. **Car is behind door 2** 3. **Car is behind door 3** --- ### Step 2: Consider the host's action You picked **door 1**. The host opens **door 3** at random. So, the host has a 50% chance of opening **door 2** or **door 3**. But we are told that the host **opened door 3**, and it showed a **goat**. So, we are only considering the cases where the host **opened door 3**. Let’s consider each case: --- ### Case 1: Car is behind **door 1** - You picked door 1. - Host opens door 3 at random. - Since the car is behind door 1, the host has a 50% chance of opening door 2 or door 3. - So, the host **could** open door 3 (and it would show a goat). - So, this case is **possible**. --- ### Case 2: Car is behind **door 2** - You picked door 1. - Host opens door 3 at random. - Since the car is behind door 2, the host has a 50% chance of opening door 2 or door 3. - So, the host **could** open door 3 (and it would show a goat). - So, this case is **possible**. --- ### Case 3: Car is behind **door 3** - You picked door 1. - Host opens door 3 at random. - But the car is behind door 3, so the host **cannot** open door 3 (because it has the car). - S Cut here for length. The model page has the full answer. | ||
The same numbers as JSON: compare.json. Answers appear there only for public questions.