Qwen3-4B against Qwen3-8B

Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.

The short version

Same model line in two sizes: Qwen3-4B has 4 billion parameters and Qwen3-8B has 8.2 billion parameters.

Qwen3-8B scores higher overall: 91.5 against 89.2 out of 100, a gap of 2.3 points.

The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.

Across the 9 categories that count toward Overall, Qwen3-4B is higher in 1, Qwen3-8B is higher in 4 and 4 are level.

The widest gap is in Multilingual: 88.3 for Qwen3-4B against 100.0 for Qwen3-8B, in favour of Qwen3-8B.

Of the 259 questions both models have a definite score for, Qwen3-4B did better on 15, Qwen3-8B did better on 20, and 224 were level.

On the hard questions, which are scored within one run, Qwen3-4B scores 58.1 and Qwen3-8B scores 55.5.

On the same backend and Ollama version, with one request at a time, Qwen3-4B generates 17.0 tokens per second and Qwen3-8B generates 9.3.

Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.

Overall

Qwen3-8B scores higher: 91.5 against 89.2, a gap of 2.3 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?

Overall (0–100) with its 95% interval, the rank range among all listed ranked models, and the hard-question score.
MeasureQwen3-4BQwen3-8B
Overall89.285.6–92.491.588.4–94.5
Rank range18–8111–72
Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over answered correctly58.155.5
Size class4–9B4–9B

By category

Programmatic mean (0–100) per category. Higher is the model with the larger mean; a gap that rounds to 0.0 is called level. A dash means not scored, which is different from 0.
CategoryQwen3-4BQwen3-8BHigherDifference
Reasoningcounts toward Overall 88.6 94.3 Qwen3-8B 5.7
Mathcounts toward Overall 88.0 88.0 Level 0
Codingcounts toward Overall 69.0 70.6 Qwen3-8B 1.6
Instruction followingcounts toward Overall 91.0 87.7 Qwen3-4B 3.3
Knowledge & honestycounts toward Overall 100.0 100.0 Level 0
Long contextcounts toward Overall 77.8 83.3 Qwen3-8B 5.5
Multilingualcounts toward Overall 88.3 100.0 Qwen3-8B 11.7
Tool usecounts toward Overall 100.0 100.0 Level 0
Safetycounts toward Overall 100.0 100.0 Level 0
Pushback resistanceshown separately 100.0 100.0 Level 0
Famous gotchasshown separately 100.0 86.7 Qwen3-4B 13.3
Writingshown separately — — — —
Self-descriptionshown separately — — — —

Speed and memory

Measured on one laptop. Decode speed is generation speed with one request at a time.
MeasureQwen3-4BQwen3-8B
Decode speed, one stream (tok/s)17.09.3
GPU memory on our server (1 to 8 answers at a time) (GB)12.724.7
Model size as loaded on our server (GB)12.724.7
BackendROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)

A dash means the figure was not measured, which is different from 0. Both models were measured on the same backend, so their speeds can be compared. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.

Where they differ

Both models have a definite score for 259 questions. Qwen3-4B did better on 15, Qwen3-8B did better on 20, and 224 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.

Questions both models have a score for, by category.
CategoryQuestionsQwen3-4B betterQwen3-8B betterLevel
Reasoning350233
Math251123
Coding304917
Instruction following306222
Knowledge & honesty300030
Long context182313
Multilingual200317
Tool use200020
Safety240024
Pushback resistance120012
Famous gotchas152013

Questions where only one got full marks

Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.

Reasoning

Reasoning: 2 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
reasoning-033private fail on its page pass on its page
reasoning-034private fail on its page pass on its page

Math

Math: 2 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
math-017private fail on its page pass on its page
math-022private pass on its page fail on its page

Coding

Coding: 4 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
coding-004private 25% on its page pass on its page
coding-014publicall models' answers 43% on its page pass on its page
Show both answers to coding-014

Qwen3-4B

```python
def reverse_words(s: str) -> str:
    return ' '.join(s.split())
```

Qwen3-8B

```python
def reverse_words(s: str) -> str:
    return ' '.join(reversed(s.split()))
```
coding-021private 89% on its page pass on its page
coding-022private 64% on its page pass on its page

Instruction following

Instruction following: 6 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
instruction-001private pass on its page 50% on its page
instruction-009private pass on its page 67% on its page
instruction-010private pass on its page 75% on its page
instruction-012private 75% on its page pass on its page
instruction-018private pass on its page 80% on its page
instruction-029private 50% on its page pass on its page

Long context

Long context: 5 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
long_context-004publicall models' answers fail on its page pass on its page
Show both answers to long_context-004

Qwen3-4B

The stakeholder briefing for Project Cobalt Fen is scheduled for 16 November 2027. 

ANSWER: 2027-11-16

Qwen3-8B

The stakeholder briefing for Project Cobalt Fen is scheduled for 6 June 2027. 

ANSWER: 2027-06-06
long_context-005private pass on its page fail on its page
long_context-008private fail on its page pass on its page
long_context-014private pass on its page fail on its page
long_context-017publicall models' answers fail on its page pass on its page
Show both answers to long_context-017

Qwen3-4B

From the documents, we can see the following orders for battery packs for the Project Ibis pilot:

1. In the "PROJECT STATUS REPORT | Project Harbinger" document, it is mentioned that "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot."

2. In the "MEETING MINUTES | Ref MIN/26/652" document, it is mentioned that "Logistics group reviewed the firmware rollout backlog and agreed to defer 25 low-priority items to the next cycle." However, this does not mention an order for battery packs.

3. In the "LAB NOTEBOOK | Entry LAB/26/105" document, it is mentioned that "The Project Ibis team asked that the shift rostering figures be checked against the ledger before wider circulation." This does not mention an order for battery packs.

4. In the "INTERNAL MEMO | Ref MEM/26/305" document, it is mentioned that "The Data Platform group reviewed the supplier onboarding backlog and agreed to defer 35 low-priority items to the next cycle." This does not mention an order for battery packs.

Based on the information provided, only Procurement placed an order for 6,236 battery packs for the Project Ibis pilot. There is no mention of Logistics placing an order for battery packs.

ANSWER: 6236

Qwen3-8B

To determine the total number of battery packs ordered for the Project Ibis pilot, we need to look for mentions of battery packs in the provided documents.

1. From the **PROJECT STATUS REPORT | Project Harbinger**:
   - "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot."

2. From the **INTERNAL MEMO | Ref MEM/26/636**:
   - "Logistics placed an order for 8,774 battery packs to equip the Project Ibis pilot."

Adding these two quantities:
- 6,236 (from Procurement) + 8,774 (from Logistics) = 15,010

ANSWER: 15010

Multilingual

Multilingual: 3 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
multilingual-005private fail on its page pass on its page
multilingual-007private fail on its page pass on its page
multilingual-020private 67% on its page pass on its page

Famous gotchas

Famous gotchas: 2 questions where exactly one model got full marks.
QuestionQwen3-4BQwen3-8B
gotchas-006private pass on its page fail on its page
gotchas-010publicall models' answers pass on its page fail on its page
Show both answers to gotchas-010

Qwen3-4B

We are given a variation of the **Monty Hall problem**, but with a twist: the **host does not know where the car is** and opens **door 3 at random**, revealing a **goat**.

Let’s analyze the situation step by step.

---

### Step 1: Initial Setup

- There are **3 doors**: 1 has a **car**, 2 have **goats**.
- You pick **door 1**.
- The host **does not know** where the car is.
- The host **randomly opens door 3**, and it happens to be a **goat**.

We are to find the **probability that the car is behind door 2**.

---

### Step 2: Consider All Possible Scenarios

There are **3 possible initial configurations** (since the car is equally likely to be behind any of the 3 doors):

1. **Car is behind door 1**  
2. **Car is behind door 2**  
3. **Car is behind door 3**

Let’s analyze each case:

---

#### Case 1: Car is behind **door 1**

- You picked door 1.
- Host opens door 3 (randomly), and it's a goat.
- This is **possible**.
- In this case, the car is **not** behind door 2.

#### Case 2: Car is behind **door 2**

- You picked door 1.
- Host opens door 3 (randomly), and it's a goat.
- This is **possible**.
- In this case, the car **is** behind door 2.

#### Case 3: Car is behind **door 3**

- You picked door 1.
- Host opens door 3 (randomly), and it's a goat.
- This is **not possible**, because the host would have revealed the car.

So, **Case 3 is invalid**.

---

### Step 3: Count Valid Cases

- Case 1: Valid → Car is behind door 1 → Not behind door 2.
- Case 2: Valid → Car is behind door 2.
- Case 3: Invalid → Car is behind door 3 → Host would have revealed the car.

So, only **2 valid cases** remain: Case 1 and Case 2.

Out of these, **1 case** has the car behind door 2.

---

### Step 4: Compute the Probability

- Total valid cases: 2
- Favorable case: 1

So, the probability that the car is behind **door 2** is:

$$
\frac{1}{2} = 0.5
$$

---

### ✅ Final Answer:

**ANSWER: 0.5**

Qwen3-8B

We are given a variation of the **Monty Hall problem**, where:

- There are **3 doors**: one has a **car**, the other two have **goats**.
- You pick **door 1**.
- The **host**, who **does not know** where the car is, opens **door 3** at random.
- It turns out that **door 3** has a **goat**.
- Now, we are to find the **probability that the car is behind door 2**.

---

### Step-by-step analysis:

Let’s define the possible scenarios:

There are **3 doors**: 1, 2, 3.

Let’s assume the car is behind one of the doors. Since the host opens a door **at random**, and it turns out to be a **goat**, we need to consider all possible placements of the car and how the host's action affects the probabilities.

---

### Step 1: List all possible placements of the car

There are 3 possible placements of the car:

1. **Car is behind door 1**
2. **Car is behind door 2**
3. **Car is behind door 3**

---

### Step 2: Consider the host's action

You picked **door 1**. The host opens **door 3** at random. So, the host has a 50% chance of opening **door 2** or **door 3**.

But we are told that the host **opened door 3**, and it showed a **goat**.

So, we are only considering the cases where the host **opened door 3**.

Let’s consider each case:

---

### Case 1: Car is behind **door 1**

- You picked door 1.
- Host opens door 3 at random.
- Since the car is behind door 1, the host has a 50% chance of opening door 2 or door 3.
- So, the host **could** open door 3 (and it would show a goat).
- So, this case is **possible**.

---

### Case 2: Car is behind **door 2**

- You picked door 1.
- Host opens door 3 at random.
- Since the car is behind door 2, the host has a 50% chance of opening door 2 or door 3.
- So, the host **could** open door 3 (and it would show a goat).
- So, this case is **possible**.

---

### Case 3: Car is behind **door 3**

- You picked door 1.
- Host opens door 3 at random.
- But the car is behind door 3, so the host **cannot** open door 3 (because it has the car).
- S

Cut here for length. The model page has the full answer.

The same numbers as JSON: compare.json. Answers appear there only for public questions.