Qwen3-4B-Instruct-2507 against Qwen3-4B-Thinking-2507

Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.

The short version

Qwen3-4B-Thinking-2507 is named as a reasoning variant of Qwen3-4B-Instruct-2507: the same family and size.

Qwen3-4B-Instruct-2507 scores higher overall: 90.3 against 83.5 out of 100, a gap of 6.8 points.

With the 95% intervals taken into account, Qwen3-4B-Instruct-2507 is clearly ahead of Qwen3-4B-Thinking-2507.

Across the 9 categories that count toward Overall, Qwen3-4B-Instruct-2507 is higher in 5, Qwen3-4B-Thinking-2507 is higher in 2 and 2 are level.

The widest gap is in Coding: 69.7 for Qwen3-4B-Instruct-2507 against 26.7 for Qwen3-4B-Thinking-2507, in favour of Qwen3-4B-Instruct-2507.

Of the 259 questions both models have a definite score for, Qwen3-4B-Instruct-2507 did better on 37, Qwen3-4B-Thinking-2507 did better on 11, and 211 were level.

Their speeds were not measured on one shared backend and Ollama version, so they are not compared.

Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.

Overall

Qwen3-4B-Instruct-2507 scores higher: 90.3 against 83.5, a gap of 6.8 points. With the 95% intervals taken into account, Qwen3-4B-Instruct-2507 is clearly ahead of Qwen3-4B-Thinking-2507. How sure is a score?

Overall (0–100) with its 95% interval, the rank range among all listed ranked models, and the hard-question score.
MeasureQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
Overall90.387.2–93.383.580.2–87.0
Rank range13–7347–111
Hard questionsmean score on the questions that half or fewer of the ranked models (chosen per run, see the methodology) answered correctly57.2—
Size class4–9B4–9B

By category

Programmatic mean (0–100) per category. Higher is the model with the larger mean; a gap that rounds to 0.0 is called level. A dash means not scored, which is different from 0.
CategoryQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507HigherDifference
Reasoningcounts toward Overall 82.9 74.3 Qwen3-4B-Instruct-2507 8.6
Mathcounts toward Overall 80.0 76.0 Qwen3-4B-Instruct-2507 4.0
Codingcounts toward Overall 69.7 26.7 Qwen3-4B-Instruct-2507 43.0
Instruction followingcounts toward Overall 94.2 88.6 Qwen3-4B-Instruct-2507 5.6
Knowledge & honestycounts toward Overall 96.7 100.0 Qwen3-4B-Thinking-2507 3.3
Long contextcounts toward Overall 88.9 94.4 Qwen3-4B-Thinking-2507 5.5
Multilingualcounts toward Overall 100.0 91.7 Qwen3-4B-Instruct-2507 8.3
Tool usecounts toward Overall 100.0 100.0 Level 0
Safetycounts toward Overall 100.0 100.0 Level 0
Pushback resistanceshown separately 100.0 100.0 Level 0
Famous gotchasshown separately 93.3 93.3 Level 0
Writingshown separately — — — —
Self-descriptionshown separately — — — —

Speed and memory

Measured on one laptop. Decode speed is generation speed with one request at a time.
MeasureQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
Decode speed, one stream (tok/s)17.121.5
GPU memory on our server (1 to 8 answers at a time) (GB)12.52.5
Model size as loaded on our server (GB)12.52.5
BackendROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots)

A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.

Where they differ

Both models have a definite score for 259 questions. Qwen3-4B-Instruct-2507 did better on 37, Qwen3-4B-Thinking-2507 did better on 11, and 211 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.

Questions both models have a score for, by category.
CategoryQuestionsQwen3-4B-Instruct-2507 betterQwen3-4B-Thinking-2507 betterLevel
Reasoning355228
Math254318
Coding3019110
Instruction following305223
Knowledge & honesty300129
Long context181215
Multilingual203017
Tool use200020
Safety240024
Pushback resistance120012
Famous gotchas150015

Questions where only one got full marks

Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.

Reasoning

Reasoning: 7 questions where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
reasoning-021private pass on its page fail on its page
reasoning-023private pass on its page fail on its page
reasoning-024private pass on its page fail on its page
reasoning-027private fail on its page pass on its page
reasoning-030private pass on its page fail on its page
reasoning-032private fail on its page pass on its page
reasoning-035private pass on its page fail on its page

Math

Math: 7 questions where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
math-008private pass on its page fail on its page
math-017private fail on its page pass on its page
math-019private pass on its page fail on its page
math-020private fail on its page pass on its page
math-022private pass on its page fail on its page
math-024private pass on its page fail on its page
math-025private fail on its page pass on its page

Coding

Coding: 7 questions where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
coding-001private pass on its page fail on its page
coding-002private pass on its page fail on its page
coding-006private pass on its page fail on its page
coding-010publicall models' answers pass on its page fail on its page
Show both answers to coding-010

Qwen3-4B-Instruct-2507

```python
def count_islands(grid: list) -> int:
    if not grid or not grid[0]:
        return 0
    
    rows = len(grid)
    cols = len(grid[0])
    visited = [[False] * cols for _ in range(rows)]
    count = 0
    
    def dfs(r, c):
        if (r < 0 or r >= rows or c < 0 or c >= cols or 
            visited[r][c] or grid[r][c] == 0):
            return
        visited[r][c] = True
        # Explore all 4 directions
        dfs(r + 1, c)
        dfs(r - 1, c)
        dfs(r, c + 1)
        dfs(r, c - 1)
    
    for i in range(rows):
        for j in range(cols):
            if grid[i][j] == 1 and not visited[i][j]:
                dfs(i, j)
                count += 1
    
    return count
```

Qwen3-4B-Thinking-2507

(no text stored)
coding-011private pass on its page fail on its page
coding-014publicall models' answers 43% on its page pass on its page
Show both answers to coding-014

Qwen3-4B-Instruct-2507

```python
def reverse_words(s: str) -> str:
    return ' '.join(s.strip().split())
```

Qwen3-4B-Thinking-2507

def reverse_words(s: str) -> str:
    return ' '.join(s.split()[::-1])
coding-025private pass on its page fail on its page

Instruction following

Instruction following: 6 questions where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
instruction-006private pass on its page 67% on its page
instruction-014publicall models' answers pass on its page 25% on its page
Show both answers to instruction-014

Qwen3-4B-Instruct-2507

"Thank you for your email and the opportunity to discuss this matter further. I appreciate your attention to the details and am confident we can reach a mutually beneficial resolution. regards"

Qwen3-4B-Thinking-2507

(no text stored)
instruction-020private 60% on its page pass on its page
instruction-021private 50% on its page pass on its page
instruction-022private pass on its page 33% on its page
instruction-030private pass on its page 67% on its page

Knowledge & honesty

Knowledge & honesty: 1 question where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
knowledge-004private fail on its page pass on its page

Long context

Long context: 3 questions where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
long_context-005private pass on its page fail on its page
long_context-014private fail on its page pass on its page
long_context-016private fail on its page pass on its page

Multilingual

Multilingual: 3 questions where exactly one model got full marks.
QuestionQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507
multilingual-017private pass on its page 33% on its page
multilingual-018private pass on its page 67% on its page
multilingual-019private pass on its page 33% on its page

The same numbers as JSON: compare.json. Answers appear there only for public questions.