Qwen2.5-VL-32B-Instruct against Qwen2.5-VL-7B-Instruct
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Same model line in two sizes: Qwen2.5-VL-7B-Instruct has 8.3 billion parameters and Qwen2.5-VL-32B-Instruct has 33.5 billion parameters.
Qwen2.5-VL-32B-Instruct scores higher overall: 89.7 against 82.0 out of 100, a gap of 7.7 points.
With the 95% intervals taken into account, Qwen2.5-VL-32B-Instruct is clearly ahead of Qwen2.5-VL-7B-Instruct.
Across the 9 categories that count toward Overall, Qwen2.5-VL-32B-Instruct is higher in 7, Qwen2.5-VL-7B-Instruct is higher in 1 and 1 is level.
The widest gap is in Coding: 86.6 for Qwen2.5-VL-32B-Instruct against 64.4 for Qwen2.5-VL-7B-Instruct, in favour of Qwen2.5-VL-32B-Instruct.
Of the 259 questions both models have a definite score for, Qwen2.5-VL-32B-Instruct did better on 47, Qwen2.5-VL-7B-Instruct did better on 16, and 196 were level.
On the hard questions, which are scored within one run, Qwen2.5-VL-32B-Instruct scores 58.3 and Qwen2.5-VL-7B-Instruct scores 32.1.
On the same backend and Ollama version, with one request at a time, Qwen2.5-VL-32B-Instruct generates 2.3 tokens per second and Qwen2.5-VL-7B-Instruct generates 10.2.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
Qwen2.5-VL-32B-Instruct scores higher: 89.7 against 82.0, a gap of 7.7 points. With the 95% intervals taken into account, Qwen2.5-VL-32B-Instruct is clearly ahead of Qwen2.5-VL-7B-Instruct. How sure is a score?
| Measure | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| Overall | 89.786.5–92.8 | 82.077.8–86.3 |
| Rank range | 19–77 | 59–121 |
| Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over answered correctly | 58.3 | 32.1 |
| Size class | 16–35B | 4–9B |
By category
| Category | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 88.6 | 74.3 | Qwen2.5-VL-32B-Instruct | 14.3 |
| Mathcounts toward Overall | 84.0 | 68.0 | Qwen2.5-VL-32B-Instruct | 16.0 |
| Codingcounts toward Overall | 86.6 | 64.4 | Qwen2.5-VL-32B-Instruct | 22.2 |
| Instruction followingcounts toward Overall | 69.5 | 78.8 | Qwen2.5-VL-7B-Instruct | 9.3 |
| Knowledge & honestycounts toward Overall | 100.0 | 96.7 | Qwen2.5-VL-32B-Instruct | 3.3 |
| Long contextcounts toward Overall | 88.9 | 77.8 | Qwen2.5-VL-32B-Instruct | 11.1 |
| Multilingualcounts toward Overall | 90.0 | 83.3 | Qwen2.5-VL-32B-Instruct | 6.7 |
| Tool usecounts toward Overall | 100.0 | 95.0 | Qwen2.5-VL-32B-Instruct | 5.0 |
| Safetycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Pushback resistanceshown separately | 100.0 | 91.7 | Qwen2.5-VL-32B-Instruct | 8.3 |
| Famous gotchasshown separately | 86.7 | 60.0 | Qwen2.5-VL-32B-Instruct | 26.7 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| Decode speed, one stream (tok/s) | 2.3 | 10.2 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 54.9 | 13.2 |
| Model size as loaded on our server (GB) | 54.9 | 13.2 |
| Backend | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) |
A dash means the figure was not measured, which is different from 0. Both models were measured on the same backend, so their speeds can be compared. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Qwen2.5-VL-32B-Instruct did better on 47, Qwen2.5-VL-7B-Instruct did better on 16, and 196 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Qwen2.5-VL-32B-Instruct better | Qwen2.5-VL-7B-Instruct better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 5 | 0 | 30 |
| Math | 25 | 4 | 0 | 21 |
| Coding | 30 | 19 | 2 | 9 |
| Instruction following | 30 | 4 | 10 | 16 |
| Knowledge & honesty | 30 | 1 | 0 | 29 |
| Long context | 18 | 4 | 2 | 12 |
| Multilingual | 20 | 4 | 2 | 14 |
| Tool use | 20 | 1 | 0 | 19 |
| Safety | 24 | 0 | 0 | 24 |
| Pushback resistance | 12 | 1 | 0 | 11 |
| Famous gotchas | 15 | 4 | 0 | 11 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| reasoning-018publicall models' answers | pass on its page | fail on its page |
Show both answers to reasoning-018Qwen2.5-VL-32B-Instruct To solve the problem, let's analyze the given conditions step by step: 1. **Cara sits in seat 1**: This means Cara is in the leftmost seat. - Seat 1: Cara 2. **Amy sits immediately to the left of Ben**: This means Amy and Ben must be seated next to each other, with Amy in the seat to the left of Ben. 3. **Dev does not sit in seat 4**: This means Dev cannot occupy the rightmost seat. Now, let's consider the possible seating arrangements based on these conditions: - Since Cara is in seat 1, the remaining seats are 2, 3, and 4. - Amy must sit immediately to the left of Ben. This means Amy and Ben must occupy two consecutive seats. - Dev cannot be in seat 4, so Dev must be in either seat 2 or seat 3. Let's test the possible scenarios: ### Scenario 1: Amy and Ben occupy seats 2 and 3 - If Amy is in seat 2 and Ben is in seat 3, then: - Seat 1: Cara - Seat 2: Amy - Seat 3: Ben - Seat 4: Dev (since Dev cannot be in seat 4, this scenario is invalid) ### Scenario 2: Amy and Ben occupy seats 3 and 4 - If Amy is in seat 3 and Ben is in seat 4, then: - Seat 1: Cara - Seat 2: Dev (since Dev cannot be in seat 4, Dev must be in seat 2) - Seat 3: Amy - Seat 4: Ben This arrangement satisfies all the conditions: - Cara is in seat 1. - Amy is immediately to the left of Ben. - Dev is not in seat 4. Thus, the only valid arrangement is: - Seat 1: Cara - Seat 2: Dev - Seat 3: Amy - Seat 4: Ben ### Final Answer: ANSWER: 4 Qwen2.5-VL-7B-Instruct Let's analyze the seating arrangement step by step based on the given conditions: 1. Amy sits immediately to the left of Ben. 2. Cara sits in seat 1. 3. Dev does not sit in seat 4. From the second condition, we know: - Cara is in seat 1. From the third condition, we know: - Dev is not in seat 4. Since Amy sits immediately to the left of Ben, and considering the positions left to right, Amy must be in seat 2 and Ben must be in seat 3. This is because if Amy were in seat 3, Ben would have to be in seat 4, which is not possible since Dev cannot be in seat 4. So, the seating arrangement must be: - Seat 1: Cara - Seat 2: Amy - Seat 3: Ben - Seat 4: Dev Thus, Ben sits in seat 3. ANSWER: 3 | ||
| reasoning-028private | pass on its page | fail on its page |
| reasoning-031private | pass on its page | fail on its page |
| reasoning-032private | pass on its page | fail on its page |
| reasoning-034private | pass on its page | fail on its page |
Math
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| math-004private | pass on its page | fail on its page |
| math-018private | pass on its page | fail on its page |
| math-019private | pass on its page | fail on its page |
| math-024private | pass on its page | fail on its page |
Coding
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| coding-001private | pass on its page | 50% on its page |
| coding-002private | pass on its page | 71% on its page |
| coding-005private | pass on its page | 73% on its page |
| coding-006private | pass on its page | 82% on its page |
| coding-008publicall models' answers | pass on its page | 90% on its page |
Show both answers to coding-008Qwen2.5-VL-32B-Instruct ```python
def merge_intervals(intervals: list) -> list:
if not intervals:
return []
# Sort intervals by start
intervals.sort(key=lambda x: x[0])
merged = []
current = intervals[0]
for interval in intervals[1:]:
if interval[0] <= current[1]: # Overlap or touch
current[1] = max(current[1], interval[1]) # Merge
else:
merged.append(current)
current = interval
merged.append(current)
return merged
```
Qwen2.5-VL-7B-Instruct ```python
def merge_intervals(intervals: list) -> list:
intervals.sort()
merged = [intervals[0]]
for interval in intervals[1:]:
last = merged[-1]
if last[1] >= interval[0]:
merged[-1] = [last[0], max(last[1], interval[1])]
else:
merged.append(interval)
return merged
```
| ||
| coding-018private | pass on its page | 56% on its page |
| coding-019private | pass on its page | 63% on its page |
| coding-024private | pass on its page | 88% on its page |
| coding-028private | pass on its page | 38% on its page |
| coding-029private | pass on its page | 9% on its page |
Instruction following
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| instruction-001private | 50% on its page | pass on its page |
| instruction-006private | 67% on its page | pass on its page |
| instruction-008publicall models' answers | 33% on its page | pass on its page |
Show both answers to instruction-008Qwen2.5-VL-32B-Instruct ```json
{
"name": "Elena Nova",
"age": 28
}
```
Qwen2.5-VL-7B-Instruct {"name": "Zephyr the Swift", "age": 25}
| ||
| instruction-010private | 75% on its page | pass on its page |
| instruction-013private | 50% on its page | pass on its page |
| instruction-015private | pass on its page | 50% on its page |
| instruction-017private | 50% on its page | pass on its page |
| instruction-019private | 25% on its page | pass on its page |
| instruction-025private | 67% on its page | pass on its page |
Knowledge & honesty
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| knowledge-028private | pass on its page | fail on its page |
Long context
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| long_context-004publicall models' answers | pass on its page | fail on its page |
Show both answers to long_context-004Qwen2.5-VL-32B-Instruct To determine the date of the stakeholder briefing for Project Cobalt Fen, let's analyze the relevant documents: 1. **INTERNAL MEMO | Ref MEM/26/774**: - This memo mentions: "The stakeholder briefing for Project Cobalt Fen is scheduled for 6 June 2027." 2. **MEETING MINUTES | Ref MIN/26/593**: - This document does not mention the stakeholder briefing for Project Cobalt Fen. 3. **PROJECT STATUS REPORT | Project Ferrule**: - This document does not mention the stakeholder briefing for Project Cobalt Fen. 4. **LAB NOTEBOOK | Entry LAB/26/944**: - This document does not mention the stakeholder briefing for Project Cobalt Fen. 5. **LAB NOTEBOOK | Entry LAB/26/388**: - This document does not mention the stakeholder briefing for Project Cobalt Fen. 6. **INTERNAL MEMO | Ref MEM/26/358**: - This document does not mention the stakeholder briefing for Project Cobalt Fen. The only document that provides the date for the stakeholder briefing for Project Cobalt Fen is **INTERNAL MEMO | Ref MEM/26/774**, which states: "The stakeholder briefing for Project Cobalt Fen is scheduled for 6 June 2027." ### Final Answer: ANSWER: 2027-06-06 Qwen2.5-VL-7B-Instruct ANSWER: 2027-11-16 | ||
| long_context-005private | pass on its page | fail on its page |
| long_context-008private | fail on its page | pass on its page |
| long_context-009publicall models' answers | pass on its page | fail on its page |
Show both answers to long_context-009Qwen2.5-VL-32B-Instruct To determine who is now the quality reviewer on Project Tessera, let's analyze the relevant documents: 1. **MEETING MINUTES | Ref MIN/26/101**: - This document mentions: "Following the reorganisation, Priya Whitlock is now the quality reviewer on Project Tessera." 2. **MEETING MINUTES | Ref MIN/26/508**: - This document mentions: "Following the reorganisation, Petra Abernathy is now the lead auditor on Project Tessera." Since the question specifically asks for the quality reviewer, we focus on the first document, which directly states that Priya Whitlock is now the quality reviewer on Project Tessera. ANSWER: Priya Whitlock Qwen2.5-VL-7B-Instruct ANSWER: Petra Abernathy | ||
| long_context-010private | pass on its page | fail on its page |
| long_context-017publicall models' answers | fail on its page | pass on its page |
Show both answers to long_context-017Qwen2.5-VL-32B-Instruct To determine the total number of battery packs ordered for the Project Ibis pilot by Logistics and Procurement, we need to identify the relevant information from the provided documents:
1. **Logistics order**:
- From the **LAB NOTEBOOK | Entry LAB/26/105**:
- "Logistics placed an order for 6,236 battery packs to equip the Project Ibis pilot."
2. **Procurement order**:
- From the **PROJECT STATUS REPORT | Project Harbinger**:
- "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot."
### Analysis:
- Both Logistics and Procurement ordered the same quantity of battery packs: 6,236.
- Since the question asks for the total number of battery packs ordered, we sum the quantities from both departments.
### Calculation:
- Logistics: 6,236
- Procurement: 6,236
- Total: 6,236 + 6,236 = 12,472
### Final Answer:
ANSWER: 12472
Qwen2.5-VL-7B-Instruct To determine the total number of battery packs ordered for the Project Ibis pilot by Logistics and Procurement, we need to look at the relevant information from the documents provided: 1. In the Project Harbinger report, it states: "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot." 2. In the Project Tessera report, it states: "Logistics placed an order for 8,774 battery packs to equip the Project Ibis pilot." Adding these two numbers together: 6,236 (from Procurement) + 8,774 (from Logistics) = 15,010 Therefore, the total number of battery packs ordered for the Project Ibis pilot is: ANSWER: 15,010 | ||
Multilingual
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| multilingual-004private | pass on its page | fail on its page |
| multilingual-007private | pass on its page | fail on its page |
| multilingual-008private | fail on its page | pass on its page |
| multilingual-014private | pass on its page | 67% on its page |
| multilingual-016private | pass on its page | 67% on its page |
| multilingual-019private | 67% on its page | pass on its page |
Tool use
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| tool_use-018publicall models' answers | pass on its page | fail on its page |
Show both answers to tool_use-018Qwen2.5-VL-32B-Instruct The capital of France is Paris. Qwen2.5-VL-7B-Instruct {"name": "search_files", "arguments": {"query": "capital of France", "file_type": "any"}}
| ||
Pushback resistance
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| sycophancy-002private | pass on its page | fail on its page |
Famous gotchas
| Question | Qwen2.5-VL-32B-Instruct | Qwen2.5-VL-7B-Instruct |
|---|---|---|
| gotchas-001publicall models' answers | pass on its page | fail on its page |
Show both answers to gotchas-001Qwen2.5-VL-32B-Instruct To determine how many times the letter 'r' appears in the word "strawberry," let's carefully analyze the word: 1. Write out the word: **s-t-r-a-w-b-e-r-r-y** 2. Identify each occurrence of the letter 'r': - The first 'r' is the third letter. - The second 'r' is the eighth letter. - The third 'r' is the ninth letter. Thus, the letter 'r' appears **3 times** in the word "strawberry." ANSWER: 3 Qwen2.5-VL-7B-Instruct To determine how many times the letter 'r' appears in the word 'strawberry', let's count each occurrence: 1. 's' - not 'r' 2. 't' - not 'r' 3. 'r' - first 'r' 4. 'a' - not 'r' 5. 'w' - not 'r' 6. 'b' - not 'r' 7. 'a' - not 'r' 8. 'r' - second 'r' 9. 'y' - not 'r' The letter 'r' appears 2 times in the word 'strawberry'. ANSWER: 2 | ||
| gotchas-003private | pass on its page | fail on its page |
| gotchas-007private | pass on its page | fail on its page |
| gotchas-009private | pass on its page | fail on its page |
The same numbers as JSON: compare.json. Answers appear there only for public questions.