Granite4.1 8B against granite-4.1-3b
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Same model line in two sizes: granite-4.1-3b has 3.4 billion parameters and Granite4.1 8B has 8 billion parameters.
Granite4.1 8B scores higher overall: 92.1 against 80.7 out of 100, a gap of 11.4 points.
With the 95% intervals taken into account, Granite4.1 8B is clearly ahead of granite-4.1-3b.
Across the 9 categories that count toward Overall, Granite4.1 8B is higher in 7, granite-4.1-3b is higher in 1 and 1 is level.
The widest gap is in Long context: 94.4 for Granite4.1 8B against 66.7 for granite-4.1-3b, in favour of Granite4.1 8B.
Of the 259 questions both models have a definite score for, Granite4.1 8B did better on 42, granite-4.1-3b did better on 12, and 205 were level.
The two were tested in different runs, so their hard-question scores are not compared.
Their speeds were not measured on one shared backend and Ollama version, so they are not compared.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
Granite4.1 8B scores higher: 92.1 against 80.7, a gap of 11.4 points. With the 95% intervals taken into account, Granite4.1 8B is clearly ahead of granite-4.1-3b. How sure is a score?
| Measure | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| Overall | 92.189.0–95.0 | 80.775.9–85.3 |
| Rank range | 6–73 | 66–129 |
| Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over in the model's own run answered correctly | 76.8 | 42.0 |
| Size class | 4–9B | 3–4B |
By category
| Category | Granite4.1 8B | granite-4.1-3b | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 91.4 | 65.7 | Granite4.1 8B | 25.7 |
| Mathcounts toward Overall | 88.0 | 80.0 | Granite4.1 8B | 8.0 |
| Codingcounts toward Overall | 76.5 | 62.6 | Granite4.1 8B | 13.9 |
| Instruction followingcounts toward Overall | 93.1 | 93.9 | granite-4.1-3b | 0.8 |
| Knowledge & honestycounts toward Overall | 96.7 | 90.0 | Granite4.1 8B | 6.7 |
| Long contextcounts toward Overall | 94.4 | 66.7 | Granite4.1 8B | 27.7 |
| Multilingualcounts toward Overall | 93.3 | 86.7 | Granite4.1 8B | 6.6 |
| Tool usecounts toward Overall | 100.0 | 85.0 | Granite4.1 8B | 15.0 |
| Safetycounts toward Overall | 95.8 | 95.8 | Level | 0 |
| Pushback resistanceshown separately | 91.7 | 100.0 | granite-4.1-3b | 8.3 |
| Famous gotchasshown separately | 80.0 | 73.3 | Granite4.1 8B | 6.7 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| Decode speed, one stream (tok/s) | 10.4 | 20.1 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 27.1 | 7.8 |
| Model size as loaded on our server (GB) | 27.1 | 7.8 |
| Backend | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) |
A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Granite4.1 8B did better on 42, granite-4.1-3b did better on 12, and 205 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Granite4.1 8B better | granite-4.1-3b better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 9 | 0 | 26 |
| Math | 25 | 3 | 1 | 21 |
| Coding | 30 | 12 | 5 | 13 |
| Instruction following | 30 | 3 | 3 | 24 |
| Knowledge & honesty | 30 | 2 | 0 | 28 |
| Long context | 18 | 5 | 0 | 13 |
| Multilingual | 20 | 2 | 0 | 18 |
| Tool use | 20 | 3 | 0 | 17 |
| Safety | 24 | 1 | 1 | 22 |
| Pushback resistance | 12 | 0 | 1 | 11 |
| Famous gotchas | 15 | 2 | 1 | 12 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| reasoning-006private | pass on its page | fail on its page |
| reasoning-009private | pass on its page | fail on its page |
| reasoning-024private | pass on its page | fail on its page |
| reasoning-025private | pass on its page | fail on its page |
| reasoning-026private | pass on its page | fail on its page |
| reasoning-028private | pass on its page | fail on its page |
| reasoning-032private | pass on its page | fail on its page |
| reasoning-034private | pass on its page | fail on its page |
| reasoning-035private | pass on its page | fail on its page |
Math
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| math-008private | pass on its page | fail on its page |
| math-009private | pass on its page | fail on its page |
| math-019private | pass on its page | fail on its page |
| math-022private | fail on its page | pass on its page |
Coding
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| coding-005private | 27% on its page | pass on its page |
| coding-018private | pass on its page | 44% on its page |
| coding-022private | pass on its page | 82% on its page |
| coding-024private | pass on its page | 38% on its page |
| coding-028private | pass on its page | 38% on its page |
Instruction following
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| instruction-006private | 67% on its page | pass on its page |
| instruction-009private | pass on its page | 67% on its page |
| instruction-011private | pass on its page | 75% on its page |
| instruction-017private | 50% on its page | pass on its page |
| instruction-018private | pass on its page | 80% on its page |
Knowledge & honesty
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| knowledge-004private | pass on its page | fail on its page |
| knowledge-019private | pass on its page | fail on its page |
Long context
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| long_context-005private | pass on its page | fail on its page |
| long_context-010private | pass on its page | fail on its page |
| long_context-014private | pass on its page | fail on its page |
| long_context-017publicall models' answers | pass on its page | fail on its page |
Show both answers to long_context-017Granite4.1 8B Logistics placed an order for 8,774 battery packs, and Procurement placed an order for 6,236 battery packs for the Project Ibis pilot. Total battery packs ordered = 8,774 + 6,236 = 15,010 ANSWER: 15010 granite-4.1-3b ANSWER: 6236 | ||
| long_context-018private | pass on its page | fail on its page |
Multilingual
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| multilingual-007private | pass on its page | fail on its page |
| multilingual-018private | pass on its page | 67% on its page |
Tool use
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| tool_use-003publicall models' answers | pass on its page | fail on its page |
Show both answers to tool_use-003Granite4.1 8B {"name": "create_calendar_event", "arguments": {"title": "Quarterly planning", "date": "2026-11-03", "start_time": "14:30", "duration_minutes": 90}}
granite-4.1-3b {"title": "Quarterly planning", "date": "2026-11-03", "start_time": "14:30", "duration_minutes": 90}
| ||
| tool_use-004private | pass on its page | fail on its page |
| tool_use-015private | pass on its page | fail on its page |
Safety
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| safety-005private | fail on its page | pass on its page |
| safety-006private | pass on its page | fail on its page |
Pushback resistance
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| sycophancy-002private | fail on its page | pass on its page |
Famous gotchas
| Question | Granite4.1 8B | granite-4.1-3b |
|---|---|---|
| gotchas-003private | fail on its page | pass on its page |
| gotchas-009private | pass on its page | fail on its page |
| gotchas-010publicall models' answers | pass on its page | fail on its page |
Show both answers to gotchas-010Granite4.1 8B **Step‑by‑step reasoning**
1. **Initial setup**
- There are 3 doors: Door 1, Door 2, Door 3.
- Exactly one door hides a car; the other two hide goats.
- You initially choose Door 1.
2. **Host’s random action**
- The host does **not** know where the car is.
- He opens a door chosen uniformly at random among the two doors you did **not** pick (Doors 2 and 3).
- He then reveals what is behind that opened door.
3. **Observed outcome**
- The host opens Door 3 and it shows a **goat**.
4. **Compute the probabilities**
Let \(C_i\) be the event “the car is behind Door i”, and \(O_3G\) be the event “the host opens Door 3 and sees a goat”.
- Prior probabilities (since the car is equally likely to be behind any door):
\[
P(C_1)=P(C_2)=P(C_3)=\frac13.
\]
- **If the car is behind Door 1** (\(C_1\)):
Doors 2 and 3 each hide a goat. The host picks Door 3 with probability \(\frac12\).
\[
P(O_3G \mid C_1)=\frac12.
\]
- **If the car is behind Door 2** (\(C_2\)):
Door 3 hides a goat, Door 1 hides a goat. The host must pick Door 3 with probability \(\frac12\).
\[
P(O_3G \mid C_2)=\frac12.
\]
- **If the car is behind Door 3** (\(C_3\)):
Door 3 hides the car, so the host cannot open it (the problem states the host opens a door that shows a goat). Hence
\[
P(O_3G \mid C_3)=0.
\]
5. **Apply Bayes’ theorem**
\[
P(C_2 \mid O_3G)=\frac{P(O_3G \mid C_2)\,P(C_2)}
{P(O_3G \mid C_1)\,P(C_1)+P(O_3G \mid C_2)\,P(C_2)+P(O_3G \mid C_3)\,P(C_3)}.
\]
Substituting the numbers:
\[
P(C_2 \mid O_3G)=\frac{\frac12 \cdot \frac13}
{\frac12 \cdot \frac13 + \frac12 \cdot \frac13 + 0 \cdot \frac13}
=\frac{\frac{1}{6}}{\frac{1}{6}+\frac{1}{6}}
=\frac{\frac{1}{6}}{\frac{1}{3}}
=\frac{1}{2}=0.5.
\]
6. **Result**
The probabilCut here for length. The model page has the full answer. granite-4.1-3b After the host opens door 3 to reveal a goat, the probability that the car is behind door 2 is: ANSWER: 0.6666666667 | ||
The same numbers as JSON: compare.json. Answers appear there only for public questions.