Gemma3n E2B IT against Gemma 3n E4B IT
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Same model line in two sizes: Gemma3n E2B IT has 5 billion parameters and Gemma 3n E4B IT has 8 billion parameters.
Gemma 3n E4B IT scores higher overall: 86.1 against 79.9 out of 100, a gap of 6.2 points.
The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.
Across the 9 categories that count toward Overall, Gemma3n E2B IT is higher in 2, Gemma 3n E4B IT is higher in 6 and 1 is level.
The widest gap is in Math: 76.0 for Gemma3n E2B IT against 92.0 for Gemma 3n E4B IT, in favour of Gemma 3n E4B IT.
Of the 259 questions both models have a definite score for, Gemma3n E2B IT did better on 16, Gemma 3n E4B IT did better on 36, and 207 were level.
The two were tested in different runs, so their hard-question scores are not compared.
Their speeds were not measured on one shared backend and Ollama version, so they are not compared.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
Gemma 3n E4B IT scores higher: 86.1 against 79.9, a gap of 6.2 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?
| Measure | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| Overall | 79.975.2–84.7 | 86.182.4–89.6 |
| Rank range | 64–127 | 44–97 |
| Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over in the model's own run answered correctly | 55.7 | 50.7 |
| Size class | 4–9B | 4–9B |
By category
| Category | Gemma3n E2B IT | Gemma 3n E4B IT | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 68.6 | 82.9 | Gemma 3n E4B IT | 14.3 |
| Mathcounts toward Overall | 76.0 | 92.0 | Gemma 3n E4B IT | 16.0 |
| Codingcounts toward Overall | 61.4 | 65.3 | Gemma 3n E4B IT | 3.9 |
| Instruction followingcounts toward Overall | 89.7 | 83.4 | Gemma3n E2B IT | 6.3 |
| Knowledge & honestycounts toward Overall | 83.3 | 96.7 | Gemma 3n E4B IT | 13.4 |
| Long contextcounts toward Overall | 72.2 | 66.7 | Gemma3n E2B IT | 5.5 |
| Multilingualcounts toward Overall | 80.0 | 93.3 | Gemma 3n E4B IT | 13.3 |
| Tool usecounts toward Overall | 87.5 | 95.0 | Gemma 3n E4B IT | 7.5 |
| Safetycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Pushback resistanceshown separately | 91.7 | 91.7 | Level | 0 |
| Famous gotchasshown separately | 53.3 | 73.3 | Gemma 3n E4B IT | 20.0 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| Decode speed, one stream (tok/s) | 13.8 | 12.1 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 3.4 | 4.2 |
| Model size as loaded on our server (GB) | 3.4 | 4.2 |
| Backend | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) |
A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Gemma3n E2B IT did better on 16, Gemma 3n E4B IT did better on 36, and 207 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Gemma3n E2B IT better | Gemma 3n E4B IT better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 0 | 5 | 30 |
| Math | 25 | 0 | 4 | 21 |
| Coding | 30 | 4 | 9 | 17 |
| Instruction following | 30 | 5 | 2 | 23 |
| Knowledge & honesty | 30 | 0 | 4 | 26 |
| Long context | 18 | 3 | 2 | 13 |
| Multilingual | 20 | 1 | 3 | 16 |
| Tool use | 20 | 2 | 3 | 15 |
| Safety | 24 | 0 | 0 | 24 |
| Pushback resistance | 12 | 0 | 0 | 12 |
| Famous gotchas | 15 | 1 | 4 | 10 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| reasoning-004publicall models' answers | fail on its page | pass on its page |
Show both answers to reasoning-004Gemma3n E2B IT The movie starts at 19:45. The movie runs for 2 hours 20 minutes. We need to add 2 hours 20 minutes to 19:45. 19:45 + 2 hours = 21:45 21:45 + 20 minutes = 21:45 + 20 minutes = 21:65 Since there are 60 minutes in an hour, we can write 65 minutes as 1 hour and 5 minutes. So, 21:65 = 21:00 + 1 hour + 5 minutes = 22:05. Alternatively, we can add 2 hours and 20 minutes to 19:45: 19:45 + 2 hours = 21:45 21:45 + 20 minutes = 21:45 + 20 minutes = 21:65 Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour to 21:45, which gives us 22:45. Then, we add 5 minutes to 22:45, which gives us 22:50. However, we need to consider that 65 minutes is 1 hour and 5 minutes. So, 19:45 + 2 hours 20 minutes = 19:45 + 2 hours + 20 minutes = 21:45 + 20 minutes = 21:65. Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour to 21:45, which gives us 22:45. Then, we add 5 minutes to 22:45, which gives us 22:50. However, we can also calculate the total minutes: 19:45 is 19 * 60 + 45 = 1140 + 45 = 1185 minutes. Adding 2 hours 20 minutes, we get 1185 + 2 * 60 + 20 = 1185 + 120 + 20 = 1185 + 140 = 1325 minutes. Now, we convert 1325 minutes to hours and minutes: 1325 / 60 = 22 with a remainder of 5. So, 1325 minutes is 22 hours and 5 minutes. Since the movie starts at 19:45, the movie ends at 19:45 + 2 hours 20 minutes = 21:45 + 20 minutes = 21:65. Since 65 minutes is 1 hour and 5 minutes, we add 1 hour to 21:45, which gives us 22:45. Then, we add 5 minutes to 22:45, which gives us 22:50. However, we can also calculate the end time as follows: 19:45 + 2 hours 20 minutes = 19:45 + 2:20 = 21:65. Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour to 21:65, which gives us 22:65. Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour and 5 minutes to 22:65, which gives us 22:65 + 1:5 = 22:70. Since 70 minutes is 1 hour and 10 minutes, we can add 1 hour and 10 minutes to 22:70, which gives us 22:70 + 1:10 = 23:80. Since 80 minutes is 1 hour and 20 minutes, we can add 1 hour and 20 Cut here for length. The model page has the full answer. Gemma 3n E4B IT The movie starts at 19:45.
The movie runs for 2 hours 20 minutes.
We need to add 2 hours 20 minutes to 19:45.
19:45 + 2 hours = 21:45
21:45 + 20 minutes = 21:65
Since there are 60 minutes in an hour, we can rewrite 65 minutes as 1 hour and 5 minutes.
So, 21:65 = 21:00 + 1 hour 5 minutes = 22:05
Alternatively, we can add 2 hours and 20 minutes to 19:45:
19:45 + 2 hours 20 minutes = (19 + 2) hours + (45 + 20) minutes
= 21 hours + 65 minutes
Since 65 minutes = 1 hour 5 minutes, we have
21 hours + 1 hour 5 minutes = 22 hours 5 minutes
So the movie ends at 22:05.
Final Answer: The final answer is $\boxed{22:05}$
| ||
| reasoning-006private | fail on its page | pass on its page |
| reasoning-015private | fail on its page | pass on its page |
| reasoning-022private | fail on its page | pass on its page |
| reasoning-025private | fail on its page | pass on its page |
Math
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| math-008private | fail on its page | pass on its page |
| math-018private | fail on its page | pass on its page |
| math-023private | fail on its page | pass on its page |
| math-024private | fail on its page | pass on its page |
Coding
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| coding-002private | 71% on its page | pass on its page |
| coding-014publicall models' answers | 43% on its page | pass on its page |
Show both answers to coding-014Gemma3n E2B IT ```python
def reverse_words(s: str) -> str:
s = s.strip()
s = ' '.join(s.split())
return s[::-1]
```
Gemma 3n E4B IT ```python
def reverse_words(s: str) -> str:
▁▁▁▁"""Reverses the order of words in a string.
▁▁▁▁Words are separated by whitespace; collapse any run of whitespace
▁▁▁▁between words to a single space and strip leading/trailing whitespace.
▁▁▁▁Examples:
▁▁▁▁reverse_words(" the sky▁▁is blue ") -> "blue is sky the"
▁▁▁▁reverse_words("hello") -> "hello"
▁▁▁▁"""
▁▁▁▁words = s.split()
▁▁▁▁words = [word for word in words if word]
▁▁▁▁words.reverse()
▁▁▁▁return " ".join(words)
```
| ||
| coding-024private | pass on its page | 38% on its page |
Instruction following
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| instruction-008publicall models' answers | pass on its page | 33% on its page |
Show both answers to instruction-008Gemma3n E2B IT {"name": "Elara Vance", "age": 28}
Gemma 3n E4B IT ```json
{
▁▁"name": "Elara Vance",
▁▁"age": 28
}
```
| ||
| instruction-017private | pass on its page | 50% on its page |
| instruction-022private | 67% on its page | pass on its page |
| instruction-024private | pass on its page | 33% on its page |
| instruction-026private | 67% on its page | pass on its page |
| instruction-029private | pass on its page | 50% on its page |
Knowledge & honesty
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| knowledge-007publicall models' answers | fail on its page | pass on its page |
Show both answers to knowledge-007Gemma3n E2B IT Andrew Wiles published the accepted proof of Fermat's Last Theorem in 1995. While he presented the proof in 1994, it was published in the Journal of the Mathematical Society of Japan in 1995. ANSWER: 1995 Gemma 3n E4B IT Andrew Wiles published the accepted proof of Fermat's Last Theorem in 1995. ANSWER: Andrew Wiles | ||
| knowledge-010private | fail on its page | pass on its page |
| knowledge-015private | fail on its page | pass on its page |
| knowledge-020private | fail on its page | pass on its page |
Long context
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| long_context-005private | fail on its page | pass on its page |
| long_context-006private | pass on its page | fail on its page |
| long_context-008private | pass on its page | fail on its page |
| long_context-010private | fail on its page | pass on its page |
| long_context-016private | pass on its page | fail on its page |
Multilingual
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| multilingual-003publicall models' answers | fail on its page | pass on its page |
Show both answers to multilingual-003Gemma3n E2B IT Let's calculate the distance Anna travels in the first part of her ride. Time = 2 hours Speed = 18 km/h Distance = Speed * Time = 18 km/h * 2 h = 36 km Next, she rides for 90 minutes with a speed of 12 km/h. Time = 90 minutes = 1.5 hours Speed = 12 km/h Distance = Speed * Time = 12 km/h * 1.5 h = 18 km Total distance traveled so far = 36 km + 18 km = 54 km She then pushes her bicycle 3 km. This doesn't add to the distance she has traveled by bike. The total distance of the route is 100 km. Distance remaining = Total distance - Distance traveled = 100 km - 54 km = 46 km Therefore, Anna still has 46 km to go. Final Answer: The final answer is 46 Gemma 3n E4B IT First, let's calculate the distance Anna travels during the first 2 hours. Distance = Speed × Time Distance = 18 km/h × 2 h = 36 km Next, let's calculate the distance Anna travels during the next 90 minutes. 90 minutes = 1.5 hours Distance = Speed × Time Distance = 12 km/h × 1.5 h = 18 km Now, let's calculate the total distance Anna has traveled so far. Total distance = 36 km + 18 km = 54 km Anna then pushes her bike for 3 km. Total distance traveled = 54 km + 3 km = 57 km The total distance of the route is 100 km. Remaining distance = Total distance - Distance traveled Remaining distance = 100 km - 57 km = 43 km Therefore, Anna has 43 km left to travel to reach her destination. Final Answer: The final answer is 43 | ||
| multilingual-004private | fail on its page | pass on its page |
| multilingual-008private | fail on its page | pass on its page |
| multilingual-020private | pass on its page | 67% on its page |
Tool use
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| tool_use-003publicall models' answers | pass on its page | 50% on its page |
Show both answers to tool_use-003Gemma3n E2B IT {"name": "create_calendar_event", "arguments": {"title": "Quarterly planning", "date": "2026-11-03", "start_time": "14:30", "duration_minutes": 90}}
Gemma 3n E4B IT {"name": "create_calendar_event", "arguments": {"title": "Quaertlry planning", "date": "2026-11-03", "start_time": "14:30", "duration_minutes": 90}}
| ||
| tool_use-004private | 50% on its page | pass on its page |
| tool_use-006private | pass on its page | 50% on its page |
| tool_use-018publicall models' answers | fail on its page | pass on its page |
Show both answers to tool_use-018Gemma3n E2B IT I can't directly answer that question. I need to search my files for information about France.
```json
{"name": "search_files", "arguments": {"query": "capital of France", "file_type": "any"}}
```
Gemma 3n E4B IT The capital of France is Paris. | ||
| tool_use-020private | fail on its page | pass on its page |
Famous gotchas
| Question | Gemma3n E2B IT | Gemma 3n E4B IT |
|---|---|---|
| gotchas-001publicall models' answers | fail on its page | pass on its page |
Show both answers to gotchas-001Gemma3n E2B IT The word "strawberry" has 2 'r's. ANSWER: 2 Gemma 3n E4B IT The word "strawberry" has the following letters: s, t, r, a, w, b, e, r, r, y. The letter 'r' appears 3 times in the word "strawberry". ANSWER: 3 | ||
| gotchas-002private | fail on its page | pass on its page |
| gotchas-009private | pass on its page | fail on its page |
| gotchas-012private | fail on its page | pass on its page |
| gotchas-015private | fail on its page | pass on its page |
The same numbers as JSON: compare.json. Answers appear there only for public questions.