Gemma3n E2B IT against Gemma 3n E4B IT

Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.

The short version

Same model line in two sizes: Gemma3n E2B IT has 5 billion parameters and Gemma 3n E4B IT has 8 billion parameters.

Gemma 3n E4B IT scores higher overall: 86.1 against 79.9 out of 100, a gap of 6.2 points.

The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.

Across the 9 categories that count toward Overall, Gemma3n E2B IT is higher in 2, Gemma 3n E4B IT is higher in 6 and 1 is level.

The widest gap is in Math: 76.0 for Gemma3n E2B IT against 92.0 for Gemma 3n E4B IT, in favour of Gemma 3n E4B IT.

Of the 259 questions both models have a definite score for, Gemma3n E2B IT did better on 16, Gemma 3n E4B IT did better on 36, and 207 were level.

The two were tested in different runs, so their hard-question scores are not compared.

Their speeds were not measured on one shared backend and Ollama version, so they are not compared.

Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.

Overall

Gemma 3n E4B IT scores higher: 86.1 against 79.9, a gap of 6.2 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?

Overall (0–100) with its 95% interval, the rank range among all listed ranked models, and the hard-question score.
MeasureGemma3n E2B ITGemma 3n E4B IT
Overall79.975.2–84.786.182.4–89.6
Rank range64–12744–97
Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over in the model's own run answered correctly55.750.7
Size class4–9B4–9B

By category

Programmatic mean (0–100) per category. Higher is the model with the larger mean; a gap that rounds to 0.0 is called level. A dash means not scored, which is different from 0.
CategoryGemma3n E2B ITGemma 3n E4B ITHigherDifference
Reasoningcounts toward Overall 68.6 82.9 Gemma 3n E4B IT 14.3
Mathcounts toward Overall 76.0 92.0 Gemma 3n E4B IT 16.0
Codingcounts toward Overall 61.4 65.3 Gemma 3n E4B IT 3.9
Instruction followingcounts toward Overall 89.7 83.4 Gemma3n E2B IT 6.3
Knowledge & honestycounts toward Overall 83.3 96.7 Gemma 3n E4B IT 13.4
Long contextcounts toward Overall 72.2 66.7 Gemma3n E2B IT 5.5
Multilingualcounts toward Overall 80.0 93.3 Gemma 3n E4B IT 13.3
Tool usecounts toward Overall 87.5 95.0 Gemma 3n E4B IT 7.5
Safetycounts toward Overall 100.0 100.0 Level 0
Pushback resistanceshown separately 91.7 91.7 Level 0
Famous gotchasshown separately 53.3 73.3 Gemma 3n E4B IT 20.0
Writingshown separately — — — —
Self-descriptionshown separately — — — —

Speed and memory

Measured on one laptop. Decode speed is generation speed with one request at a time.
MeasureGemma3n E2B ITGemma 3n E4B IT
Decode speed, one stream (tok/s)13.812.1
GPU memory on our server (1 to 8 answers at a time) (GB)3.44.2
Model size as loaded on our server (GB)3.44.2
BackendROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots)ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)

A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.

Where they differ

Both models have a definite score for 259 questions. Gemma3n E2B IT did better on 16, Gemma 3n E4B IT did better on 36, and 207 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.

Questions both models have a score for, by category.
CategoryQuestionsGemma3n E2B IT betterGemma 3n E4B IT betterLevel
Reasoning350530
Math250421
Coding304917
Instruction following305223
Knowledge & honesty300426
Long context183213
Multilingual201316
Tool use202315
Safety240024
Pushback resistance120012
Famous gotchas151410

Questions where only one got full marks

Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.

Reasoning

Reasoning: 5 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
reasoning-004publicall models' answers fail on its page pass on its page
Show both answers to reasoning-004

Gemma3n E2B IT

The movie starts at 19:45.
The movie runs for 2 hours 20 minutes.
We need to add 2 hours 20 minutes to 19:45.

19:45 + 2 hours = 21:45
21:45 + 20 minutes = 21:45 + 20 minutes = 21:65
Since there are 60 minutes in an hour, we can write 65 minutes as 1 hour and 5 minutes.
So, 21:65 = 21:00 + 1 hour + 5 minutes = 22:05.

Alternatively, we can add 2 hours and 20 minutes to 19:45:
19:45 + 2 hours = 21:45
21:45 + 20 minutes = 21:45 + 20 minutes = 21:65
Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour to 21:45, which gives us 22:45.
Then, we add 5 minutes to 22:45, which gives us 22:50.
However, we need to consider that 65 minutes is 1 hour and 5 minutes.
So, 19:45 + 2 hours 20 minutes = 19:45 + 2 hours + 20 minutes = 21:45 + 20 minutes = 21:65.
Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour to 21:45, which gives us 22:45.
Then, we add 5 minutes to 22:45, which gives us 22:50.
However, we can also calculate the total minutes:
19:45 is 19 * 60 + 45 = 1140 + 45 = 1185 minutes.
Adding 2 hours 20 minutes, we get 1185 + 2 * 60 + 20 = 1185 + 120 + 20 = 1185 + 140 = 1325 minutes.
Now, we convert 1325 minutes to hours and minutes:
1325 / 60 = 22 with a remainder of 5.
So, 1325 minutes is 22 hours and 5 minutes.
Since the movie starts at 19:45, the movie ends at 19:45 + 2 hours 20 minutes = 21:45 + 20 minutes = 21:65.
Since 65 minutes is 1 hour and 5 minutes, we add 1 hour to 21:45, which gives us 22:45.
Then, we add 5 minutes to 22:45, which gives us 22:50.
However, we can also calculate the end time as follows:
19:45 + 2 hours 20 minutes = 19:45 + 2:20 = 21:65.
Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour to 21:65, which gives us 22:65.
Since 65 minutes is 1 hour and 5 minutes, we can add 1 hour and 5 minutes to 22:65, which gives us 22:65 + 1:5 = 22:70.
Since 70 minutes is 1 hour and 10 minutes, we can add 1 hour and 10 minutes to 22:70, which gives us 22:70 + 1:10 = 23:80.
Since 80 minutes is 1 hour and 20 minutes, we can add 1 hour and 20

Cut here for length. The model page has the full answer.

Gemma 3n E4B IT

The movie starts at 19:45.
The movie runs for 2 hours 20 minutes.
We need to add 2 hours 20 minutes to 19:45.
19:45 + 2 hours = 21:45
21:45 + 20 minutes = 21:65
Since there are 60 minutes in an hour, we can rewrite 65 minutes as 1 hour and 5 minutes.
So, 21:65 = 21:00 + 1 hour 5 minutes = 22:05
Alternatively, we can add 2 hours and 20 minutes to 19:45:
19:45 + 2 hours 20 minutes = (19 + 2) hours + (45 + 20) minutes
= 21 hours + 65 minutes
Since 65 minutes = 1 hour 5 minutes, we have
21 hours + 1 hour 5 minutes = 22 hours 5 minutes
So the movie ends at 22:05.

Final Answer: The final answer is $\boxed{22:05}$
reasoning-006private fail on its page pass on its page
reasoning-015private fail on its page pass on its page
reasoning-022private fail on its page pass on its page
reasoning-025private fail on its page pass on its page

Math

Math: 4 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
math-008private fail on its page pass on its page
math-018private fail on its page pass on its page
math-023private fail on its page pass on its page
math-024private fail on its page pass on its page

Coding

Coding: 3 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
coding-002private 71% on its page pass on its page
coding-014publicall models' answers 43% on its page pass on its page
Show both answers to coding-014

Gemma3n E2B IT

```python
def reverse_words(s: str) -> str:
    s = s.strip()
    s = ' '.join(s.split())
    return s[::-1]
```

Gemma 3n E4B IT

```python
def reverse_words(s: str) -> str:
▁▁▁▁"""Reverses the order of words in a string.

▁▁▁▁Words are separated by whitespace; collapse any run of whitespace
▁▁▁▁between words to a single space and strip leading/trailing whitespace.

▁▁▁▁Examples:
▁▁▁▁reverse_words(" the sky▁▁is blue ") -> "blue is sky the"
▁▁▁▁reverse_words("hello") -> "hello"
▁▁▁▁"""
▁▁▁▁words = s.split()
▁▁▁▁words = [word for word in words if word]
▁▁▁▁words.reverse()
▁▁▁▁return " ".join(words)
```
coding-024private pass on its page 38% on its page

Instruction following

Instruction following: 6 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
instruction-008publicall models' answers pass on its page 33% on its page
Show both answers to instruction-008

Gemma3n E2B IT

{"name": "Elara Vance", "age": 28}

Gemma 3n E4B IT

```json
{
▁▁"name": "Elara Vance",
▁▁"age": 28
}
```
instruction-017private pass on its page 50% on its page
instruction-022private 67% on its page pass on its page
instruction-024private pass on its page 33% on its page
instruction-026private 67% on its page pass on its page
instruction-029private pass on its page 50% on its page

Knowledge & honesty

Knowledge & honesty: 4 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
knowledge-007publicall models' answers fail on its page pass on its page
Show both answers to knowledge-007

Gemma3n E2B IT

Andrew Wiles published the accepted proof of Fermat's Last Theorem in 1995. While he presented the proof in 1994, it was published in the Journal of the Mathematical Society of Japan in 1995.

ANSWER: 1995

Gemma 3n E4B IT

Andrew Wiles published the accepted proof of Fermat's Last Theorem in 1995.

ANSWER: Andrew Wiles
knowledge-010private fail on its page pass on its page
knowledge-015private fail on its page pass on its page
knowledge-020private fail on its page pass on its page

Long context

Long context: 5 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
long_context-005private fail on its page pass on its page
long_context-006private pass on its page fail on its page
long_context-008private pass on its page fail on its page
long_context-010private fail on its page pass on its page
long_context-016private pass on its page fail on its page

Multilingual

Multilingual: 4 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
multilingual-003publicall models' answers fail on its page pass on its page
Show both answers to multilingual-003

Gemma3n E2B IT

Let's calculate the distance Anna travels in the first part of her ride.
Time = 2 hours
Speed = 18 km/h
Distance = Speed * Time = 18 km/h * 2 h = 36 km

Next, she rides for 90 minutes with a speed of 12 km/h.
Time = 90 minutes = 1.5 hours
Speed = 12 km/h
Distance = Speed * Time = 12 km/h * 1.5 h = 18 km

Total distance traveled so far = 36 km + 18 km = 54 km

She then pushes her bicycle 3 km. This doesn't add to the distance she has traveled by bike.

The total distance of the route is 100 km.
Distance remaining = Total distance - Distance traveled = 100 km - 54 km = 46 km

Therefore, Anna still has 46 km to go.

Final Answer: The final answer is 46

Gemma 3n E4B IT

First, let's calculate the distance Anna travels during the first 2 hours.
Distance = Speed × Time
Distance = 18 km/h × 2 h = 36 km

Next, let's calculate the distance Anna travels during the next 90 minutes.
90 minutes = 1.5 hours
Distance = Speed × Time
Distance = 12 km/h × 1.5 h = 18 km

Now, let's calculate the total distance Anna has traveled so far.
Total distance = 36 km + 18 km = 54 km

Anna then pushes her bike for 3 km.
Total distance traveled = 54 km + 3 km = 57 km

The total distance of the route is 100 km.
Remaining distance = Total distance - Distance traveled
Remaining distance = 100 km - 57 km = 43 km

Therefore, Anna has 43 km left to travel to reach her destination.

Final Answer: The final answer is 43
multilingual-004private fail on its page pass on its page
multilingual-008private fail on its page pass on its page
multilingual-020private pass on its page 67% on its page

Tool use

Tool use: 5 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
tool_use-003publicall models' answers pass on its page 50% on its page
Show both answers to tool_use-003

Gemma3n E2B IT

{"name": "create_calendar_event", "arguments": {"title": "Quarterly planning", "date": "2026-11-03", "start_time": "14:30", "duration_minutes": 90}}

Gemma 3n E4B IT

{"name": "create_calendar_event", "arguments": {"title": "Quaertlry planning", "date": "2026-11-03", "start_time": "14:30", "duration_minutes": 90}}
tool_use-004private 50% on its page pass on its page
tool_use-006private pass on its page 50% on its page
tool_use-018publicall models' answers fail on its page pass on its page
Show both answers to tool_use-018

Gemma3n E2B IT

I can't directly answer that question. I need to search my files for information about France.

```json
{"name": "search_files", "arguments": {"query": "capital of France", "file_type": "any"}}
```

Gemma 3n E4B IT

The capital of France is Paris.
tool_use-020private fail on its page pass on its page

Famous gotchas

Famous gotchas: 5 questions where exactly one model got full marks.
QuestionGemma3n E2B ITGemma 3n E4B IT
gotchas-001publicall models' answers fail on its page pass on its page
Show both answers to gotchas-001

Gemma3n E2B IT

The word "strawberry" has 2 'r's.

ANSWER: 2

Gemma 3n E4B IT

The word "strawberry" has the following letters: s, t, r, a, w, b, e, r, r, y.
The letter 'r' appears 3 times in the word "strawberry".

ANSWER: 3
gotchas-002private fail on its page pass on its page
gotchas-009private pass on its page fail on its page
gotchas-012private fail on its page pass on its page
gotchas-015private fail on its page pass on its page

The same numbers as JSON: compare.json. Answers appear there only for public questions.