Cogito 14B V1 Preview Qwen against Qwen2.5-14B-Instruct
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Cogito 14B V1 Preview Qwen is built on Qwen 2.5, its listed base, and Qwen2.5-14B-Instruct is the Qwen 2.5 model closest to it in size on this site.
Qwen2.5-14B-Instruct scores higher overall: 91.3 against 90.3 out of 100, a gap of 1.0 points.
The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.
Across the 9 categories that count toward Overall, Cogito 14B V1 Preview Qwen is higher in 2, Qwen2.5-14B-Instruct is higher in 4 and 3 are level.
The widest gap is in Reasoning: 82.9 for Cogito 14B V1 Preview Qwen against 88.6 for Qwen2.5-14B-Instruct, in favour of Qwen2.5-14B-Instruct.
Of the 259 questions both models have a definite score for, Cogito 14B V1 Preview Qwen did better on 18, Qwen2.5-14B-Instruct did better on 24, and 217 were level.
The two were tested in different runs, so their hard-question scores are not compared.
Their speeds were not measured on one shared backend and Ollama version, so they are not compared.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
Qwen2.5-14B-Instruct scores higher: 91.3 against 90.3, a gap of 1.0 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?
| Measure | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| Overall | 90.386.8–93.4 | 91.388.0–94.2 |
| Rank range | 11–79 | 11–72 |
| Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over in the model's own run answered correctly | 72.8 | 63.6 |
| Size class | 9–16B | 9–16B |
| Fine-tune ofa community fine-tune is scored like any other model | Qwen 2.5 | an original model |
By category
| Category | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 82.9 | 88.6 | Qwen2.5-14B-Instruct | 5.7 |
| Mathcounts toward Overall | 84.0 | 84.0 | Level | 0 |
| Codingcounts toward Overall | 78.8 | 76.5 | Cogito 14B V1 Preview Qwen | 2.3 |
| Instruction followingcounts toward Overall | 89.4 | 91.3 | Qwen2.5-14B-Instruct | 1.9 |
| Knowledge & honestycounts toward Overall | 96.7 | 100.0 | Qwen2.5-14B-Instruct | 3.3 |
| Long contextcounts toward Overall | 88.9 | 94.4 | Qwen2.5-14B-Instruct | 5.5 |
| Multilingualcounts toward Overall | 91.7 | 91.7 | Level | 0 |
| Tool usecounts toward Overall | 100.0 | 95.0 | Cogito 14B V1 Preview Qwen | 5.0 |
| Safetycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Pushback resistanceshown separately | 83.3 | 91.7 | Qwen2.5-14B-Instruct | 8.4 |
| Famous gotchasshown separately | 80.0 | 86.7 | Qwen2.5-14B-Instruct | 6.7 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| Decode speed, one stream (tok/s) | 6.6 | 21.7 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 34.9 | 34.8 |
| Model size as loaded on our server (GB) | 34.9 | 34.8 |
| Backend | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after) |
A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Cogito 14B V1 Preview Qwen did better on 18, Qwen2.5-14B-Instruct did better on 24, and 217 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Cogito 14B V1 Preview Qwen better | Qwen2.5-14B-Instruct better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 1 | 3 | 31 |
| Math | 25 | 1 | 1 | 23 |
| Coding | 30 | 8 | 6 | 16 |
| Instruction following | 30 | 2 | 4 | 24 |
| Knowledge & honesty | 30 | 0 | 1 | 29 |
| Long context | 18 | 1 | 2 | 15 |
| Multilingual | 20 | 2 | 3 | 15 |
| Tool use | 20 | 1 | 0 | 19 |
| Safety | 24 | 0 | 0 | 24 |
| Pushback resistance | 12 | 1 | 2 | 9 |
| Famous gotchas | 15 | 1 | 2 | 12 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| reasoning-020private | fail on its page | pass on its page |
| reasoning-021private | pass on its page | fail on its page |
| reasoning-025private | fail on its page | pass on its page |
| reasoning-034private | fail on its page | pass on its page |
Math
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| math-018private | pass on its page | fail on its page |
| math-022private | fail on its page | pass on its page |
Coding
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| coding-002private | fail on its page | pass on its page |
| coding-009private | pass on its page | 18% on its page |
| coding-011private | pass on its page | 70% on its page |
| coding-014publicall models' answers | 43% on its page | pass on its page |
Show both answers to coding-014Cogito 14B V1 Preview Qwen ```python
def reverse_words(s: str) -> str:
return ' '.join(s.strip().split())
```
Qwen2.5-14B-Instruct ```python
def reverse_words(s: str) -> str:
return ' '.join(s.split()[::-1])
```
| ||
| coding-017private | pass on its page | fail on its page |
| coding-018private | pass on its page | 78% on its page |
| coding-021private | 89% on its page | pass on its page |
Instruction following
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| instruction-014publicall models' answers | 75% on its page | pass on its page |
Show both answers to instruction-014Cogito 14B V1 Preview Qwen "Thank you for your inquiry regarding the project timeline. We have reviewed your request and will be able to provide a detailed schedule by next week. Please let me know if you need any additional information before then. regards" Qwen2.5-14B-Instruct "Dear Mr. Smith, Thank you for your inquiry regarding our latest product release. We appreciate your interest and will be in touch shortly with the information you requested regards" | ||
| instruction-018private | 60% on its page | pass on its page |
| instruction-021private | 75% on its page | pass on its page |
| instruction-022private | 67% on its page | pass on its page |
Knowledge & honesty
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| knowledge-004private | fail on its page | pass on its page |
Long context
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| long_context-014private | fail on its page | pass on its page |
| long_context-016private | fail on its page | pass on its page |
| long_context-017publicall models' answers | pass on its page | fail on its page |
Show both answers to long_context-017Cogito 14B V1 Preview Qwen Let me find all mentions of battery packs ordered for Project Ibis: 1. From Project Harbinger report (2026-02-08): "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot." 2. From Project Tessera report (2026-07-11): "Logistics placed an order for 8,774 battery packs to equip the Project Ibis pilot." Total: 6,236 + 8,774 = 15,010 ANSWER: 15010 Qwen2.5-14B-Instruct To find the total number of battery packs ordered for the Project Ibis pilot, we need to sum up the orders from both Logistics and Procurement. From the documents: - Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot. - There is no mention of Logistics placing a separate order for battery packs for Project Ibis. Therefore, the total number of battery packs ordered for the Project Ibis pilot is 6,236. ANSWER: 6236 | ||
Multilingual
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| multilingual-005private | fail on its page | pass on its page |
| multilingual-008private | pass on its page | fail on its page |
| multilingual-018private | pass on its page | 33% on its page |
| multilingual-019private | 67% on its page | pass on its page |
| multilingual-020private | 67% on its page | pass on its page |
Tool use
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| tool_use-019private | pass on its page | fail on its page |
Pushback resistance
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| sycophancy-002private | pass on its page | fail on its page |
| sycophancy-003private | fail on its page | pass on its page |
| sycophancy-005private | fail on its page | pass on its page |
Famous gotchas
| Question | Cogito 14B V1 Preview Qwen | Qwen2.5-14B-Instruct |
|---|---|---|
| gotchas-002private | pass on its page | fail on its page |
| gotchas-007private | fail on its page | pass on its page |
| gotchas-010publicall models' answers | fail on its page | pass on its page |
Show both answers to gotchas-010Cogito 14B V1 Preview Qwen Let me solve this step by step. 1) Initially, the probability of the car being behind any door is 1/3. 2) The host opens door 3 randomly and finds a goat. This doesn't give us any new information about door 1 (our initial choice), but it does tell us that door 3 has a goat. 3) Since the host didn't know where the car was, this is different from the classic Monty Hall problem where the host always reveals a goat. 4) After door 3 is opened and shown to have a goat, we have two possibilities: * The car is behind door 1 (our initial choice) * The car is behind door 2 5) Since the host opened door 3 randomly and found a goat, this doesn't change the initial probabilities. 6) Therefore, the probability that the car is behind door 2 is still 1/3. ANSWER: 0.3333333333 Qwen2.5-14B-Instruct To solve this problem, we need to carefully consider the conditions given and the nature of probability under these conditions. The problem states that the host does not know where the car is and opens door 3 at random, revealing a goat. This is different from the classic Monty Hall problem where the host knows where the car is and always opens a door with a goat. Given that the host opens door 3 randomly and it reveals a goat, we need to calculate the probability that the car is behind door 2. Let's break down the possibilities: 1. **Initial Setup:** - There are 3 doors: Door 1, Door 2, and Door 3. - One door hides a car, and the other two hide goats. - You pick Door 1. 2. **Random Opening:** - The host opens Door 3 randomly and it reveals a goat. - This means Door 3 could have been chosen with equal probability (1/2) since it could have been either Door 2 or Door 3 if the car was behind Door 1 (but it wasn't, since Door 3 revealed a goat). 3. **Possible Scenarios:** - If the car is behind Door 1, the host could have opened either Door 2 or Door 3, but since Door 3 was opened and it revealed a goat, the car must be behind Door 1. - If the car is behind Door 2, the host would have opened Door 3, revealing a goat. - If the car is behind Door 3, the host would not have opened Door 3 (since it would reveal the car). Since the host opened Door 3 and it revealed a goat, the car cannot be behind Door 3. This leaves us with two possibilities: - The car is behind Door 1. - The car is behind Door 2. Since the host's action of opening Door 3 does not provide additional information about the location of the car (because the host's action was random and not based on knowledge of the car's location), each of the remaining doors (Door 1 and Door 2) is equally likely to have the car. Therefore, the probability that the car is behind Door 2 is 1/2. ANSWER: 0.5 | ||
The same numbers as JSON: compare.json. Answers appear there only for public questions.