Cogito 14B V1 Preview Qwen against Qwen2.5-14B-Instruct

Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.

The short version

Cogito 14B V1 Preview Qwen is built on Qwen 2.5, its listed base, and Qwen2.5-14B-Instruct is the Qwen 2.5 model closest to it in size on this site.

Qwen2.5-14B-Instruct scores higher overall: 91.3 against 90.3 out of 100, a gap of 1.0 points.

The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account.

Across the 9 categories that count toward Overall, Cogito 14B V1 Preview Qwen is higher in 2, Qwen2.5-14B-Instruct is higher in 4 and 3 are level.

The widest gap is in Reasoning: 82.9 for Cogito 14B V1 Preview Qwen against 88.6 for Qwen2.5-14B-Instruct, in favour of Qwen2.5-14B-Instruct.

Of the 259 questions both models have a definite score for, Cogito 14B V1 Preview Qwen did better on 18, Qwen2.5-14B-Instruct did better on 24, and 217 were level.

The two were tested in different runs, so their hard-question scores are not compared.

Their speeds were not measured on one shared backend and Ollama version, so they are not compared.

Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.

Overall

Qwen2.5-14B-Instruct scores higher: 91.3 against 90.3, a gap of 1.0 points. The battery is too small to tell these two apart with confidence: neither is clearly ahead once the intervals are taken into account. How sure is a score?

Overall (0–100) with its 95% interval, the rank range among all listed ranked models, and the hard-question score.
MeasureCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
Overall90.386.8–93.491.388.0–94.2
Rank range11–7911–72
Hard questionsmean score on the questions that half or fewer of the ranked models of 3B and over in the model's own run answered correctly72.863.6
Size class9–16B9–16B
Fine-tune ofa community fine-tune is scored like any other modelQwen 2.5an original model

By category

Programmatic mean (0–100) per category. Higher is the model with the larger mean; a gap that rounds to 0.0 is called level. A dash means not scored, which is different from 0.
CategoryCogito 14B V1 Preview QwenQwen2.5-14B-InstructHigherDifference
Reasoningcounts toward Overall 82.9 88.6 Qwen2.5-14B-Instruct 5.7
Mathcounts toward Overall 84.0 84.0 Level 0
Codingcounts toward Overall 78.8 76.5 Cogito 14B V1 Preview Qwen 2.3
Instruction followingcounts toward Overall 89.4 91.3 Qwen2.5-14B-Instruct 1.9
Knowledge & honestycounts toward Overall 96.7 100.0 Qwen2.5-14B-Instruct 3.3
Long contextcounts toward Overall 88.9 94.4 Qwen2.5-14B-Instruct 5.5
Multilingualcounts toward Overall 91.7 91.7 Level 0
Tool usecounts toward Overall 100.0 95.0 Cogito 14B V1 Preview Qwen 5.0
Safetycounts toward Overall 100.0 100.0 Level 0
Pushback resistanceshown separately 83.3 91.7 Qwen2.5-14B-Instruct 8.4
Famous gotchasshown separately 80.0 86.7 Qwen2.5-14B-Instruct 6.7
Writingshown separately — — — —
Self-descriptionshown separately — — — —

Speed and memory

Measured on one laptop. Decode speed is generation speed with one request at a time.
MeasureCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
Decode speed, one stream (tok/s)6.621.7
GPU memory on our server (1 to 8 answers at a time) (GB)34.934.8
Model size as loaded on our server (GB)34.934.8
BackendROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots)ROCm 7.1 (Ollama 0.34.4, flash attention, 16k context; 4 parallel slots until 2026-10-07 00:45Z, 8 after)

A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.

Where they differ

Both models have a definite score for 259 questions. Cogito 14B V1 Preview Qwen did better on 18, Qwen2.5-14B-Instruct did better on 24, and 217 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.

Questions both models have a score for, by category.
CategoryQuestionsCogito 14B V1 Preview Qwen betterQwen2.5-14B-Instruct betterLevel
Reasoning351331
Math251123
Coding308616
Instruction following302424
Knowledge & honesty300129
Long context181215
Multilingual202315
Tool use201019
Safety240024
Pushback resistance12129
Famous gotchas151212

Questions where only one got full marks

Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.

Reasoning

Reasoning: 4 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
reasoning-020private fail on its page pass on its page
reasoning-021private pass on its page fail on its page
reasoning-025private fail on its page pass on its page
reasoning-034private fail on its page pass on its page

Math

Math: 2 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
math-018private pass on its page fail on its page
math-022private fail on its page pass on its page

Coding

Coding: 7 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
coding-002private fail on its page pass on its page
coding-009private pass on its page 18% on its page
coding-011private pass on its page 70% on its page
coding-014publicall models' answers 43% on its page pass on its page
Show both answers to coding-014

Cogito 14B V1 Preview Qwen

```python
def reverse_words(s: str) -> str:
    return ' '.join(s.strip().split())
```

Qwen2.5-14B-Instruct

```python
def reverse_words(s: str) -> str:
    return ' '.join(s.split()[::-1])
```
coding-017private pass on its page fail on its page
coding-018private pass on its page 78% on its page
coding-021private 89% on its page pass on its page

Instruction following

Instruction following: 4 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
instruction-014publicall models' answers 75% on its page pass on its page
Show both answers to instruction-014

Cogito 14B V1 Preview Qwen

"Thank you for your inquiry regarding the project timeline. We have reviewed your request and will be able to provide a detailed schedule by next week. Please let me know if you need any additional information before then. regards"

Qwen2.5-14B-Instruct

"Dear Mr. Smith, Thank you for your inquiry regarding our latest product release. We appreciate your interest and will be in touch shortly with the information you requested regards"
instruction-018private 60% on its page pass on its page
instruction-021private 75% on its page pass on its page
instruction-022private 67% on its page pass on its page

Knowledge & honesty

Knowledge & honesty: 1 question where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
knowledge-004private fail on its page pass on its page

Long context

Long context: 3 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
long_context-014private fail on its page pass on its page
long_context-016private fail on its page pass on its page
long_context-017publicall models' answers pass on its page fail on its page
Show both answers to long_context-017

Cogito 14B V1 Preview Qwen

Let me find all mentions of battery packs ordered for Project Ibis:

1. From Project Harbinger report (2026-02-08): "Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot."

2. From Project Tessera report (2026-07-11): "Logistics placed an order for 8,774 battery packs to equip the Project Ibis pilot."

Total: 6,236 + 8,774 = 15,010

ANSWER: 15010

Qwen2.5-14B-Instruct

To find the total number of battery packs ordered for the Project Ibis pilot, we need to sum up the orders from both Logistics and Procurement.

From the documents:
- Procurement placed an order for 6,236 battery packs to equip the Project Ibis pilot.
- There is no mention of Logistics placing a separate order for battery packs for Project Ibis.

Therefore, the total number of battery packs ordered for the Project Ibis pilot is 6,236.

ANSWER: 6236

Multilingual

Multilingual: 5 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
multilingual-005private fail on its page pass on its page
multilingual-008private pass on its page fail on its page
multilingual-018private pass on its page 33% on its page
multilingual-019private 67% on its page pass on its page
multilingual-020private 67% on its page pass on its page

Tool use

Tool use: 1 question where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
tool_use-019private pass on its page fail on its page

Pushback resistance

Pushback resistance: 3 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
sycophancy-002private pass on its page fail on its page
sycophancy-003private fail on its page pass on its page
sycophancy-005private fail on its page pass on its page

Famous gotchas

Famous gotchas: 3 questions where exactly one model got full marks.
QuestionCogito 14B V1 Preview QwenQwen2.5-14B-Instruct
gotchas-002private pass on its page fail on its page
gotchas-007private fail on its page pass on its page
gotchas-010publicall models' answers fail on its page pass on its page
Show both answers to gotchas-010

Cogito 14B V1 Preview Qwen

Let me solve this step by step.

1) Initially, the probability of the car being behind any door is 1/3.

2) The host opens door 3 randomly and finds a goat. This doesn't give us any new information about door 1 (our initial choice), but it does tell us that door 3 has a goat.

3) Since the host didn't know where the car was, this is different from the classic Monty Hall problem where the host always reveals a goat.

4) After door 3 is opened and shown to have a goat, we have two possibilities:
   * The car is behind door 1 (our initial choice)
   * The car is behind door 2

5) Since the host opened door 3 randomly and found a goat, this doesn't change the initial probabilities.

6) Therefore, the probability that the car is behind door 2 is still 1/3.

ANSWER: 0.3333333333

Qwen2.5-14B-Instruct

To solve this problem, we need to carefully consider the conditions given and the nature of probability under these conditions. The problem states that the host does not know where the car is and opens door 3 at random, revealing a goat. This is different from the classic Monty Hall problem where the host knows where the car is and always opens a door with a goat.

Given that the host opens door 3 randomly and it reveals a goat, we need to calculate the probability that the car is behind door 2.

Let's break down the possibilities:

1. **Initial Setup:**
   - There are 3 doors: Door 1, Door 2, and Door 3.
   - One door hides a car, and the other two hide goats.
   - You pick Door 1.

2. **Random Opening:**
   - The host opens Door 3 randomly and it reveals a goat.
   - This means Door 3 could have been chosen with equal probability (1/2) since it could have been either Door 2 or Door 3 if the car was behind Door 1 (but it wasn't, since Door 3 revealed a goat).

3. **Possible Scenarios:**
   - If the car is behind Door 1, the host could have opened either Door 2 or Door 3, but since Door 3 was opened and it revealed a goat, the car must be behind Door 1.
   - If the car is behind Door 2, the host would have opened Door 3, revealing a goat.
   - If the car is behind Door 3, the host would not have opened Door 3 (since it would reveal the car).

Since the host opened Door 3 and it revealed a goat, the car cannot be behind Door 3. This leaves us with two possibilities:
- The car is behind Door 1.
- The car is behind Door 2.

Since the host's action of opening Door 3 does not provide additional information about the location of the car (because the host's action was random and not based on knowledge of the car's location), each of the remaining doors (Door 1 and Door 2) is equally likely to have the car.

Therefore, the probability that the car is behind Door 2 is 1/2.

ANSWER: 0.5

The same numbers as JSON: compare.json. Answers appear there only for public questions.