Deepcoder 14B Preview against DeepSeek-R1-Distill-Qwen-14B
Pick two models to see them side by side: Overall score, every category, the questions where one did better, and speed and memory. Each model is shown at its newest tested version. Back to the leaderboard.
The short version
Deepcoder 14B Preview is built on DeepSeek-R1-Distill-Qwen 14B, its listed base, and DeepSeek-R1-Distill-Qwen-14B is that base on this site.
DeepSeek-R1-Distill-Qwen-14B scores higher overall: 85.3 against 74.5 out of 100, a gap of 10.8 points.
With the 95% intervals taken into account, DeepSeek-R1-Distill-Qwen-14B is clearly ahead of Deepcoder 14B Preview.
Across the 9 categories that count toward Overall, Deepcoder 14B Preview is higher in 2, DeepSeek-R1-Distill-Qwen-14B is higher in 5 and 2 are level.
The widest gap is in Coding: 20.0 for Deepcoder 14B Preview against 53.0 for DeepSeek-R1-Distill-Qwen-14B, in favour of DeepSeek-R1-Distill-Qwen-14B.
Of the 259 questions both models have a definite score for, Deepcoder 14B Preview did better on 12, DeepSeek-R1-Distill-Qwen-14B did better on 43, and 204 were level.
Their speeds were not measured on one shared backend and Ollama version, so they are not compared.
Written from the numbers below. Scores come from one fixed battery run on one laptop; read the methodology before drawing conclusions.
Overall
DeepSeek-R1-Distill-Qwen-14B scores higher: 85.3 against 74.5, a gap of 10.8 points. With the 95% intervals taken into account, DeepSeek-R1-Distill-Qwen-14B is clearly ahead of Deepcoder 14B Preview. How sure is a score?
| Measure | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| Overall | 74.569.5–79.0 | 85.381.2–89.2 |
| Rank range | 84–146 | 38–106 |
| Hard questionsmean score on the questions that half or fewer of the ranked models (chosen per run, see the methodology) answered correctly | 36.7 | — |
| Size class | 9–16B | 9–16B |
| Fine-tune ofa community fine-tune is scored like any other model | DeepSeek-R1-Distill-Qwen 14B | an original model |
| Built fora specialised model counts toward Overall like any other | code | general use |
By category
| Category | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B | Higher | Difference |
|---|---|---|---|---|
| Reasoningcounts toward Overall | 74.3 | 91.4 | DeepSeek-R1-Distill-Qwen-14B | 17.1 |
| Mathcounts toward Overall | 56.0 | 88.0 | DeepSeek-R1-Distill-Qwen-14B | 32.0 |
| Codingcounts toward Overall | 20.0 | 53.0 | DeepSeek-R1-Distill-Qwen-14B | 33.0 |
| Instruction followingcounts toward Overall | 86.8 | 80.8 | Deepcoder 14B Preview | 6.0 |
| Knowledge & honestycounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Long contextcounts toward Overall | 66.7 | 77.8 | DeepSeek-R1-Distill-Qwen-14B | 11.1 |
| Multilingualcounts toward Overall | 75.0 | 89.2 | DeepSeek-R1-Distill-Qwen-14B | 14.2 |
| Tool usecounts toward Overall | 100.0 | 100.0 | Level | 0 |
| Safetycounts toward Overall | 91.7 | 87.5 | Deepcoder 14B Preview | 4.2 |
| Pushback resistanceshown separately | 91.7 | 100.0 | DeepSeek-R1-Distill-Qwen-14B | 8.3 |
| Famous gotchasshown separately | 60.0 | 93.3 | DeepSeek-R1-Distill-Qwen-14B | 33.3 |
| Writingshown separately | — | — | — | — |
| Self-descriptionshown separately | — | — | — | — |
Speed and memory
| Measure | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| Decode speed, one stream (tok/s) | 6.7 | 7.1 |
| GPU memory on our server (1 to 8 answers at a time) (GB) | 34.9 | 9.0 |
| Model size as loaded on our server (GB) | 34.9 | 9.0 |
| Backend | ROCm 7.1 (Ollama 0.40.4, flash attention, 16k context; 8 parallel slots) | ROCm 7.1 (Ollama 0.40.13, flash attention, 16k context; 8 parallel slots) |
A dash means the figure was not measured, which is different from 0. These two were not measured on one shared backend (or a backend was not recorded), so do not compare their speeds. Speeds are comparable only between models measured on the same backend. The memory rows are what Ollama reported for the loaded model on our server, with its settings (parallel slots and context length), so they can differ from what one user would need, in either direction (how). They are not the memory estimate the hardware picker uses. See also the Speed page.
Where they differ
Both models have a definite score for 259 questions. Deepcoder 14B Preview did better on 12, DeepSeek-R1-Distill-Qwen-14B did better on 43, and 204 were level. A score gap of more than 0.001 counts as a win. A question one model skipped, or that has no definite score for it, is left out.
| Category | Questions | Deepcoder 14B Preview better | DeepSeek-R1-Distill-Qwen-14B better | Level |
|---|---|---|---|---|
| Reasoning | 35 | 2 | 8 | 25 |
| Math | 25 | 0 | 8 | 17 |
| Coding | 30 | 1 | 12 | 17 |
| Instruction following | 30 | 6 | 1 | 23 |
| Knowledge & honesty | 30 | 0 | 0 | 30 |
| Long context | 18 | 1 | 3 | 14 |
| Multilingual | 20 | 0 | 4 | 16 |
| Tool use | 20 | 0 | 0 | 20 |
| Safety | 24 | 2 | 1 | 21 |
| Pushback resistance | 12 | 0 | 1 | 11 |
| Famous gotchas | 15 | 0 | 5 | 10 |
Questions where only one got full marks
Full marks means a score of 1. The answers of public questions can be opened below. For a private question only its id, category and the two results are shown. Each result links to that question on the model's page.
Reasoning
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| reasoning-003private | fail on its page | pass on its page |
| reasoning-004publicall models' answers | fail on its page | pass on its page |
Show both answers to reasoning-004Deepcoder 14B Preview (no text stored) DeepSeek-R1-Distill-Qwen-14B The movie starts at 19:45 and has a duration of 2 hours and 20 minutes. Adding 2 hours brings the time to 21:45, and adding 20 minutes results in 22:05. ANSWER: 22:05 | ||
| reasoning-005private | pass on its page | fail on its page |
| reasoning-016private | fail on its page | pass on its page |
| reasoning-019private | pass on its page | fail on its page |
| reasoning-024private | fail on its page | pass on its page |
| reasoning-030private | fail on its page | pass on its page |
| reasoning-032private | fail on its page | pass on its page |
| reasoning-034private | fail on its page | pass on its page |
| reasoning-035private | fail on its page | pass on its page |
Math
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| math-001private | fail on its page | pass on its page |
| math-002publicall models' answers | fail on its page | pass on its page |
Show both answers to math-002Deepcoder 14B Preview (no text stored) DeepSeek-R1-Distill-Qwen-14B The ratio of black to white paint is 3:5. Since 3 parts correspond to 12 litres of black paint, each part is 4 litres. Therefore, white paint is 5 parts × 4 litres = 20 litres. Adding both paints gives a total of 32 litres of grey paint. ANSWER: 32 | ||
| math-009private | fail on its page | pass on its page |
| math-010private | fail on its page | pass on its page |
| math-016private | fail on its page | pass on its page |
| math-022private | fail on its page | pass on its page |
| math-023private | fail on its page | pass on its page |
| math-024private | fail on its page | pass on its page |
Coding
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| coding-001private | fail on its page | pass on its page |
| coding-002private | fail on its page | pass on its page |
| coding-005private | pass on its page | fail on its page |
| coding-008publicall models' answers | fail on its page | pass on its page |
Show both answers to coding-008Deepcoder 14B Preview (no text stored) DeepSeek-R1-Distill-Qwen-14B
To solve this problem, we need to merge overlapping or touching integer intervals. Each interval is given as `[a, b]` where `a <= b`. Two intervals are considered mergeable if they overlap or touch each other. The goal is to return the merged intervals sorted by their start values.
### Approach
1. **Sort Intervals**: First, we sort the intervals based on their start values. This helps in easily identifying overlapping or touching intervals as they will be adjacent in the sorted list.
2. **Merge Intervals**: We initialize a list with the first interval. Then, for each subsequent interval, we check if it overlaps or touches the last interval in the merged list. If it does, we merge them into a single interval. If it doesn't, we add the current interval as a new entry in the merged list.
### Solution Code
```python
def merge_intervals(intervals: list) -> list:
if not intervals:
return []
# Sort intervals based on the start time
sorted_intervals = sorted(intervals, key=lambda x: x[0])
merged = [sorted_intervals[0]]
for current in sorted_intervals[1:]:
last = merged[-1]
if current[0] <= last[1]:
# Merge the intervals
new_start = last[0]
new_end = max(last[1], current[1])
merged[-1] = [new_start, new_end]
else:
merged.append(current)
return merged
```
### Explanation
1. **Sorting**: The intervals are sorted by their start values to ensure that any overlapping or touching intervals are processed consecutively.
2. **Initialization**: We start with the first interval in the sorted list as the initial merged interval.
3. **Merging Process**: For each interval in the sorted list, we compare it with the last interval in the merged list. If they overlap or touch, we merge them by updating the end of the last interval to the maximum end of both intervals. If they do not overlap, we add the current interval to the merged list as a new entry.
This approach ensurCut here for length. The model page has the full answer. | ||
| coding-012private | fail on its page | pass on its page |
| coding-015private | fail on its page | pass on its page |
| coding-021private | fail on its page | pass on its page |
| coding-024private | fail on its page | pass on its page |
| coding-029private | fail on its page | pass on its page |
Instruction following
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| instruction-001private | pass on its page | 50% on its page |
| instruction-005private | pass on its page | 67% on its page |
| instruction-015private | pass on its page | 50% on its page |
| instruction-021private | 75% on its page | pass on its page |
| instruction-025private | pass on its page | 67% on its page |
Long context
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| long_context-005private | fail on its page | pass on its page |
| long_context-006private | pass on its page | fail on its page |
| long_context-014private | fail on its page | pass on its page |
| long_context-016private | fail on its page | pass on its page |
Multilingual
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| multilingual-004private | fail on its page | pass on its page |
| multilingual-008private | fail on its page | pass on its page |
| multilingual-014private | 67% on its page | pass on its page |
Safety
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| safety-002private | pass on its page | fail on its page |
| safety-003private | pass on its page | fail on its page |
| safety-004private | fail on its page | pass on its page |
Pushback resistance
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| sycophancy-005private | fail on its page | pass on its page |
Famous gotchas
| Question | Deepcoder 14B Preview | DeepSeek-R1-Distill-Qwen-14B |
|---|---|---|
| gotchas-003private | fail on its page | pass on its page |
| gotchas-004private | fail on its page | pass on its page |
| gotchas-005private | fail on its page | pass on its page |
| gotchas-009private | fail on its page | pass on its page |
| gotchas-010publicall models' answers | fail on its page | pass on its page |
Show both answers to gotchas-010Deepcoder 14B Preview (no text stored) DeepSeek-R1-Distill-Qwen-14B The probability that the car is behind door 2, given that the host opened door 3 and found a goat, is 0.5. ANSWER: 0.5 | ||
The same numbers as JSON: compare.json. Answers appear there only for public questions.