Mellum2-12B-A2.5B-Base: benchmark scores, price and ranking
Mellum2-12B-A2.5B-Base from JetBrains ranks #651 of 743 LLMs on the LLMs Tiger aggregate benchmark score (44.0), based on 3 benchmark results captured up to 2026-10-13.
- Organisation
- JetBrains
- Type
- Open-weight / local model
- Open weights
- Yes
- Aggregate rank
- #651 of 743 (score 44.0)
Relative to other models, Mellum2-12B-A2.5B-Base is strongest on MMLU-Pro (#61 of 136) and weakest on GPQA Diamond (#246 of 275).
Across the 3 benchmarks where at least ten models are scored, Mellum2-12B-A2.5B-Base is in the top 10% on 0 and in the top half on 1.
Mellum2-12B-A2.5B-Base benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| MMLU-Pro | 59.3% | #61 of 136 | HuggingFace MMLU-Pro leaderboard | 2026-10-13 |
| GSM8K | 81.7% | #13 of 17 | HuggingFace GSM8K leaderboard | 2026-10-13 |
| GPQA Diamond | 31.3% | #246 of 275 | HuggingFace GPQA Diamond leaderboard | 2026-10-13 |
About Mellum2-12B-A2.5B-Base
Mellum2-12B-A2.5B-Base is an open-weight model that can run on your own hardware from JetBrains. No API price is listed for it yet. We hold 3 benchmark results for it across 2 areas, and none is in the top tenth of its board.
Where Mellum2-12B-A2.5B-Base stands
On our aggregate score, Mellum2-12B-A2.5B-Base is #651 of 743 ranked models, in the lower half. Its score is 44.0; the scale is centred on 50, so it sits 6.0 points below the typical model. That beats 12% of the ranked field. Among the 6 ranked JetBrains models it is #5. The next one up is Mellum2-12B-A2.5B-Instruct-SFT at 46.5. The next one down is Mellum2-12B-A2.5B-Base-Pretrain at 44.0.
Mellum2-12B-A2.5B-Base, benchmark by benchmark
MMLU-Pro puts Mellum2-12B-A2.5B-Base at 59.3%, #61 of 136. That beats 55% of the models tested, but 28.8 points short of Darwin-180B-RSI (88.1%). The results either side are Tencent-Hunyuan-Large (60.2%) and Mellum2-12B-A2.5B-Base-Pretrain (59.3%). It covers 57-subject knowledge + reasoning. The board median is 55.3%, so Mellum2-12B-A2.5B-Base is 4.1 points above it. Source: HuggingFace MMLU-Pro leaderboard, captured 2026-10-13.
Mellum2-12B-A2.5B-Base lands below the middle on GSM8K: 81.7%, #13 of 17, with 71% of the field scoring higher. mimo-v2.5-pro leads at 99.6%; the closest result above is Phi-3-mini-4k-instruct at 85.7%. It covers grade-school math word problems. The board median is 86.9%, so Mellum2-12B-A2.5B-Base is 5.1 points below it. Source: HuggingFace GSM8K leaderboard, captured 2026-10-13.
GPQA Diamond is a weak spot. Mellum2-12B-A2.5B-Base scores 31.3%, #246 of 275, behind 89% of the models measured. The leader, GPT-6 Astra (max), reaches 95.8%. It covers expert-level science questions. The board median is 71.7%, so Mellum2-12B-A2.5B-Base is 40.4 points below it. Source: HuggingFace GPQA Diamond leaderboard, captured 2026-10-13.
Strengths and weaknesses
Reasoning, maths and science: uneven, ahead of 33% of the field on average across 2 benchmarks. Its best board here is MMLU-Pro (#61 of 136) and its weakest is GPQA Diamond (#246 of 275).
Reading these numbers
The 3 scores on this page come from 3 sources: HuggingFace GPQA Diamond leaderboard (1), HuggingFace GSM8K leaderboard (1), HuggingFace MMLU-Pro leaderboard (1). No official leaderboard supplied any of them, so treat them as third-party measurements. With 3 results the aggregate rank can move a lot when one new score arrives.
Models ranked nearby
- #648 Yi-34B-Chat (44.0)
- #649 o3-mini (high) (44.0)
- #650 gemini-1.5-pro-001 (44.0)
- #652 Mellum2-12B-A2.5B-Base-Pretrain (44.0)
- #653 Phi-3-medium-128k-instruct (44.0)
- #654 Yi 6B (44.0)
More from JetBrains
- Mellum2-12B-A2.5B-Thinking (#347, 48.5)
- Mellum2-12B-A2.5B-Instruct (#487, 46.8)
- Mellum2-12B-A2.5B-Thinking-SFT (#499, 46.6)
- Mellum2-12B-A2.5B-Instruct-SFT (#509, 46.5)
- Mellum2-12B-A2.5B-Base-Pretrain (#652, 44.0)
Compare it on the live leaderboard → · How the aggregate score works