Mellum2-12B-A2.5B-Base-Pretrain: benchmark scores, price and ranking
Mellum2-12B-A2.5B-Base-Pretrain from JetBrains ranks #652 of 743 LLMs on the LLMs Tiger aggregate benchmark score (44.0), based on 3 benchmark results captured up to 2026-10-13.
- Organisation
- JetBrains
- Type
- Open-weight / local model
- Open weights
- Yes
- Aggregate rank
- #652 of 743 (score 44.0)
Relative to other models, Mellum2-12B-A2.5B-Base-Pretrain is strongest on MMLU-Pro (#62 of 136) and weakest on GPQA Diamond (#247 of 275).
Across the 3 benchmarks where at least ten models are scored, Mellum2-12B-A2.5B-Base-Pretrain is in the top 10% on 0 and in the top half on 1.
Mellum2-12B-A2.5B-Base-Pretrain benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| MMLU-Pro | 59.3% | #62 of 136 | HuggingFace MMLU-Pro leaderboard | 2026-10-13 |
| GSM8K | 81.7% | #14 of 17 | HuggingFace GSM8K leaderboard | 2026-10-13 |
| GPQA Diamond | 31.3% | #247 of 275 | HuggingFace GPQA Diamond leaderboard | 2026-10-13 |
About Mellum2-12B-A2.5B-Base-Pretrain
Mellum2-12B-A2.5B-Base-Pretrain is an open-weight model that can run on your own hardware from JetBrains. No API price is listed for it yet. We hold 3 benchmark results for it across 2 areas, and none is in the top tenth of its board.
Where Mellum2-12B-A2.5B-Base-Pretrain stands
On our aggregate score, Mellum2-12B-A2.5B-Base-Pretrain is #652 of 743 ranked models, in the lower half. Its score is 44.0; the scale is centred on 50, so it sits 6.0 points below the typical model. That beats 12% of the ranked field. Among the 6 ranked JetBrains models it is #6. The next one up is Mellum2-12B-A2.5B-Base at 44.0.
Mellum2-12B-A2.5B-Base-Pretrain, benchmark by benchmark
MMLU-Pro puts Mellum2-12B-A2.5B-Base-Pretrain at 59.3%, #62 of 136. That beats 54% of the models tested, but 28.8 points short of Darwin-180B-RSI (88.1%). The results either side are Mellum2-12B-A2.5B-Base (59.3%) and EXAONE-3.5-32B-Instruct (58.9%). It covers 57-subject knowledge + reasoning. The board median is 55.3%, so Mellum2-12B-A2.5B-Base-Pretrain is 4.1 points above it. Source: HuggingFace MMLU-Pro leaderboard, captured 2026-10-13.
GSM8K is a weak spot. Mellum2-12B-A2.5B-Base-Pretrain scores 81.7%, #14 of 17, behind 76% of the models measured. The leader, mimo-v2.5-pro, reaches 99.6%. It covers grade-school math word problems. The board median is 86.9%, so Mellum2-12B-A2.5B-Base-Pretrain is 5.1 points below it. Source: HuggingFace GSM8K leaderboard, captured 2026-10-13.
GPQA Diamond is a weak spot. Mellum2-12B-A2.5B-Base-Pretrain scores 31.3%, #247 of 275, behind 89% of the models measured. The leader, GPT-6 Astra (max), reaches 95.8%. It covers expert-level science questions. The board median is 71.7%, so Mellum2-12B-A2.5B-Base-Pretrain is 40.4 points below it. Source: HuggingFace GPQA Diamond leaderboard, captured 2026-10-13.
Strengths and weaknesses
Reasoning, maths and science: uneven, ahead of 32% of the field on average across 2 benchmarks. Its best board here is MMLU-Pro (#62 of 136) and its weakest is GPQA Diamond (#247 of 275).
Reading these numbers
The 3 scores on this page come from 3 sources: HuggingFace GPQA Diamond leaderboard (1), HuggingFace GSM8K leaderboard (1), HuggingFace MMLU-Pro leaderboard (1). No official leaderboard supplied any of them, so treat them as third-party measurements. With 3 results the aggregate rank can move a lot when one new score arrives.
Models ranked nearby
- #649 o3-mini (high) (44.0)
- #650 gemini-1.5-pro-001 (44.0)
- #651 Mellum2-12B-A2.5B-Base (44.0)
- #653 Phi-3-medium-128k-instruct (44.0)
- #654 Yi 6B (44.0)
- #655 Mistral Small 3.2 (44.0)
More from JetBrains
- Mellum2-12B-A2.5B-Thinking (#347, 48.5)
- Mellum2-12B-A2.5B-Instruct (#487, 46.8)
- Mellum2-12B-A2.5B-Thinking-SFT (#499, 46.6)
- Mellum2-12B-A2.5B-Instruct-SFT (#509, 46.5)
- Mellum2-12B-A2.5B-Base (#651, 44.0)
Compare it on the live leaderboard → · How the aggregate score works