Llama-3.1-8B-Instruct: benchmark scores, price and ranking
Llama-3.1-8B-Instruct from Meta AI ranks #743 of 743 LLMs on the LLMs Tiger aggregate benchmark score (36.7), based on 10 benchmark results captured up to 2026-10-13.
- Organisation
- Meta AI
- Type
- Open-weight / local model
- Open weights
- Yes
- Released
- 2024-07-23 (source: epoch)
- Cheapest API price
- $0.020 in / $0.050 out per million tokens (via novita, observed 2026-10-13T19:47Z)
- Aggregate rank
- #743 of 743 (score 36.7)
Relative to other models, Llama-3.1-8B-Instruct is strongest on BALROG (#27 of 34) and weakest on WeirdML (#109 of 109).
Across the 10 benchmarks where at least ten models are scored, Llama-3.1-8B-Instruct is in the top 10% on 0 and in the top half on 0.
Its cheapest listed input price, $0.020 per million tokens, is lower than that of 97% of the 427 models with a listed price.
Llama-3.1-8B-Instruct benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| BALROG | 15.1% | #27 of 34 | Epoch AI Benchmarking Hub | 2026-10-13 |
| Aider Code Editing | 37.6% | #77 of 92 | Aider Aider Code Editing leaderboard | 2026-10-13 |
| CritPt | 0.0% | #100 of 115 | Epoch AI Benchmarking Hub | 2026-10-13 |
| DTBench | 50.9% | #129 of 146 | Epoch AI Benchmarking Hub | 2026-10-13 |
| Chess Puzzles | 0.0% | #114 of 122 | Epoch AI Benchmarking Hub | 2026-10-13 |
| OTIS Mock AIME | 1.7% | #177 of 189 | Epoch AI Benchmarking Hub | 2026-10-13 |
| GPQA Diamond | 27.0% | #260 of 275 | Epoch AI Benchmarking Hub | 2026-10-13 |
| LMCA | 5.4% | #105 of 108 | Epoch AI Benchmarking Hub | 2026-10-13 |
| SciCode | 13.2% | #105 of 106 | Epoch AI Benchmarking Hub | 2026-10-13 |
| WeirdML | 1.7% | #109 of 109 | Epoch AI Benchmarking Hub | 2026-10-13 |
About Llama-3.1-8B-Instruct
Llama-3.1-8B-Instruct is an open-weight model that can run on your own hardware from Meta AI. It was released on 2024-07-23, according to Epoch AI. The cheapest API listing we track charges $0.020 per million input tokens and $0.050 per million output tokens, through novita. We hold 10 benchmark results for it across 4 areas, and none is in the top tenth of its board.
Where Llama-3.1-8B-Instruct stands
On our aggregate score, Llama-3.1-8B-Instruct is #743 of 743 ranked models, in the lower half. Its score is 36.7; the scale is centred on 50, so it sits 13.3 points below the typical model. That beats 0% of the ranked field. Among the 28 ranked Meta AI models it is #28. The next one up is Meta-Llama-3-8B-Instruct at 39.4.
Llama-3.1-8B-Instruct, benchmark by benchmark
Llama-3.1-8B-Instruct lands below the middle on BALROG: 15.1%, #27 of 34, with 76% of the field scoring higher. GPT-6 Astra (max) leads at 68.3%; the closest result above is qwen2.5-72b-instruct at 16.2%. It covers long-horizon game-playing agents. The board median is 27.6%, so Llama-3.1-8B-Instruct is 12.5 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Aider Code Editing is a weak spot. Llama-3.1-8B-Instruct scores 37.6%, #77 of 92, behind 83% of the models measured. The leader, Claude 3.5-Sonnet-20241022, reaches 84.2%. It covers Aider official benchmark. The board median is 55.6%, so Llama-3.1-8B-Instruct is 18.0 points below it. Source: Aider Aider Code Editing leaderboard, captured 2026-10-13.
CritPt is a weak spot. Llama-3.1-8B-Instruct scores 0.0%, #100 of 115, behind 86% of the models measured. The leader, GPT-5.6 Sol (max), reaches 32.3%. It covers unpublished research-level physics problems. The board median is 2.9%, so Llama-3.1-8B-Instruct is 2.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
DTBench is a weak spot. Llama-3.1-8B-Instruct scores 50.9%, #129 of 146, behind 88% of the models measured. The leader, Claude Opus 5.5 (max), reaches 98.9%. It covers dynamic tool-use benchmark. The board median is 68.8%, so Llama-3.1-8B-Instruct is 17.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Chess Puzzles is a weak spot. Llama-3.1-8B-Instruct scores 0.0%, #114 of 122, behind 93% of the models measured. The leader, GPT-6 Astra (max), reaches 72.0%. It covers chess tactics puzzles. The board median is 12.5%, so Llama-3.1-8B-Instruct is 12.5 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
OTIS Mock AIME is a weak spot. Llama-3.1-8B-Instruct scores 1.7%, #177 of 189, behind 93% of the models measured. The leader, Claude Fable 5.1 (max), reaches 100.0%. It covers competition maths (AIME 2024-25 style). The board median is 57.8%, so Llama-3.1-8B-Instruct is 56.1 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
GPQA Diamond is a weak spot. Llama-3.1-8B-Instruct scores 27.0%, #260 of 275, behind 94% of the models measured. The leader, GPT-6 Astra (max), reaches 95.8%. It covers expert-level science questions. The board median is 71.7%, so Llama-3.1-8B-Instruct is 44.8 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
LMCA is a weak spot. Llama-3.1-8B-Instruct scores 5.4%, #105 of 108, behind 96% of the models measured. The leader, Claude Opus 5.5 (max), reaches 68.2%. It covers language-model capability assessment. The board median is 29.2%, so Llama-3.1-8B-Instruct is 23.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
SciCode is a weak spot. Llama-3.1-8B-Instruct scores 13.2%, #105 of 106, behind 98% of the models measured. The leader, Claude Opus 5.5 (max), reaches 66.9%. It covers scientific research code. The board median is 44.3%, so Llama-3.1-8B-Instruct is 31.1 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
WeirdML is a weak spot. Llama-3.1-8B-Instruct scores 1.7%, #109 of 109, behind 99% of the models measured. The leader, GPT-6 Astra (max), reaches 93.3%. It covers unusual machine-learning tasks. The board median is 41.2%, so Llama-3.1-8B-Instruct is 39.4 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Strengths and weaknesses
Software engineering: weak, ahead of 9% of the field on average across 2 benchmarks. Its best board here is Aider Code Editing (#77 of 92) and its weakest is SciCode (#105 of 106). Reasoning, maths and science: weak, ahead of 8% of the field on average across 3 benchmarks. Its best board here is CritPt (#100 of 115) and its weakest is GPQA Diamond (#260 of 275). Agents and tool use: weak, ahead of 16% of the field on average across 2 benchmarks. Its best board here is BALROG (#27 of 34) and its weakest is DTBench (#129 of 146). General knowledge and chat: weak, ahead of 3% of the field on average across 3 benchmarks. Its best board here is Chess Puzzles (#114 of 122) and its weakest is WeirdML (#109 of 109).
Price and value
The median input price across the 427 priced models is $0.500 per million tokens, so Llama-3.1-8B-Instruct costs less than the typical listing. Models that score within 1.5 points or better and cost less: Qwen2.5-Coder-7B-Instruct ($0.010, 49.5), DeepSeek V4.1 Flash (high) ($0.010, 48.9), Qwen2.5-Coder-3B-Instruct ($0.010, 46.5).
Release history
The Meta AI model released before it was Llama 3.1-8B on 2024-07-23, which scores 45.3. The next one was Llama-3.2-11B-Vision-Instruct on 2024-09-24, at 48.0.
Reading these numbers
The 10 scores on this page come from 2 sources: Epoch AI Benchmarking Hub (9), Aider Aider Code Editing leaderboard (1). 1 of them come from an official leaderboard and 9 from datasets or third-party runs.
Models ranked nearby
- #740 GPT-4.1 nano (38.7)
- #741 GPT-3.5 Turbo (Jan 2024) (38.5)
- #742 gpt-4o-mini-2024-07-18 (38.3)
More from Meta AI
- Muse Spark 1.3 (max) (#18, 55.1)
- Muse Spark (#29, 53.7)
- Muse Spark 1.1 (xhigh) (#66, 52.4)
- Muse Spark 1.2 (xhigh) (#113, 51.5)
- Llama 3.1-405B (#142, 51.2)
- Muse Wonder (max) (#209, 50.1)
Compare it on the live leaderboard → · How the aggregate score works