Claude 3.5 Haiku (Oct 2024): benchmark scores, price and ranking
Claude 3.5 Haiku (Oct 2024) from Anthropic ranks #724 of 743 LLMs on the LLMs Tiger aggregate benchmark score (41.3), based on 7 benchmark results captured up to 2026-10-13.
- Organisation
- Anthropic
- Type
- Fast / low-cost (flash) model
- Open weights
- No
- Released
- 2024-10-22 (source: epoch)
- Cheapest API price
- $0.800 in / $4.00 out per million tokens (via vercel_ai_gateway, observed 2026-10-13T20:47Z)
- Aggregate rank
- #724 of 743 (score 41.3)
Relative to other models, Claude 3.5 Haiku (Oct 2024) is strongest on BALROG (#22 of 34) and weakest on SciCode (#93 of 106).
Across the 7 benchmarks where at least ten models are scored, Claude 3.5 Haiku (Oct 2024) is in the top 10% on 0 and in the top half on 0.
Its cheapest listed input price, $0.800 per million tokens, is lower than that of 38% of the 427 models with a listed price.
Claude 3.5 Haiku (Oct 2024) benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| BALROG | 19.3% | #22 of 34 | Epoch AI Benchmarking Hub | 2026-10-13 |
| CritPt | 0.0% | #85 of 115 | Epoch AI Benchmarking Hub | 2026-10-13 |
| DTBench | 56.7% | #110 of 146 | Epoch AI Benchmarking Hub | 2026-10-13 |
| WeirdML | 30.7% | #86 of 109 | Epoch AI Benchmarking Hub | 2026-10-13 |
| GPQA Diamond | 38.1% | #227 of 275 | Epoch AI Benchmarking Hub | 2026-10-13 |
| OTIS Mock AIME | 4.3% | #161 of 189 | Epoch AI Benchmarking Hub | 2026-10-13 |
| SciCode | 27.4% | #93 of 106 | Epoch AI Benchmarking Hub | 2026-10-13 |
About Claude 3.5 Haiku (Oct 2024)
Claude 3.5 Haiku (Oct 2024) is a smaller, lower-cost model from Anthropic. It was released on 2024-10-22, according to Epoch AI. The cheapest API listing we track charges $0.800 per million input tokens and $4.00 per million output tokens, through vercel_ai_gateway. We hold 7 benchmark results for it across 4 areas, and none is in the top tenth of its board.
Where Claude 3.5 Haiku (Oct 2024) stands
On our aggregate score, Claude 3.5 Haiku (Oct 2024) is #724 of 743 ranked models, in the lower half. Its score is 41.3; the scale is centred on 50, so it sits 8.7 points below the typical model. That beats 3% of the ranked field. Among the 70 ranked Anthropic models it is #68. The next one up is claude-3-sonnet-20240229 at 41.9. The next one down is claude-2.1 at 40.6.
Claude 3.5 Haiku (Oct 2024), benchmark by benchmark
Claude 3.5 Haiku (Oct 2024) lands below the middle on BALROG: 19.3%, #22 of 34, with 62% of the field scoring higher. GPT-6 Astra (max) leads at 68.3%; the closest result above is DeepSeek-R1-Distill-Qwen-32B at 19.5%. It covers long-horizon game-playing agents. The board median is 27.6%, so Claude 3.5 Haiku (Oct 2024) is 8.3 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Haiku (Oct 2024) lands below the middle on CritPt: 0.0%, #85 of 115, with 73% of the field scoring higher. GPT-5.6 Sol (max) leads at 32.3%; the closest result above is Qwen3.6 35B-A3B at 0.3%. It covers unpublished research-level physics problems. The board median is 2.9%, so Claude 3.5 Haiku (Oct 2024) is 2.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Haiku (Oct 2024) lands below the middle on DTBench: 56.7%, #110 of 146, with 75% of the field scoring higher. Claude Opus 5.5 (max) leads at 98.9%; the closest result above is Llama-4-Scout-17B-16E-Instruct at 57.9%. It covers dynamic tool-use benchmark. The board median is 68.8%, so Claude 3.5 Haiku (Oct 2024) is 12.1 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Haiku (Oct 2024) lands below the middle on WeirdML: 30.7%, #86 of 109, with 78% of the field scoring higher. GPT-6 Astra (max) leads at 93.3%; the closest result above is Claude 3.5 Sonnet (Jun 2024) at 31.0%. It covers unusual machine-learning tasks. The board median is 41.2%, so Claude 3.5 Haiku (Oct 2024) is 10.4 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
GPQA Diamond is a weak spot. Claude 3.5 Haiku (Oct 2024) scores 38.1%, #227 of 275, behind 82% of the models measured. The leader, GPT-6 Astra (max), reaches 95.8%. It covers expert-level science questions. The board median is 71.7%, so Claude 3.5 Haiku (Oct 2024) is 33.6 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
OTIS Mock AIME is a weak spot. Claude 3.5 Haiku (Oct 2024) scores 4.3%, #161 of 189, behind 85% of the models measured. The leader, Claude Fable 5.1 (max), reaches 100.0%. It covers competition maths (AIME 2024-25 style). The board median is 57.8%, so Claude 3.5 Haiku (Oct 2024) is 53.5 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
SciCode is a weak spot. Claude 3.5 Haiku (Oct 2024) scores 27.4%, #93 of 106, behind 87% of the models measured. The leader, Claude Opus 5.5 (max), reaches 66.9%. It covers scientific research code. The board median is 44.3%, so Claude 3.5 Haiku (Oct 2024) is 16.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Strengths and weaknesses
Reasoning, maths and science: weak, ahead of 19% of the field on average across 3 benchmarks. Its best board here is CritPt (#85 of 115) and its weakest is OTIS Mock AIME (#161 of 189). Agents and tool use: weak, ahead of 30% of the field on average across 2 benchmarks. Its best board here is BALROG (#22 of 34) and its weakest is DTBench (#110 of 146).
Price and value
The median input price across the 427 priced models is $0.500 per million tokens, so Claude 3.5 Haiku (Oct 2024) costs more than the typical listing. Models that score within 1.5 points or better and cost less: Qwen2.5-Coder-7B-Instruct ($0.010, 49.5), DeepSeek V4.1 Flash (high) ($0.010, 48.9), Qwen2.5-Coder-3B-Instruct ($0.010, 46.5). Similar scores cost more on GPT-4o (Nov 2024) ($2.50) and GPT-4o (Aug 2024) ($2.50).
Release history
The Anthropic model released before it was claude-3-5-haiku-20241022 on 2024-10-22, which scores 50.8. The next one was Claude 3.5-Sonnet-20241022 on 2024-10-22, at 54.4.
Reading these numbers
The 7 scores on this page come from 1 source: Epoch AI Benchmarking Hub (7). No official leaderboard supplied any of them, so treat them as third-party measurements. With 7 results the aggregate rank can move a lot when one new score arrives.
Models ranked nearby
- #721 Phi-4-mini-instruct (41.8)
- #722 granite-4.0-350m (41.6)
- #723 gemini-1.0-pro-001 (41.5)
- #725 ministral-3b-2410 (41.3)
- #726 c4ai-command-r-08-2024 (41.3)
- #727 GPT-4 (Jun 2023) (41.2)
More from Anthropic
- Claude Opus 5.5 (max) (#1, 64.3)
- Claude Fable 5.1 (max) (#3, 61.9)
- Claude Sonnet 5.5 (max) (#4, 60.8)
- Claude Opus 5 (max) (#8, 60.0)
- Claude Fable 5 (max) (#9, 59.9)
- claude-opus-4-8-xhigh-effort (#14, 56.6)
Compare it on the live leaderboard → · How the aggregate score works