Claude 3.5 Sonnet (Jun 2024): benchmark scores, price and ranking
Claude 3.5 Sonnet (Jun 2024) from Anthropic ranks #646 of 743 LLMs on the LLMs Tiger aggregate benchmark score (44.2), based on 7 benchmark results captured up to 2026-10-13.
- Organisation
- Anthropic
- Type
- Frontier model
- Open weights
- No
- Released
- 2024-06-20 (source: epoch)
- Cheapest API price
- $3.00 in / $15.00 out per million tokens (via snowflake, observed 2026-10-13T20:47Z)
- Aggregate rank
- #646 of 743 (score 44.2)
Relative to other models, Claude 3.5 Sonnet (Jun 2024) is strongest on Cybench (#8 of 16) and weakest on OTIS Mock AIME (#150 of 189).
Across the 7 benchmarks where at least ten models are scored, Claude 3.5 Sonnet (Jun 2024) is in the top 10% on 0 and in the top half on 1.
Its cheapest listed input price, $3.00 per million tokens, is lower than that of 11% of the 427 models with a listed price.
Claude 3.5 Sonnet (Jun 2024) benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| Cybench | 17.5% | #8 of 16 | Epoch AI Benchmarking Hub | 2026-10-13 |
| DTBench | 67.8% | #78 of 146 | Epoch AI Benchmarking Hub | 2026-10-13 |
| METR time horizon | 11 min | #20 of 31 | Epoch AI Benchmarking Hub | 2026-10-13 |
| GPQA Diamond | 54.0% | #179 of 275 | Epoch AI Benchmarking Hub | 2026-10-13 |
| SimpleBench | 27.5% | #45 of 61 | Epoch AI Benchmarking Hub | 2026-10-13 |
| WeirdML | 31.0% | #85 of 109 | Epoch AI Benchmarking Hub | 2026-10-13 |
| OTIS Mock AIME | 6.5% | #150 of 189 | Epoch AI Benchmarking Hub | 2026-10-13 |
About Claude 3.5 Sonnet (Jun 2024)
Claude 3.5 Sonnet (Jun 2024) is a frontier model from Anthropic. It was released on 2024-06-20, according to Epoch AI. The cheapest API listing we track charges $3.00 per million input tokens and $15.00 per million output tokens, through snowflake. We hold 7 benchmark results for it across 3 areas, and none is in the top tenth of its board.
Where Claude 3.5 Sonnet (Jun 2024) stands
On our aggregate score, Claude 3.5 Sonnet (Jun 2024) is #646 of 743 ranked models, in the lower half. Its score is 44.2; the scale is centred on 50, so it sits 5.8 points below the typical model. That beats 13% of the ranked field. Among the 70 ranked Anthropic models it is #63. The next one up is claude-haiku-4-5-20251001 at 45.2. The next one down is Claude 2 at 43.7.
Claude 3.5 Sonnet (Jun 2024), benchmark by benchmark
Cybench puts Claude 3.5 Sonnet (Jun 2024) at 17.5%, #8 of 16. That beats 50% of the models tested, but 25.5 points short of grok-4-0709 (43.0%). The results either side are claude-3-7-sonnet-20250219 (20.0%) and GPT-4.5 Preview (Feb 2025) (17.5%). It covers cybersecurity capture-the-flag tasks. The median score on this board is also 17.5%. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Jun 2024) lands below the middle on DTBench: 67.8%, #78 of 146, with 53% of the field scoring higher. Claude Opus 5.5 (max) leads at 98.9%; the closest result above is Claude 3.5 Sonnet (Oct 2024) at 67.8%. It covers dynamic tool-use benchmark. The board median is 68.8%, so Claude 3.5 Sonnet (Jun 2024) is 1.0 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Jun 2024) lands below the middle on METR time horizon: 11 min, #20 of 31, with 61% of the field scoring higher. Claude Mythos Preview (Early) leads at 1,045 min; the closest result above is DeepSeek-V3 at 18 min. It covers length of task (minutes) done at 50% success. The board median is 23 min, so Claude 3.5 Sonnet (Jun 2024) is 12 minutes below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Jun 2024) lands below the middle on GPQA Diamond: 54.0%, #179 of 275, with 65% of the field scoring higher. GPT-6 Astra (max) leads at 95.8%; the closest result above is Claude 3.5 Sonnet (Oct 2024) at 55.3%. It covers expert-level science questions. The board median is 71.7%, so Claude 3.5 Sonnet (Jun 2024) is 17.7 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Jun 2024) lands below the middle on SimpleBench: 27.5%, #45 of 61, with 72% of the field scoring higher. Claude Fable 5 (max) leads at 81.9%; the closest result above is Llama 4 Maverick at 27.7%. It covers trick questions that test common sense. The board median is 41.4%, so Claude 3.5 Sonnet (Jun 2024) is 13.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Jun 2024) lands below the middle on WeirdML: 31.0%, #85 of 109, with 77% of the field scoring higher. GPT-6 Astra (max) leads at 93.3%; the closest result above is mistral-medium-2508 at 33.1%. It covers unusual machine-learning tasks. The board median is 41.2%, so Claude 3.5 Sonnet (Jun 2024) is 10.2 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Jun 2024) lands below the middle on OTIS Mock AIME: 6.5%, #150 of 189, with 79% of the field scoring higher. Claude Fable 5.1 (max) leads at 100.0%; the closest result above is mistral-small-3-2501 at 6.7%. It covers competition maths (AIME 2024-25 style). The board median is 57.8%, so Claude 3.5 Sonnet (Jun 2024) is 51.2 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Strengths and weaknesses
Reasoning, maths and science: weak, ahead of 27% of the field on average across 3 benchmarks. Its best board here is GPQA Diamond (#179 of 275) and its weakest is OTIS Mock AIME (#150 of 189). General knowledge and chat: uneven, ahead of 36% of the field on average across 3 benchmarks. Its best board here is Cybench (#8 of 16) and its weakest is WeirdML (#85 of 109).
Price and value
The median input price across the 427 priced models is $0.500 per million tokens, so Claude 3.5 Sonnet (Jun 2024) costs more than the typical listing. Models that score within 1.5 points or better and cost less: Qwen2.5-Coder-7B-Instruct ($0.010, 49.5), DeepSeek V4.1 Flash (high) ($0.010, 48.9), Qwen2.5-Coder-3B-Instruct ($0.010, 46.5). Similar scores cost more on Llama-3.1-405B-Instruct ($3.50) and mistral-large-2402 ($8.00).
Release history
The Anthropic model released before it was claude-3.5-sonnet-20240620 on 2024-06-20, which scores 52.7. The next one was Claude Sonnet 3.5 on 2024-06-20, at 46.7.
Reading these numbers
The 7 scores on this page come from 1 source: Epoch AI Benchmarking Hub (7). No official leaderboard supplied any of them, so treat them as third-party measurements. With 7 results the aggregate rank can move a lot when one new score arrives.
Models ranked nearby
- #643 GPT-4o (May 2024) (44.3)
- #644 gpt-oss-20b (high) (44.3)
- #645 Yi-6B-Chat (44.3)
- #647 DeepSeek-V3 (Mar 2025) (44.1)
- #648 Yi-34B-Chat (44.0)
- #649 o3-mini (high) (44.0)
More from Anthropic
- Claude Opus 5.5 (max) (#1, 64.3)
- Claude Fable 5.1 (max) (#3, 61.9)
- Claude Sonnet 5.5 (max) (#4, 60.8)
- Claude Opus 5 (max) (#8, 60.0)
- Claude Fable 5 (max) (#9, 59.9)
- claude-opus-4-8-xhigh-effort (#14, 56.6)
Compare it on the live leaderboard → · How the aggregate score works