Claude 3.5 Sonnet (Oct 2024): benchmark scores, price and ranking
Claude 3.5 Sonnet (Oct 2024) from Anthropic ranks #583 of 743 LLMs on the LLMs Tiger aggregate benchmark score (45.4), based on 11 benchmark results captured up to 2026-10-13.
- Organisation
- Anthropic
- Type
- Frontier model
- Open weights
- No
- Released
- 2024-10-22 (source: epoch)
- Cheapest API price
- $3.00 in / $15.00 out per million tokens (via snowflake, observed 2026-10-13T19:47Z)
- Aggregate rank
- #583 of 743 (score 45.4)
Relative to other models, Claude 3.5 Sonnet (Oct 2024) is strongest on BALROG (#11 of 34) and weakest on Humanity's Last Exam (#25 of 27).
Across the 11 benchmarks where at least ten models are scored, Claude 3.5 Sonnet (Oct 2024) is in the top 10% on 0 and in the top half on 2.
Its cheapest listed input price, $3.00 per million tokens, is lower than that of 11% of the 427 models with a listed price.
Claude 3.5 Sonnet (Oct 2024) benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| BALROG | 32.6% | #11 of 34 | Epoch AI Benchmarking Hub | 2026-10-13 |
| TheAgentCompany | 24.0% | #5 of 13 | Epoch AI Benchmarking Hub | 2026-10-13 |
| SimpleBench | 41.4% | #31 of 61 | Epoch AI Benchmarking Hub | 2026-10-13 |
| DTBench | 67.8% | #77 of 146 | Epoch AI Benchmarking Hub | 2026-10-13 |
| METR time horizon | 21 min | #17 of 31 | Epoch AI Benchmarking Hub | 2026-10-13 |
| WeirdML | 40.0% | #60 of 109 | Epoch AI Benchmarking Hub | 2026-10-13 |
| GSO-Bench | 4.6% | #12 of 19 | Epoch AI Benchmarking Hub | 2026-10-13 |
| GPQA Diamond | 55.3% | #178 of 275 | Epoch AI Benchmarking Hub | 2026-10-13 |
| EnigmaEval | 0.9% | #17 of 24 | Epoch AI Benchmarking Hub | 2026-10-13 |
| OTIS Mock AIME | 8.5% | #139 of 189 | Epoch AI Benchmarking Hub | 2026-10-13 |
| Humanity's Last Exam | 4.1% | #25 of 27 | Epoch AI Benchmarking Hub | 2026-10-13 |
About Claude 3.5 Sonnet (Oct 2024)
Claude 3.5 Sonnet (Oct 2024) is a frontier model from Anthropic. It was released on 2024-10-22, according to Epoch AI. The cheapest API listing we track charges $3.00 per million input tokens and $15.00 per million output tokens, through snowflake. We hold 11 benchmark results for it across 4 areas, and none is in the top tenth of its board.
Where Claude 3.5 Sonnet (Oct 2024) stands
On our aggregate score, Claude 3.5 Sonnet (Oct 2024) is #583 of 743 ranked models, in the lower half. Its score is 45.4; the scale is centred on 50, so it sits 4.6 points below the typical model. That beats 22% of the ranked field. Among the 70 ranked Anthropic models it is #61. The next one up is Claude 3 Opus at 45.7. The next one down is claude-haiku-4-5-20251001 at 45.2.
Claude 3.5 Sonnet (Oct 2024), benchmark by benchmark
BALROG puts Claude 3.5 Sonnet (Oct 2024) at 32.6%, #11 of 34. That beats 68% of the models tested, but 35.7 points short of GPT-6 Astra (max) (68.3%). The results either side are gemini-2.5-flash (33.5%) and GPT-4o (May 2024) (32.3%). It covers long-horizon game-playing agents. The board median is 27.6%, so Claude 3.5 Sonnet (Oct 2024) is 5.0 points above it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
On TheAgentCompany, Claude 3.5 Sonnet (Oct 2024) reaches 24.0%. It ranks #5 of 13, ahead of 62% of the field, and 17.1 points behind gemini-2.5-flash-preview-09-2025 at 41.1%. The results either side are Gemini 2.5 Pro Preview (Jun 2025) (30.3%) and Gemini 2.0 Flash (Feb 2025) (11.4%). It covers work tasks in a simulated software company. The board median is 8.6%, so Claude 3.5 Sonnet (Oct 2024) is 15.4 points above it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on SimpleBench: 41.4%, #31 of 61, with 49% of the field scoring higher. Claude Fable 5 (max) leads at 81.9%; the closest result above is o1-preview at 41.7%. It covers trick questions that test common sense. The median score on this board is also 41.4%. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on DTBench: 67.8%, #77 of 146, with 52% of the field scoring higher. Claude Opus 5.5 (max) leads at 98.9%; the closest result above is gpt-oss-20b (high) at 68.0%. It covers dynamic tool-use benchmark. The board median is 68.8%, so Claude 3.5 Sonnet (Oct 2024) is 1.0 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on METR time horizon: 21 min, #17 of 31, with 52% of the field scoring higher. Claude Mythos Preview (Early) leads at 1,045 min; the closest result above is DeepSeek-V3 (Mar 2025) at 23 min. It covers length of task (minutes) done at 50% success. The board median is 23 min, so Claude 3.5 Sonnet (Oct 2024) is 3 minutes below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on WeirdML: 40.0%, #60 of 109, with 54% of the field scoring higher. GPT-6 Astra (max) leads at 93.3%; the closest result above is GLM-4.5 Thinking at 40.6%. It covers unusual machine-learning tasks. The board median is 41.2%, so Claude 3.5 Sonnet (Oct 2024) is 1.2 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on GPQA Diamond: 55.3%, #178 of 275, with 64% of the field scoring higher. GPT-6 Astra (max) leads at 95.8%; the closest result above is qwen3-30b-a3b-instruct-2507 at 55.6%. It covers expert-level science questions. The board median is 71.7%, so Claude 3.5 Sonnet (Oct 2024) is 16.4 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on EnigmaEval: 0.9%, #17 of 24, with 67% of the field scoring higher. Gemini 3 Pro Preview leads at 18.2%; the closest result above is Gemini 2.0 Flash Thinking Exp at 1.1%. It covers multi-step puzzle hunts. The board median is 2.9%, so Claude 3.5 Sonnet (Oct 2024) is 2.0 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet (Oct 2024) lands below the middle on OTIS Mock AIME: 8.5%, #139 of 189, with 73% of the field scoring higher. Claude Fable 5.1 (max) leads at 100.0%; the closest result above is Llama-3.1-405B-Instruct at 9.7%. It covers competition maths (AIME 2024-25 style). The board median is 57.8%, so Claude 3.5 Sonnet (Oct 2024) is 49.3 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Humanity's Last Exam is a weak spot. Claude 3.5 Sonnet (Oct 2024) scores 4.1%, #25 of 27, behind 89% of the models measured. The leader, Muse Spark, reaches 40.6%. It covers expert-level questions across domains. The board median is 10.7%, so Claude 3.5 Sonnet (Oct 2024) is 6.6 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
The full table above also lists the other 1 result, including boards with fewer than ten models, where a rank says little.
Strengths and weaknesses
Reasoning, maths and science: weak, ahead of 29% of the field on average across 5 benchmarks. Its best board here is SimpleBench (#31 of 61) and its weakest is Humanity's Last Exam (#25 of 27). Agents and tool use: solid, ahead of 59% of the field on average across 3 benchmarks. Its best board here is BALROG (#11 of 34) and its weakest is DTBench (#77 of 146). General knowledge and chat: uneven, ahead of 45% of the field on average across 2 benchmarks. Its best board here is METR time horizon (#17 of 31) and its weakest is WeirdML (#60 of 109).
Price and value
The median input price across the 427 priced models is $0.500 per million tokens, so Claude 3.5 Sonnet (Oct 2024) costs more than the typical listing. Models that score within 1.5 points or better and cost less: Qwen2.5-Coder-7B-Instruct ($0.010, 49.5), DeepSeek V4.1 Flash (high) ($0.010, 48.9), Qwen2.5-Coder-3B-Instruct ($0.010, 46.5). Similar scores cost more on GPT-4 Turbo Preview (Nov 2023) ($10.00) and GPT-4 Turbo Preview (January 2024) ($10.00).
Release history
The Anthropic model released before it was Claude 3.5-Sonnet-20241022 on 2024-10-22, which scores 54.4. The next one was claude-3-7-sonnet-20250219 on 2025-02-24, at 48.2.
Reading these numbers
The 11 scores on this page come from 1 source: Epoch AI Benchmarking Hub (11). No official leaderboard supplied any of them, so treat them as third-party measurements.
Models ranked nearby
- #580 qwen1.5-72b-chat (45.5)
- #581 Qwen3-Coder-480B-A35B-Instruct (45.5)
- #582 ollama/codegeex4 (45.5)
- #584 phi-4 (45.4)
- #585 gpt-4-turbo (45.4)
- #586 GPT-3.5 Turbo (Nov 2023) (45.4)
More from Anthropic
- Claude Opus 5.5 (max) (#1, 64.3)
- Claude Fable 5.1 (max) (#3, 61.9)
- Claude Sonnet 5.5 (max) (#4, 60.8)
- Claude Opus 5 (max) (#8, 60.0)
- Claude Fable 5 (max) (#9, 59.9)
- claude-opus-4-8-xhigh-effort (#14, 56.6)
Compare it on the live leaderboard → · How the aggregate score works