Claude 3.5 Sonnet: benchmark scores, price and ranking
Claude 3.5 Sonnet from Anthropic ranks #282 of 743 LLMs on the LLMs Tiger aggregate benchmark score (49.2), based on 5 benchmark results captured up to 2026-10-13.
- Organisation
- Anthropic
- Type
- Frontier model
- Open weights
- No
- Released
- 2024-06-20 (source: curated)
- Cheapest API price
- $3.00 in / $15.00 out per million tokens (via snowflake, observed 2026-10-13T20:47Z)
- Aggregate rank
- #282 of 743 (score 49.2)
Relative to other models, Claude 3.5 Sonnet is strongest on SWE-bench Lite (#5 of 20) and weakest on SWE-bench Verified (#58 of 73).
Across the 4 benchmarks where at least ten models are scored, Claude 3.5 Sonnet is in the top 10% on 0 and in the top half on 3.
Its cheapest listed input price, $3.00 per million tokens, is lower than that of 11% of the 427 models with a listed price.
Claude 3.5 Sonnet benchmark results
| Benchmark | Score | Rank | Source | Captured |
|---|---|---|---|---|
| SWE-bench Lite | 39.0% | #5 of 20 | SWE-bench Lite leaderboard | 2026-10-13 |
| SWE-bench Test | 18.1% | #4 of 11 | SWE-bench Test leaderboard | 2026-10-13 |
| Epoch ECI | 130 | #89 of 204 | Epoch AI Benchmarking Hub | 2026-10-13 |
| SWE-bench Verified | 33.6% | #58 of 73 | SWE-bench Verified leaderboard | 2026-10-13 |
| SWE-bench Multimodal | 25.3% | #4 of 5 | SWE-bench Multimodal leaderboard | 2026-10-13 |
About Claude 3.5 Sonnet
Claude 3.5 Sonnet is a frontier model from Anthropic. It was released on 2024-06-20, according to the developer's announcement. The cheapest API listing we track charges $3.00 per million input tokens and $15.00 per million output tokens, through snowflake. We hold 5 benchmark results for it across 2 areas, and none is in the top tenth of its board.
Where Claude 3.5 Sonnet stands
On our aggregate score, Claude 3.5 Sonnet is #282 of 743 ranked models, around the middle of the table. Its score is 49.2; the scale is centred on 50, so it sits 0.8 points below the typical model. That beats 62% of the ranked field. Among the 70 ranked Anthropic models it is #46. The next one up is claude-opus-4-1-20250805_16K at 49.5. The next one down is Claude Opus 4.1 (unknown thinking) at 49.1.
Claude 3.5 Sonnet, benchmark by benchmark
SWE-bench Lite puts Claude 3.5 Sonnet at 39.0%, #5 of 20. That beats 75% of the models tested, but 17.7 points short of Claude 4 Sonnet (56.7%). The results either side are GPT-4 (0806) (39.7%) and DeepSeek-V3 (30.7%). It covers real GitHub issues resolved. The board median is 24.7%, so Claude 3.5 Sonnet is 14.3 points above it. Source: SWE-bench Lite leaderboard, captured 2026-10-13.
On SWE-bench Test, Claude 3.5 Sonnet reaches 18.1%. It ranks #4 of 11, ahead of 64% of the field, and 34.5 points behind Claude 4.5 Opus at 52.6%. The results either side are Claude 3.5-Sonnet-20241022 (24.9%) and GPT-4o (14.6%). It covers real GitHub issues resolved. The board median is 12.5%, so Claude 3.5 Sonnet is 5.7 points above it. Source: SWE-bench Test leaderboard, captured 2026-10-13.
Claude 3.5 Sonnet manages 130 on Epoch ECI, good for #89 of 204. Qwen 3.8 Max leads at 156, 26 points further up, and 56% of models sit below. The results either side are Gemma 3 27B (130) and Llama 4 Scout (130). It covers Epoch Capabilities Index: Epoch's own aggregate of ~40 benchmarks. The board median is 127, so Claude 3.5 Sonnet is 3 points above it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.
Claude 3.5 Sonnet lands below the middle on SWE-bench Verified: 33.6%, #58 of 73, with 78% of the field scoring higher. Claude Opus 4.7 (max) leads at 83.5%; the closest result above is Qwen2.5 Coder 32B Instruct at 38.0%. It covers real GitHub issues resolved. The board median is 56.4%, so Claude 3.5 Sonnet is 22.8 points below it. Source: SWE-bench Verified leaderboard, captured 2026-10-13.
The full table above also lists the other 1 result, including boards with fewer than ten models, where a rank says little.
Strengths and weaknesses
Software engineering: uneven, ahead of 48% of the field on average across 2 benchmarks. Its best board here is SWE-bench Lite (#5 of 20) and its weakest is SWE-bench Verified (#58 of 73). General knowledge and chat: solid, ahead of 60% of the field on average across 2 benchmarks. Its best board here is SWE-bench Test (#4 of 11) and its weakest is Epoch ECI (#89 of 204).
Price and value
The median input price across the 427 priced models is $0.500 per million tokens, so Claude 3.5 Sonnet costs more than the typical listing. Models that score within 1.5 points or better and cost less: Qwen2.5-Coder-7B-Instruct ($0.010, 49.5), DeepSeek V4.1 Flash (high) ($0.010, 48.9), DeepSeek V4 Flash 0731 (max) ($0.016, 53.1). Similar scores cost more on Muse Wonder (max) ($4.00) and claude-opus-4-5-20251101-thinking-64k-high-effort ($5.00).
Release history
The Anthropic model released before it was claude-3-haiku-20240307 on 2024-03-13, which scores 39.9. The next one was claude-3.5-sonnet-20240620 on 2024-06-20, at 52.7.
Reading these numbers
The 5 scores on this page come from 5 sources: Epoch AI Benchmarking Hub (1), SWE-bench Lite leaderboard (1), SWE-bench Multimodal leaderboard (1), SWE-bench Test leaderboard (1). 4 of them come from an official leaderboard and 1 from datasets or third-party runs. With 5 results the aggregate rank can move a lot when one new score arrives.
Models ranked nearby
- #279 qwq-plus (49.3)
- #280 Mistral-Large-Instruct-2407 (49.3)
- #281 gemini-2.5-flash-preview-05-20_16K (49.3)
- #283 GLM-4.5 Thinking (49.2)
- #284 gemini-2.5-pro_32K (49.2)
- #285 gpt-3.5-turbo-1106 (49.2)
More from Anthropic
- Claude Opus 5.5 (max) (#1, 64.3)
- Claude Fable 5.1 (max) (#3, 61.9)
- Claude Sonnet 5.5 (max) (#4, 60.8)
- Claude Opus 5 (max) (#8, 60.0)
- Claude Fable 5 (max) (#9, 59.9)
- claude-opus-4-8-xhigh-effort (#14, 56.6)
Compare it on the live leaderboard → · How the aggregate score works