gpt-4o-mini-2024-07-18: benchmark scores, price and ranking

gpt-4o-mini-2024-07-18 from OpenAI ranks #742 of 743 LLMs on the LLMs Tiger aggregate benchmark score (38.3), based on 12 benchmark results captured up to 2026-10-13.

Organisation

OpenAI

Type

Fast / low-cost (flash) model

Open weights

No

Released

2024-07-18 (source: name)

Context window

128,000 tokens

Cheapest API price

$0.150 in / $0.600 out per million tokens (via azure, observed 2026-10-13T19:47Z)

Aggregate rank

#742 of 743 (score 38.3)

Relative to other models, gpt-4o-mini-2024-07-18 is strongest on BALROG (#24 of 34) and weakest on Aider Polyglot (#49 of 49).

Across the 12 benchmarks where at least ten models are scored, gpt-4o-mini-2024-07-18 is in the top 10% on 0 and in the top half on 0.

Its cheapest listed input price, $0.150 per million tokens, is lower than that of 72% of the 427 models with a listed price.

gpt-4o-mini-2024-07-18 benchmark results

About gpt-4o-mini-2024-07-18

gpt-4o-mini-2024-07-18 is a smaller, lower-cost model from OpenAI. It was released on 2024-07-18, according to the model's own name. It accepts up to 128,000 tokens of input at once. The cheapest API listing we track charges $0.150 per million input tokens and $0.600 per million output tokens, through azure. We hold 12 benchmark results for it across 4 areas, and none is in the top tenth of its board.

Where gpt-4o-mini-2024-07-18 stands

On our aggregate score, gpt-4o-mini-2024-07-18 is #742 of 743 ranked models, in the lower half. Its score is 38.3; the scale is centred on 50, so it sits 11.7 points below the typical model. That beats 0% of the ranked field. Among the 88 ranked OpenAI models it is #88. The next one up is GPT-3.5 Turbo (Jan 2024) at 38.5.

gpt-4o-mini-2024-07-18, benchmark by benchmark

gpt-4o-mini-2024-07-18 lands below the middle on BALROG: 17.4%, #24 of 34, with 68% of the field scoring higher. GPT-6 Astra (max) leads at 68.3%; the closest result above is Mistral-Nemo-Instruct-2407 at 17.6%. It covers long-horizon game-playing agents. The board median is 27.6%, so gpt-4o-mini-2024-07-18 is 10.2 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

gpt-4o-mini-2024-07-18 lands below the middle on Mystery Games: 12.0%, #44 of 59, with 73% of the field scoring higher. GPT-6 Astra (max) leads at 84.0%; the closest result above is GPT-4 (Jun 2023) at 12.0%. It covers deductive reasoning puzzles. The board median is 23.0%, so gpt-4o-mini-2024-07-18 is 11.0 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

gpt-4o-mini-2024-07-18 lands below the middle on OTIS Mock AIME: 6.9%, #146 of 189, with 77% of the field scoring higher. Claude Fable 5.1 (max) leads at 100.0%; the closest result above is qwen2.5-32b-instruct at 7.4%. It covers competition maths (AIME 2024-25 style). The board median is 57.8%, so gpt-4o-mini-2024-07-18 is 50.8 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

gpt-4o-mini-2024-07-18 lands below the middle on DTBench: 54.4%, #115 of 146, with 78% of the field scoring higher. Claude Opus 5.5 (max) leads at 98.9%; the closest result above is c4ai-command-r-plus-08-2024 at 54.9%. It covers dynamic tool-use benchmark. The board median is 68.8%, so gpt-4o-mini-2024-07-18 is 14.4 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

GPQA Diamond is a weak spot. gpt-4o-mini-2024-07-18 scores 37.7%, #229 of 275, behind 83% of the models measured. The leader, GPT-6 Astra (max), reaches 95.8%. It covers expert-level science questions. The board median is 71.7%, so gpt-4o-mini-2024-07-18 is 34.0 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

LMCA is a weak spot. gpt-4o-mini-2024-07-18 scores 10.4%, #93 of 108, behind 85% of the models measured. The leader, Claude Opus 5.5 (max), reaches 68.2%. It covers language-model capability assessment. The board median is 29.2%, so gpt-4o-mini-2024-07-18 is 18.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

FrontierMath T1-3 is a weak spot. gpt-4o-mini-2024-07-18 scores 0.7%, #66 of 68, behind 96% of the models measured. The leader, GPT-6 Astra (max), reaches 93.7%. It covers research-level maths, tiers 1-3. The board median is 55.6%, so gpt-4o-mini-2024-07-18 is 54.9 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

SimpleQA Verified is a weak spot. gpt-4o-mini-2024-07-18 scores 8.3%, #68 of 69, behind 97% of the models measured. The leader, gemini-4-argon-high, reaches 76.0%. It covers short factual questions. The board median is 41.0%, so gpt-4o-mini-2024-07-18 is 32.7 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

SimpleBench is a weak spot. gpt-4o-mini-2024-07-18 scores 10.7%, #61 of 61, behind 98% of the models measured. The leader, Claude Fable 5 (max), reaches 81.9%. It covers trick questions that test common sense. The board median is 41.4%, so gpt-4o-mini-2024-07-18 is 30.7 points below it. Source: Epoch AI Benchmarking Hub, captured 2026-10-13.

Aider Polyglot is a weak spot. gpt-4o-mini-2024-07-18 scores 3.6%, #49 of 49, behind 98% of the models measured. The leader, gemini-2.5-pro-preview-06-05, reaches 83.1%. It covers edits across 6 languages (225 exercises). The board median is 44.9%, so gpt-4o-mini-2024-07-18 is 41.3 points below it. Source: Aider polyglot leaderboard, captured 2026-10-13.

The full table above also lists the other 2 results, including boards with fewer than ten models, where a rank says little.

Strengths and weaknesses

Reasoning, maths and science: weak, ahead of 11% of the field on average across 4 benchmarks. Its best board here is OTIS Mock AIME (#146 of 189) and its weakest is SimpleBench (#61 of 61). Agents and tool use: weak, ahead of 25% of the field on average across 2 benchmarks. Its best board here is BALROG (#24 of 34) and its weakest is DTBench (#115 of 146). General knowledge and chat: weak, ahead of 12% of the field on average across 5 benchmarks. Its best board here is Mystery Games (#44 of 59) and its weakest is SimpleQA Verified (#68 of 69).

Price and value

The median input price across the 427 priced models is $0.500 per million tokens, so gpt-4o-mini-2024-07-18 costs less than the typical listing. Models that score within 1.5 points or better and cost less: Qwen2.5-Coder-7B-Instruct ($0.010, 49.5), DeepSeek V4.1 Flash (high) ($0.010, 48.9), Qwen2.5-Coder-3B-Instruct ($0.010, 46.5). Similar scores cost more on GPT-3.5 Turbo (Jan 2024) ($0.500).

Release history

The OpenAI model released before it was GPT-4o mini on 2024-07-18, which scores 47.1. The next one was gpt-4o-2024-08-06 on 2024-08-06, at 49.5.

Reading these numbers

The 12 scores on this page come from 2 sources: Epoch AI Benchmarking Hub (11), Aider polyglot leaderboard (1). 1 of them come from an official leaderboard and 11 from datasets or third-party runs.

Models ranked nearby

• #739 gemma-3-12b-it (39.1)
• #740 GPT-4.1 nano (38.7)
• #741 GPT-3.5 Turbo (Jan 2024) (38.5)
• #743 Llama-3.1-8B-Instruct (36.7)

More from OpenAI

• GPT-6 Astra (max) (#2, 64.0)
• GPT-6.1 Sol (max) (#5, 60.6)
• GPT-5.6 Sol (max) (#6, 60.1)
• GPT-6 Sol (max) (#7, 60.0)
• GPT-5.5 (xhigh) (#11, 58.0)
• GPT-5.6 Terra (max) (#12, 57.8)

Compare it on the live leaderboard → · How the aggregate score works