MATH-500: what the benchmark actually measures
A practical guide to math-500, using data from the AI & LLM Index. The goal is a shortlist and a decision, not a tour of every option.
What MATH-500 measures
MATH-500 is a mathematics benchmark. A 500-problem subset of competition mathematics problems.
The index scores all 600 models on a 0–100 scale. The average is 72.5 and the top score is 96.7.
How to read the scores
A benchmark score is a compressed signal. It tells you how a model performed on a specific test distribution, not how it will perform on your data. Read the leaderboard as a way to form a shortlist, then validate the shortlist with your own examples.
On MATH-500, the gap between the leader and the median is 24.2 points. That gap is the useful part of the chart: it shows how much separation the benchmark creates.
Leaders
GPT-5.2 Max leads at 96.7, followed by GPT-5.1 Reasoning at 95.9 and GPT-5.1 Pro at 95.1.
Limitations
- Scores in this index are modelled estimates, not official results.
- Benchmark contamination can inflate scores for models trained on similar data.
- A high score does not guarantee good performance on a different task shape.
- Prompt format and evaluation harness can move results by several points.
Top 5 on MATH-500
| Model | Provider | Score | Input / 1M |
|---|---|---|---|
| GPT-5.2 Max | OpenAI | 96.7 | $21.39 |
| GPT-5.1 Reasoning | OpenAI | 95.9 | $20.27 |
| GPT-5.1 Pro | OpenAI | 95.1 | $19.77 |
| o4 Reasoning | OpenAI | 94.8 | $19.69 |
| Grok 4 Max | xAI | 94.6 | $19.70 |
Related
FAQ
Who leads MATH-500?
GPT-5.2 Max leads this index at 96.7.
What is the most important factor in math-500?
Fit to the task beats raw benchmark rank. Define the failure modes, pick the benchmarks that resemble them, then test a small shortlist on your own examples.
How many models should I evaluate?
Three is usually enough: a quality candidate, a value candidate and a fallback. More candidates add work without changing the decision.
How do I control cost?
Compare blended 3:1 price, watch output tokens, use caching where available and route simple requests to a cheaper model.