HumanEval: scores, leaders and methodology
HumanEval leaderboard data from the index. Python function synthesis from docstrings, scored by unit tests. The top score is 96.8 and the average is 74.0.
What the data shows
GPT-5.2 Pro leads at 96.8. The gap to the average is 22.8 points, and the gap to the 10th-ranked model is 3.9 points.
The median score is 77.4. Models above that line are in the upper half of the index for this benchmark.
How to read the ranking
- Use the score to shortlist, not to declare a winner.
- Check price and context for the top candidates.
- Read the benchmark methodology before quoting a score.
- Pair this benchmark with a task-specific evaluation.
Top 20 on HumanEval
| # | Model | Provider | Score | Input / 1M | Context |
|---|---|---|---|---|---|
| 1 | GPT-5.2 Pro | OpenAI | 96.8 | $21.32 | 2,000,000 |
| 2 | Gemini 3 Pro Pro | Google DeepMind | 95.0 | $19.80 | 200,000 |
| 3 | o4 Max | OpenAI | 94.6 | $19.51 | 2,000,000 |
| 4 | Grok 4 Reasoning | xAI | 93.8 | $19.67 | 2,000,000 |
| 5 | Claude Opus 4.5 | Anthropic | 93.7 | $19.42 | 400,000 |
| 6 | GPT-5.2 Max | OpenAI | 93.6 | $21.39 | 1,000,000 |
| 7 | Claude Opus 4.5 Pro | Anthropic | 93.4 | $20.98 | 200,000 |
| 8 | Grok 4 Max | xAI | 93.4 | $19.70 | 2,000,000 |
| 9 | GPT-5.2 Turbo | OpenAI | 93.2 | $20.73 | 1,000,000 |
| 10 | GPT-5.1 Reasoning | OpenAI | 92.9 | $20.27 | 400,000 |
| 11 | Grok 4 Turbo | xAI | 92.9 | $19.21 | 1,000,000 |
| 12 | Claude Sonnet 4.5 | Anthropic | 92.7 | $17.84 | 400,000 |
| 13 | Claude Sonnet 4.5 Pro | Anthropic | 92.7 | $18.73 | 1,000,000 |
| 14 | Qwen3-Max Reasoning | Alibaba Cloud | 92.6 | $18.12 | 2,000,000 |
| 15 | Qwen3-Max Turbo | Alibaba Cloud | 92.5 | $17.70 | 1,000,000 |
| 16 | Claude Sonnet 4.5 Max | Anthropic | 92.4 | $19.90 | 1,000,000 |
| 17 | GPT-5.2 Reasoning | OpenAI | 92.4 | $20.98 | 1,000,000 |
| 18 | GPT-5.1 Max | OpenAI | 92.2 | $20.50 | 2,000,000 |
| 19 | Claude Opus 4.5 Reasoning | Anthropic | 92.1 | $20.91 | 2,000,000 |
| 20 | Qwen3-Max Pro | Alibaba Cloud | 92.1 | $18.87 | 400,000 |
Methodology
This story uses the AI & LLM Index dataset: 600 models, 40 providers and 12 benchmarks. Prices are listed per million tokens. Scores are modelled estimates on a 0–100 scale and are not official benchmark results. Figures are rounded for readability.
Related
FAQ
What does the HumanEval: scores, leaders and methodology data show?
HumanEval leaderboard data from the index. Python function synthesis from docstrings, scored by unit tests. The top score is 96.8 and the average is 74.0.
Are the numbers official?
No. Prices and scores are compiled for this demo index. Benchmark values are modelled estimates, not official results. Verify figures with the provider or benchmark owner before making decisions.
How should I use this ranking?
Use it to form a shortlist. Check the specific model pages, compare price and context, and test the shortlist on your own examples before committing.