HumanEval: leaderboard, scores and methodology
Python function synthesis from docstrings, scored by unit tests.
Top 10
Bars are scaled to the highest observed score. The leader sets the pace; the spread below it shows how much separation exists in this benchmark.
Leaderboard
| # | Model | Provider | Context | Input / 1M | Avg score |
|---|---|---|---|---|---|
| 1 | GPT-5.2 Pro | OpenAI | 2000000 | $21.32 | 89.6 |
| 2 | Gemini 3 Pro Pro | Google DeepMind | 200000 | $19.80 | 87.3 |
| 3 | o4 Max | OpenAI | 2000000 | $19.51 | 86.2 |
| 4 | Grok 4 Reasoning | xAI | 2000000 | $19.67 | 87.8 |
| 5 | Claude Opus 4.5 | Anthropic | 400000 | $19.42 | 87.7 |
| 6 | GPT-5.2 Max | OpenAI | 1000000 | $21.39 | 89.2 |
| 7 | Claude Opus 4.5 Pro | Anthropic | 200000 | $20.98 | 89.1 |
| 8 | Grok 4 Max | xAI | 2000000 | $19.70 | 86.6 |
| 9 | GPT-5.2 Turbo | OpenAI | 1000000 | $20.73 | 87.4 |
| 10 | GPT-5.1 Reasoning | OpenAI | 400000 | $20.27 | 88.3 |
| 11 | Grok 4 Turbo | xAI | 1000000 | $19.21 | 86.7 |
| 12 | Claude Sonnet 4.5 | Anthropic | 400000 | $17.84 | 86.0 |
| 13 | Claude Sonnet 4.5 Pro | Anthropic | 1000000 | $18.73 | 86.7 |
| 14 | Qwen3-Max Reasoning | Alibaba Cloud | 2000000 | $18.12 | 85.9 |
| 15 | Qwen3-Max Turbo | Alibaba Cloud | 1000000 | $17.70 | 84.7 |
| 16 | Claude Sonnet 4.5 Max | Anthropic | 1000000 | $19.90 | 87.2 |
| 17 | GPT-5.2 Reasoning | OpenAI | 1000000 | $20.98 | 87.5 |
| 18 | GPT-5.1 Max | OpenAI | 2000000 | $20.50 | 87.9 |
| 19 | Claude Opus 4.5 Reasoning | Anthropic | 2000000 | $20.91 | 88.2 |
| 20 | Qwen3-Max Pro | Alibaba Cloud | 400000 | $18.87 | 86.2 |
| 21 | o4 Pro | OpenAI | 1000000 | $19.37 | 87.3 |
| 22 | Grok 4 | xAI | 200000 | $17.90 | 86.5 |
| 23 | GPT-4.1 Pro | OpenAI | 400000 | $17.56 | 85.2 |
| 24 | Gemini 3 Pro Max | Google DeepMind | 2000000 | $20.57 | 87.9 |
| 25 | Grok 3 Max | xAI | 2000000 | $17.66 | 85.9 |
| 26 | Gemini 3 Pro | Google DeepMind | 2000000 | $18.24 | 86.1 |
| 27 | Claude Opus 4.5 Max | Anthropic | 1000000 | $20.90 | 87.8 |
| 28 | Gemini 3 Pro Vision | Google DeepMind | 200000 | $17.59 | 85.5 |
| 29 | GPT-5.1 Turbo | OpenAI | 2000000 | $19.19 | 86.1 |
| 30 | o3 Reasoning | OpenAI | 200000 | $18.07 | 85.3 |
| 31 | GPT-5.2 Vision | OpenAI | 1000000 | $19.03 | 87.4 |
| 32 | GPT-5.1 Pro | OpenAI | 200000 | $19.77 | 87.8 |
| 33 | GPT-5.2 | OpenAI | 2000000 | $19.71 | 88.0 |
| 34 | o4 Turbo | OpenAI | 2000000 | $19.34 | 86.6 |
| 35 | Claude 3.7 Sonnet Max | Anthropic | 128000 | $16.88 | 83.7 |
| 36 | GPT-5.1 | OpenAI | 2000000 | $19.01 | 86.0 |
| 37 | Gemini 3 Flash Turbo | Google DeepMind | 200000 | $17.07 | 84.1 |
| 38 | o3 Pro | OpenAI | 400000 | $17.89 | 85.8 |
| 39 | Claude Sonnet 4.5 Turbo | Anthropic | 1000000 | $18.37 | 85.9 |
| 40 | Claude Sonnet 4.5 Reasoning | Anthropic | 400000 | $19.48 | 87.7 |
| 41 | Gemini 3 Pro Turbo | Google DeepMind | 400000 | $19.62 | 86.4 |
| 42 | DeepSeek-V3.2 Instruct | DeepSeek | 128000 | $5.91 | 83.7 |
| 43 | Gemini 3 Pro Reasoning | Google DeepMind | 200000 | $19.69 | 87.5 |
| 44 | Qwen3-Max Max | Alibaba Cloud | 400000 | $18.53 | 85.9 |
| 45 | o4 Reasoning | OpenAI | 1000000 | $19.69 | 87.5 |
| 46 | Claude 3.7 Sonnet Pro | Anthropic | 128000 | $16.07 | 83.3 |
| 47 | Gemini 3 Flash Pro | Google DeepMind | 400000 | $17.71 | 84.8 |
| 48 | Grok 3 Pro | xAI | 400000 | $16.61 | 83.8 |
| 49 | Claude Opus 4.5 Turbo | Anthropic | 2000000 | $19.57 | 85.7 |
| 50 | GPT-4.1 Turbo | OpenAI | 1000000 | $16.12 | 82.9 |
Methodology
HumanEval is included as a coding benchmark with a typical score range of 0 to 100. The values in this index are deterministic modelled estimates generated for comparison at scale. They are not official leaderboard results and should not be quoted as vendor benchmarks.
Each model receives a score derived from its capability tier, release generation and benchmark difficulty. Scores are normalised to a 0–100 scale so that models from different providers can be placed side by side. The average shown is the arithmetic mean across all 600 scored models.
Read the leaderboard as a relative map. A five-point gap is meaningful at the top and less meaningful in the middle of the distribution. Pair the score with price and context data before choosing a model.
How to use this leaderboard
- Use the top 10 to shortlist candidates, not to declare a winner.
- Check whether the leader is open weights or proprietary before assuming you can deploy it.
- Compare the score against input and output price on each model page.
- Look at the model's scores on related benchmarks before trusting a single number.
FAQ
What is HumanEval?
Python function synthesis from docstrings, scored by unit tests.
Who leads HumanEval?
GPT-5.2 Pro from OpenAI leads this index with a score of 96.8, followed by Gemini 3 Pro Pro at 95.0.
What is a good HumanEval score?
The top quartile starts around 83.4. The index average is 74.0, so a score above 83.4 is a strong result relative to this dataset.
How are HumanEval scores calculated?
The values are modelled estimates on a 0–100 scale, generated from capability tier, release generation and benchmark difficulty. They are not official leaderboard results.
Which models are included?
All 600 models in the index are scored on HumanEval, across 40 providers and every model category.