SWE-bench Verified: leaderboard, scores and methodology
Real GitHub issues resolved by an AI coding agent, verified by human reviewers.
Top 10
Bars are scaled to the highest observed score. The leader sets the pace; the spread below it shows how much separation exists in this benchmark.
Leaderboard
| # | Model | Provider | Context | Input / 1M | Avg score |
|---|---|---|---|---|---|
| 1 | GPT-5.2 Pro | OpenAI | 2000000 | $21.32 | 89.6 |
| 2 | Claude Opus 4.5 Pro | Anthropic | 200000 | $20.98 | 89.1 |
| 3 | GPT-5.2 Max | OpenAI | 1000000 | $21.39 | 89.2 |
| 4 | Claude Sonnet 4.5 Max | Anthropic | 1000000 | $19.90 | 87.2 |
| 5 | GPT-5.2 | OpenAI | 2000000 | $19.71 | 88.0 |
| 6 | GPT-5.1 Pro | OpenAI | 200000 | $19.77 | 87.8 |
| 7 | o4 Max | OpenAI | 2000000 | $19.51 | 86.2 |
| 8 | Claude Sonnet 4.5 Reasoning | Anthropic | 400000 | $19.48 | 87.7 |
| 9 | Claude Opus 4.5 Max | Anthropic | 1000000 | $20.90 | 87.8 |
| 10 | Claude Opus 4.5 Reasoning | Anthropic | 2000000 | $20.91 | 88.2 |
| 11 | GPT-5.2 Vision | OpenAI | 1000000 | $19.03 | 87.4 |
| 12 | Gemini 3 Pro | Google DeepMind | 2000000 | $18.24 | 86.1 |
| 13 | o4 Pro | OpenAI | 1000000 | $19.37 | 87.3 |
| 14 | o4 Turbo | OpenAI | 2000000 | $19.34 | 86.6 |
| 15 | GPT-5.2 Reasoning | OpenAI | 1000000 | $20.98 | 87.5 |
| 16 | Grok 4 Reasoning | xAI | 2000000 | $19.67 | 87.8 |
| 17 | Grok 4 | xAI | 200000 | $17.90 | 86.5 |
| 18 | Claude Opus 4.5 Vision | Anthropic | 200000 | $17.70 | 85.9 |
| 19 | GPT-5.2 Turbo | OpenAI | 1000000 | $20.73 | 87.4 |
| 20 | Gemini 3 Pro Turbo | Google DeepMind | 400000 | $19.62 | 86.4 |
| 21 | GPT-4.1 Pro | OpenAI | 400000 | $17.56 | 85.2 |
| 22 | Claude Opus 4.5 | Anthropic | 400000 | $19.42 | 87.7 |
| 23 | o3 Pro | OpenAI | 400000 | $17.89 | 85.8 |
| 24 | Gemini 3 Pro Max | Google DeepMind | 2000000 | $20.57 | 87.9 |
| 25 | Gemini 3 Pro Vision | Google DeepMind | 200000 | $17.59 | 85.5 |
| 26 | o4 Reasoning | OpenAI | 1000000 | $19.69 | 87.5 |
| 27 | Gemini 3 Pro Reasoning | Google DeepMind | 200000 | $19.69 | 87.5 |
| 28 | Claude 3.7 Sonnet Max | Anthropic | 128000 | $16.88 | 83.7 |
| 29 | GPT-5.1 | OpenAI | 2000000 | $19.01 | 86.0 |
| 30 | GPT-5.1 Reasoning | OpenAI | 400000 | $20.27 | 88.3 |
| 31 | Qwen3-Max Turbo | Alibaba Cloud | 1000000 | $17.70 | 84.7 |
| 32 | Gemini 3 Pro Pro | Google DeepMind | 200000 | $19.80 | 87.3 |
| 33 | GPT-5.1 Max | OpenAI | 2000000 | $20.50 | 87.9 |
| 34 | Qwen3-Max Pro | Alibaba Cloud | 400000 | $18.87 | 86.2 |
| 35 | o4 Vision | OpenAI | 200000 | $17.01 | 84.5 |
| 36 | Gemini 2.5 Pro Max | Google DeepMind | 200000 | $16.94 | 84.8 |
| 37 | Qwen3-Max Reasoning | Alibaba Cloud | 2000000 | $18.12 | 85.9 |
| 38 | Claude Sonnet 4.5 Turbo | Anthropic | 1000000 | $18.37 | 85.9 |
| 39 | DeepSeek-V3.2 Instruct | DeepSeek | 128000 | $5.91 | 83.7 |
| 40 | Kimi K2 Instruct | Moonshot AI | 1000000 | $5.80 | 82.3 |
| 41 | Claude Opus 4.5 Turbo | Anthropic | 2000000 | $19.57 | 85.7 |
| 42 | GPT-4.1 Max | OpenAI | 400000 | $17.34 | 84.7 |
| 43 | o4 | OpenAI | 2000000 | $17.74 | 85.4 |
| 44 | GPT-4.1 Turbo | OpenAI | 1000000 | $16.12 | 82.9 |
| 45 | GPT-5.1 Turbo | OpenAI | 2000000 | $19.19 | 86.1 |
| 46 | Grok 4 Pro | xAI | 1000000 | $19.00 | 85.4 |
| 47 | DeepSeek-V3.2 | DeepSeek | 1000000 | $5.88 | 84.0 |
| 48 | Grok 4 Max | xAI | 2000000 | $19.70 | 86.6 |
| 49 | Qwen3-Max Max | Alibaba Cloud | 400000 | $18.53 | 85.9 |
| 50 | Claude Sonnet 4.5 | Anthropic | 400000 | $17.84 | 86.0 |
Methodology
SWE-bench Verified is included as a coding benchmark with a typical score range of 0 to 100. The values in this index are deterministic modelled estimates generated for comparison at scale. They are not official leaderboard results and should not be quoted as vendor benchmarks.
Each model receives a score derived from its capability tier, release generation and benchmark difficulty. Scores are normalised to a 0–100 scale so that models from different providers can be placed side by side. The average shown is the arithmetic mean across all 600 scored models.
Read the leaderboard as a relative map. A five-point gap is meaningful at the top and less meaningful in the middle of the distribution. Pair the score with price and context data before choosing a model.
How to use this leaderboard
- Use the top 10 to shortlist candidates, not to declare a winner.
- Check whether the leader is open weights or proprietary before assuming you can deploy it.
- Compare the score against input and output price on each model page.
- Look at the model's scores on related benchmarks before trusting a single number.
FAQ
What is SWE-bench Verified?
Real GitHub issues resolved by an AI coding agent, verified by human reviewers.
Who leads SWE-bench Verified?
GPT-5.2 Pro from OpenAI leads this index with a score of 81.7, followed by Claude Opus 4.5 Pro at 81.1.
What is a good SWE-bench Verified score?
The top quartile starts around 67.8. The index average is 57.5, so a score above 67.8 is a strong result relative to this dataset.
How are SWE-bench Verified scores calculated?
The values are modelled estimates on a 0–100 scale, generated from capability tier, release generation and benchmark difficulty. They are not official leaderboard results.
Which models are included?
All 600 models in the index are scored on SWE-bench Verified, across 40 providers and every model category.