← All visual benchmarksEvalPlus / 2026-10-23
EvalPlus · CODING

MBPP+ code tests

Pass@1 on MBPP+ rigorous test cases.

Hover or tap a bar to inspect. Select up to 3 models to highlight and compare.
Model scores10 models
MBPP+ code tests. Zoomed horizontal axis, from 71.92 to 81.28ZOOMED X-AXIS · 71.92 – 81.28 · NOT FROM ZERO71.9274.2676.6078.9481.28O1 Preview (Sept 2024)O1 Preview (Sept 2024) · 80.2 pass@1 (%) · zoomed horizontal axis80.2O1 Mini (Sept 2024)O1 Mini (Sept 2024) · 78.8 pass@1 (%) · zoomed horizontal axis78.8Qwen2.5-Coder-32B-InstructQwen2.5-Coder-32B-Instruct · 77 pass@1 (%) · zoomed horizontal axis77DeepSeek-Coder-V2-InstructDeepSeek-Coder-V2-Instruct · 75.1 pass@1 (%) · zoomed horizontal axis75.1Gemini 1.5 Pro 002Gemini 1.5 Pro 002 · 74.6 pass@1 (%) · zoomed horizontal axis74.6Claude Sonnet 3.5 (June 2024)Claude Sonnet 3.5 (June 2024) · 74.3 pass@1 (%) · zoomed horizontal axis74.3DeepSeek-V2.5 (Nov 2024)DeepSeek-V2.5 (Nov 2024) · 74.1 pass@1 (%) · zoomed horizontal axis74.1GPT-4-Turbo (Nov 2023)GPT-4-Turbo (Nov 2023) · 73.3 pass@1 (%) · zoomed horizontal axis73.3claude-3-opus (Mar 2024)claude-3-opus (Mar 2024) · 73.3 pass@1 (%) · zoomed horizontal axis73.3DeepSeek-V3 (Nov 2024)DeepSeek-V3 (Nov 2024) · 73 pass@1 (%) · zoomed horizontal axis73
Choose a bar for details.

How to read this chart

EvalPlus pass@1 scores are from official reports; prompting and model configurations may differ.

Score unit: pass@1 (%). Values can only be compared within the same task and protocol.

Data retrieved: 2026-10-23. Higher values indicate better results for the reported metric.

Source: EvalPlus · Apache-2.0. Redrawn from released data; reduced to selected models and rounded. Not affiliated with the source organization.

Explore related charts