OpenAI's new GPT-4.1 AI models focus on coding | TechCrunch

Impressive results and shows what Anthropic and Google are able to do on similar benchmarks. Notably they are even higher.

According to OpenAI’s internal testing, GPT-4.1, which can generate more tokens at once than GPT-4o (32,768 versus 16,384), scored between 52% and 54.6% on SWE-bench Verified, a human-validated subset of SWE-bench. (OpenAI noted in a blog post that some solutions to SWE-bench Verified problems couldn’t run on its infrastructure, hence the range of scores.) Those figures are slightly under the scores reported by Google and Anthropic for Gemini 2.5 Pro (63.8%) and Claude 3.7 Sonnet (62.3%), respectively, on the same benchmark.

Don't miss the footnote at the end though that these models still routinely fail tasks that would never trip up an average developer. Nonetheless, clearly there is power here to leverage and benefit from.