Deep dive: Anthropic's Claude 3.7 Sonnet is the new king 👑 of code generation (but only with help), and DeepSeek R1 disappoints (Deep dives from the DevQualityEval v1.0)

[zimmski]

Read the whole article:

• https://symflower.com/en/company/blog/2025/dev-quality-eval-v1.0-anthropic-s-claude-3.7-sonnet-is-the-king-with-help-and-deepseek-r1-disappoints/

We love your continued support by purchasing the latest full results: https://buy.stripe.com/5kA3g962hfP0dGMeUX

• Full leaderboard with all exported metrics and summaries
• Full list of graphs including by company comparisons
• Your vote in what we put on the roadmap

Discussions:

• https://www.reddit.com/r/LocalLLaMA/comments/1jajoyo/insights_of_analyzing_100_llms_for_the/
• https://x.com/zimmskal/status/1900258721630130359
• https://x.com/zimmskal/status/1899110467294908607
• https://x.com/zimmskal/status/1895159997530530302
• https://x.com/zimmskal/status/1894315652766118263
• https://x.com/zimmskal/status/1892903280100946279
• https://x.com/zimmskal/status/1891546987365040512
• https://www.linkedin.com/posts/markus-zimmermann_insights-of-analyzing-100-llms-for-the-devqualityeval-activity-7306023879087415296-Gfru?utm_source=share&utm_medium=member_desktop&rcm=ACoAAAnsNzYBGMECYWMO6Xn5U3V4e6_ozs2hlDs