← Founder Notes
Archive ·

The top coding agents have converged

21:17 ISTby Yethikrishna R

the top coding agents have converged: they solve the same 285 of 500 swe-bench verified problems and fail the same 51, and swapping the scaffold a model runs in moves its score by up to 29.8 points, more than the spread across the top thirty agents. openai says the benchmark no longer gives meaningful signal, and a fresh pro re-test drops every model by 20+ points. leaderboard order stopped meaning anything and most people haven’t noticed.

Share

Embed this note

<iframe src="https://founder.myndlabs.tech/notes/embed/the-top-coding-agents-have-converged-they-solve-Dd9Q9r4iH4a" width="480" height="420" style="border:0;max-width:100%" loading="lazy" title="The top coding agents have converged"></iframe>

Original

More notes