SWE-Bench tainted by answer leakage; real pass rates significantly lower

Article URL: https://arxiv.org/abs/2410.06992

Comments URL: https://news.ycombinator.com/item?id=43130732

Points: 213

# Comments: 63

https://arxiv.org/abs/2410.06992

Établi 19h | 21 févr. 2025 à 20:50:13

Connectez-vous pour ajouter un commentaire

Autres messages de ce groupe

Binned staff, slashed stock options. What's next? Ah yes, bigger C-suite bonuses

Article URL: https://www.theregister.com/2025/02/22/meta_pumps_executive_bonuses/

Comments URL:

22 févr. 2025 à 15:30:07 | Hacker news

Show HN: I Built a Visual Workflow Automation Platform – FlowRipple

FlowRipple is designed to streamline and automate business processes with ease. Whether you're a developer, business owner, or marketer, our platform lets you build custom workflows that can be tr

22 févr. 2025 à 15:30:06 | Hacker news

Ancient switch to soft food gave us overbite–the ability to pronounce 'f's,'v'

Article URL: https://www.science.org/content/article/ancient-switch-so

22 févr. 2025 à 13:10:09 | Hacker news

Doge says its savings total $55B, but analysis shows it's less

Article URL: https://www.washingtonpost.com/business/2025/02/22/doge-savings-found-list-analysis/

22 févr. 2025 à 13:10:08 | Hacker news

Twitch's new storage limits will purge swaths of Internet gaming history

Article URL: https://arstechnica.com/gaming/2025/02/history-eva

22 févr. 2025 à 10:50:03 | Hacker news

'The tyranny of apps': those without smartphones are unfairly penalised

Article URL: https://www.theguardian.com/money/2025/feb

22 févr. 2025 à 10:50:02 | Hacker news

DigiKey's Tariff Resources

Article URL: https://www.digikey.com/en/resources/tariff-resources

Comments URL:

22 févr. 2025 à 08:30:06 | Hacker news

Techie