ASTRA: HackerRank's coding benchmark for LLMs

We help companies hire & upskill developers. A customer recently asked: What % of HackerRank problems can LLMs solve? That got us thinking—how should hiring evolve when AI can translate natural language to code?

Our belief: AI will handle much of code generation, so developers will be assessed more on SDLC skills with AI assistants.

To explore this, we’re benchmarking LLMs on real-world software dev scenarios—starting with 65 unseen problems across 10 domains. Beyond correctness, we eval

18d | Hacker news

Поиск