RepoGym
Validate the task before you benchmark the coding agent
What is RepoGym?
Turn repository tasks into repeatable coding-agent evaluations. Define a task, check that doing nothing fails and the reference fix passes, then inspect scores and trajectories. Includes Python and JavaScript examples, composable graders, task mining and HTML reports. Free, MIT-licensed Python library. Start from the source checkout; local execution is for trusted code.
Screenshots
?
No comments yet. Be the first!