APIEval-20
An open benchmark for AI agents that test APIs
APIEval-20이란?
APIEval-20 is a black-box benchmark for API testing agents. Each agent gets only a JSON schema and one sample payload, then generates a test suite. We run those tests against live reference APIs with planted bugs and score bug detection, API coverage, and efficiency. Unlike LLM-as-judge evals, scoring is fully objective: a bug is either caught or it isn’t. Tasks span auth, errors, pagination, schemas, and multi-step flows. Open on Hugging Face.
스크린샷
?
아직 댓글이 없어요. 가장 먼저 남겨보세요!
APIEval-20에 대한 X의 실제 대화
X에 게시