APIEval-20

An open benchmark for AI agents that test APIs

API Developer Tools Artificial Intelligence發布於 2026年5月8日

什麼是 APIEval-20？

APIEval-20 is a black-box benchmark for API testing agents. Each agent gets only a JSON schema and one sample payload, then generates a test suite. We run those tests against live reference APIs with planted bugs and score bug detection, API coverage, and efficiency. Unlike LLM-as-judge evals, scoring is fully objective: a bug is either caught or it isn’t. Tasks span auth, errors, pagination, schemas, and multi-step flows. Open on Hugging Face.

截圖

還沒有評論，來搶沙發吧！