Inferbench

What is Inferbench?

InferBench is a vendor-neutral CLI tool that benchmarks local LLM inference engines like omlx and llama.cpp directly on your own hardware. Instead of relying on external metrics, it reports real, measured tokens per second. InferBench auto-detects engines, runs a fixed prompt set, and recommends the fastest configuration for your setup. It is self-hosted and Apache 2.0 licensed.