Hyperion

LLM API gateway, microsecond latency, 250× faster vs LiteLLM

Software Engineering Developer Tools Artificial Intelligence GitHub

Hyperion이란?

Hyperion is a high-performance LLM gateway with ~5µs overhead (~250× lower than LiteLLM). It handles smart routing, semantic + exact-match caching, rate limiting, budget control, PII redaction, and failover through a single OpenAI-compatible API. Built in Go with Redis, Qdrant, and ClickHouse.

스크린샷

아직 댓글이 없어요. 가장 먼저 남겨보세요!