Routes LLM requests across 23+ providers through one OpenAI-compatible API, with automatic failover, load balancing, semantic caching, MCP tool access, and budget governance.
- Access
- APICLI
- Auth
- API Key
- Skills
- 5
Skills
Route Across Providers
Routes chat completion and embedding requests to 23+ model providers through a single OpenAI-compatible HTTP endpoint.
Fail Over Automatically
Falls back to backup providers, models, and API keys automatically when a request errors, with load balancing across keys.
Cache Responses Semantically
Serves cached responses when an incoming prompt is semantically similar to an earlier one, cutting token spend and latency.
Gateway MCP Tools
Connects models to external MCP tools such as filesystem, web search, and databases through the gateway MCP layer.
Enforce Budgets And Limits
Tracks usage and enforces hierarchical budgets, rate limits, and access control using virtual keys scoped to teams and customers.
Related Agents
Portkey
APIAI gateway with unified API for 200+ LLMs, plus observability, caching, fallbacks, and cost management. Drop-in…
Vercel AI Gateway
APIRoutes model calls from any app or coding agent across providers with one API key, adding fallbacks, request logs…
LiteLLM
API · CLIAI gateway unifying 100+ LLM providers under one OpenAI-compatible API. Cost tracking, guardrails, load balancing. pip…
Nango
APIOpen-source OAuth and API integration platform. 300+ pre-built integrations. Handles token refresh, rate limits, and…