What it does
Bifrost is an open-source gateway that sits between your applications or agents and the model providers they call. It exposes a single OpenAI-compatible API and routes requests to providers such as OpenAI, Anthropic, AWS Bedrock and Google Vertex. It is meant to keep AI applications running and under control. If a provider fails, requests can fall back to another provider or model, and traffic can be spread across several API keys. Responses can be cached by semantic similarity to cut cost and latency, and virtual keys, budgets, rate limits and access controls govern usage. It can also act as an MCP gateway so models can use external tools. It can be started with a single npx or Docker command, configured through a built-in web interface, and existing OpenAI, Anthropic or Google GenAI SDK code can be pointed at it by changing the base URL. A Go SDK is available for embedding it directly. The code is released under the Apache 2.0 license. Private enterprise deployments add capabilities such as clustering, adaptive load balancing and guardrails.
Capabilities
- One OpenAI-compatible API across many model providers
- Automatic failover and load balancing across keys and providers
- Semantic caching to reduce cost and latency
- Virtual keys, budgets, rate limits and access control
- MCP gateway for external tool calling
- Web UI, Prometheus metrics and request logging
When to use it
- You call several model providers and want a single endpoint with failover
- You need to track and cap AI spend per team or customer
- You want to switch an existing SDK-based app to a gateway with a base URL change
- You want to self-host the gateway in your own infrastructure
