Mozilla AI Releases Otari: An Open-Source LLM Control Plane
Otari is a self-hosted, OpenAI-compatible proxy server that centralizes API key management, budget enforcement, and usage tracking across 40+ LLM providers. It allows developers to enforce spending limits before requests are fired.
Why it matters
It provides a unified gateway to control costs and proxy calls across multiple LLM providers locally.
TL;DR
- 01Centralize keys and budgets across 40+ LLM providers
- 02Enforce budgets before LLM provider costs are incurred
- 03Self-hostable with simple Docker/Postgres configuration
Deployment and Configuration
Otari can run in standalone mode via Docker, making it easy to test without complex dependencies:
docker run --rm -p 8000:8000 \
-e OTARI_CONFIG_YAML='default_pricing: true' \
mozillaai/otari:latestBudget and Security
When a request hits the gateway, Otari enforces budgets defined per-key before the request is sent to the provider. The gw- bearer tokens act as virtual keys; you can generate these via the management endpoint and revoke them instantly without rotating your master provider keys (e.g., your actual OpenAI secret).
Built-in Tools
Otari includes a sandboxed Python execution environment (otari_code_execution) and a web search service (otari_web_search). These can be toggled via the tools field in your API requests. For production usage, ensure you configure an external database backend like Postgres rather than the default ephemeral SQLite.
✓ When to use
- Centralizing multiple model providers in an app
- Preventing runaway costs in dev/staging environments
- Managing access for multiple internal teams
What to do today
- Run Otari locally using the Docker Quickstart command
- Set your master provider keys in the config file
- Generate a gateway key for your first test client
Sources