Oleksandr Kuzmenko, EditorPublished 5 min readAI-assisted · editor-reviewed
TL;DR
- Migrating internal coding traffic from commercial APIs to self-hosted models can lower tooling costs up to 10x.
- Tiered routing reserves expensive commercial LLM calls for complex architecture while local models handle boilerplate.
- Private hosting prevents proprietary source code and credentials from traversing third-party API providers.
Shifting from API Meters to Fixed Infrastructure
Deploying tools like Claude Code or Cursor across large engineering teams quickly leads to escalating token usage. Revolut's decision to drop reliance on external Claude API endpoints in favor of self-hosted open-weights models demonstrates an effective strategy for reigning in operational AI budgets by up to 10x.
Workload Partitioning Architecture
High-performing engineering teams adopt a hybrid tiered approach rather than relying on a single model tier:
- Tier 1 (Internal Open Models): Routine inline autocompletions, test suite scaffolding, and syntax transformations run on internal GPU clusters at fixed infrastructure costs.
- Tier 2 (Frontier Commercial APIs): Multi-repository refactoring, novel system design, and high-complexity agentic loops leverage Claude or GPT models on demand.
- Data Sovereignty: Proprietary algorithms and sensitive financial data remain strictly within company VPC boundaries, preventing external leakage.
Practical Implementation for Development Teams
Smaller teams can emulate this setup by deploying modern open-weights coding models using frameworks like vLLM or Ollama, pointing Cursor or local IDE extensions to private OpenAI-compatible endpoints.
When to use
- Mid-to-large engineering teams spending heavily on recurring per-token commercial coding subscriptions.
- Companies operating under strict banking, healthcare, or government data-isolation compliance mandates.
When NOT to use
- Solo developers whose monthly API bill is under $20, where hardware hosting overhead outweighs cost savings.
- Deep multi-repository autonomous reasoning workflows that still mandate frontier-class model intelligence.
What to do today
- Audit monthly team token spending across Cursor and coding agents to identify high-volume repetitive tasks.
- Deploy a test vLLM instance serving modern open-weights coding models on an internal server.
- Configure IDE tooling to route routine completions through private endpoints while keeping frontier fallbacks.
Step names are taken from What to do today.
Step names are taken from What to do today. Audit monthly team token spending across Cursor and coding agents to identify high-volume repetitive tasks.. Deploy a test vLLM instance serving modern open-weights coding models on an internal server.. Configure IDE tooling to route routine completions through private endpoints while keeping frontier fallbacks.
- Audit monthly team token spending across Cursor and coding agents to identify high-volume repetitive tasks.
- Deploy a test vLLM instance serving modern open-weights coding models on an internal server.
- Configure IDE tooling to route routine completions through private endpoints while keeping frontier fallbacks.