Self-host LiteLLM Proxy: virtual keys, budgets, and how it compares to New API and Magpie

Putting raw upstream API keys directly into every app and developer laptop makes it impossible to cap what a single service can spend, and rotating a key means updating every project by hand. LiteLLM is one of the most widely used open-source LLM proxies in the Python and AI engineering ecosystem: it wraps more than 100 model providers behind one OpenAI-compatible endpoint, with a /ui admin dashboard, Virtual Keys, per-key budgets and spend logs.

This guide uses Zenavo's LiteLLM template to deploy LiteLLM together with PostgreSQL on your own Linux server, and compares it with New API and Magpie. Your server, Zenavo and upstream model APIs are billed separately.

1. Deploy LiteLLM and PostgreSQL

Connect a Linux server to Zenavo (at least 1 GB of memory is recommended), open the LiteLLM template, choose your server and deploy.

The project starts with two services connected over the project private network:

Service / Variable	Value or purpose
litellm	Image ghcr.io/berriai/litellm:v1.109.1, HTTP port 4000
postgresql	Image postgres:16, database litellm, persistent volume /var/lib/postgresql/data
LITELLM_MASTER_KEY	Auto-generated sk-... master key for signing in to /ui and calling admin APIs
LITELLM_SALT_KEY	Auto-generated encryption salt used to encrypt upstream API keys stored in PostgreSQL
STORE_MODEL_IN_DB	True, so models added in /ui are persisted in PostgreSQL

Never change LITELLM_SALT_KEY after adding credentials: LiteLLM uses LITELLM_SALT_KEY to encrypt and decrypt provider API keys in the database. Changing it later makes your stored provider keys unreadable. Copy both LITELLM_MASTER_KEY and LITELLM_SALT_KEY into your password manager (such as self-hosted Vaultwarden) right after deployment.

2. Bind a domain and sign in to /ui

1. On the litellm service's Networking tab, bind a custom domain such as llm.example.com to port 4000 and add a DNS A record pointing to your server. See Domains & HTTPS.
2. On the litellm service's Variable tab, copy LITELLM_MASTER_KEY.
3. Open https://llm.example.com/ui and sign in with username admin and LITELLM_MASTER_KEY.

postgresql only talks to litellm over the private network and does not need public access.

3. Add models and issue virtual keys

Add upstream models

Because the template sets STORE_MODEL_IN_DB="True", you don't need to edit a YAML config file:

1. In /ui, open Models → Add Model.
2. Pick a provider (OpenAI, Anthropic, DeepSeek, Gemini, Azure, Bedrock, etc.), enter the model alias your clients will request (such as claude-sonnet or gpt-4o) and paste the upstream API key.
3. Save and send a test prompt in Test Key / Playground to confirm it responds.

Issue budget-capped Virtual Keys

Don't use LITELLM_MASTER_KEY in application code or daily tools. Open Virtual Keys → Create New Key:

• Create a separate key per app, teammate or tool (for example nextjs-prod, open-webui, claude-code-alex);
• Restrict which models the key may call;
• Set a Max Budget (for example $20), a budget reset period (daily or monthly) and rate limits (TPM / RPM).

4. Connect apps, coding agents and chat UIs

Any SDK or client that accepts a custom OpenAI Base URL connects straight to LiteLLM:

• Application backends (OpenAI SDK / LangChain / Vercel AI SDK): set Base URL to https://llm.example.com/v1 and API Key to a virtual key. If your app runs in the same Zenavo project (such as a Next.js + PostgreSQL app), call http://litellm.zenavo.internal:4000/v1 over the private network instead.
• Claude Code and Codex CLI: LiteLLM also serves /v1/messages and /v1/responses; configure them with the same environment variables shown in the New API guide. To also inspect logs, deploy and roll back Zenavo services from Claude Code, see Deploy and debug Zenavo projects from Claude Code.
• Chat UIs and personal agents: in self-hosted Open WebUI, LobeHub or OpenClaw, point the OpenAI base URL at https://llm.example.com/v1 (or http://litellm.zenavo.internal:4000/v1 within the same project).

5. LiteLLM vs New API vs Magpie: which gateway fits your stack?

Zenavo's catalog offers three open-source LLM gateways, each built for a distinct workflow:

Gateway template	Core focus	Storage	Best for
LiteLLM	Engineering-first OpenAI proxy with virtual keys, budgets, guardrails and observability callbacks (Langfuse, OpenTelemetry)	PostgreSQL 16	Production AI app backends and engineering teams managing per-service budgets and spend logs
New API	Multi-user distribution and billing hub with user accounts, token quotas and channel load balancing	SQLite (/data) by default	Issuing per-person accounts and tokens across a team for Open WebUI, LobeHub, Claude Code and Codex
Magpie	Lightweight single-container multi-protocol gateway translating between Anthropic Messages, OpenAI Responses/Chat and Gemini, with failover across keys and subscriptions	/config volume	Solo developers and small teams running Claude Code and Codex CLI who want cross-protocol model routing and a /me self-service usage page

6. Backups and upgrades

LiteLLM stores its models, virtual keys and spend logs in the postgresql service:

1. Open the postgresql service's Backup & Restore tab to create and download database backups, or schedule automatic backups. See Backups & Restore.
2. Always keep a copy of LITELLM_SALT_KEY alongside your database backups — without the original LITELLM_SALT_KEY, a restored database cannot decrypt the provider keys inside it.
3. Before upgrading, back up PostgreSQL first, then update the image tag (default v1.109.1) under Settings → Source on the litellm service.

FAQ

Why does /ui return 502 right after deployment? On startup, LiteLLM waits for PostgreSQL and runs its Prisma database migrations; the template includes a 40-second health check grace period (start_period: 40). Wait until the service state turns Running, or open the built-in Zenavo Agent on the right edge of the console to inspect the startup logs.

Adding a model in /ui says it cannot save to the database. Verify that STORE_MODEL_IN_DB="True" is set in the litellm service's environment variables and that the postgresql service is Running.

A client gets BudgetExceededError. That virtual key has reached its configured Max Budget. Open Virtual Keys in /ui to raise its budget, reset its spend or configure a monthly reset cycle.

Can I run LiteLLM and New API together? Yes. Some teams manage upstream provider channels in New API and place LiteLLM in front of their application services for per-microservice budgets and Langfuse tracing, connecting the two over .zenavo.internal.

Deploy the LiteLLM template to put your model providers behind a PostgreSQL-backed proxy with per-key budgets in a few minutes.