LiteLLM MCP. Control key lifecycle, routing, and spending across all LLMs.

LiteLLM Proxy & Spend Tracking: You run the whole show.

The LiteLLM Proxy gives you a single control layer for your entire LLM gateway. You ditch guessing about costs and manage everything—from API keys to model fallbacks—in one place using your AI client. Here's how it works.

Cost Management: Knowing Exactly Who's Spending What

You gotta track spending down to the user level, period. When you use create_user, you register a specific end-user identity into the proxy logs. This lets your agent monitor that unique token consumption against real-time usage data. Later, with get_user_info, you pull precise metrics for that end-user—it tells you their total tokens consumed and calculates exactly what that costs in USD.

For departmental tracking, you use create_team to set up an isolated profile. This tracks cost limits and operational boundaries specifically for a department or division. You can then call get_team_info to review the internal logic and established cost ceilings tied to that specific team ID.

Access Control & Isolation: Keys, Limits, and Boundaries

The proxy lets you build rock-solid isolation layers. If one microservice goes sideways, it shouldn't take down the whole stack. You generate unique sub-keys using generate_key. Each of these new proxy API keys can be configured with its own defined budget limits and rate controls, which is killer for multi-tenant setups.

When you need to check what key you're dealing with, or verify its current cost bounds, just run get_key_info against any specific API key. If a key gets compromised, don't sweat it; you can instantly eliminate it using delete_key, which removes the existing proxy API key entirely and prevents all associated costs.

Reliability & Fallbacks: Keeping Things From Crashing

Model failures are inevitable, but this setup handles them. You define complex model fallback paths—for instance, if OpenAI bails out, it automatically tries Anthropic or Groq. You check the exact sequence of models used when a primary provider fails using get_model_info, which returns a list of all configured routing fallbacks and shows exactly which models route to which providers.

The proxy also lets you manage the core model endpoints themselves. If a specific deployment is throwing 500 errors, you don't have to wait for it to fix itself; you use delete_model to deactivate that problematic LLM deployment immediately, keeping your routing path clean. You can also inject entirely new API endpoints into the proxy runtime for an untested or fresh model using create_model.

Operational Tools: The Full Control Panel

This setup lets you manage the infrastructure itself. By generating isolated keys and assigning them to specific teams, you enforce strict organizational spending boundaries right at the source. When your agent uses these tools, it's managing resources programmatically: It creates a team profile with cost tracking (create_team), registers individual users for auditing (create_user), generates highly restricted API access points (generate_key), and ensures that all operational costs are tied back to specific user or team IDs.

You get complete visibility into who consumed tokens, how many total tokens were used, and the corresponding USD cost, giving you full financial accountability over every single call made through your gateway.