# LiteLLM (LLM Proxy & Spend Tracking) MCP for AI Agents AI Agent Connect > LiteLLM (LLM Proxy & Spend Tracking) lets you manage your entire LLM infrastructure through your AI client. You can generate API keys with specific budgets, monitor real-time spending across different teams, and configure complex model fallback paths. It gives you full control over how your agent interacts with providers while keeping costs in check and security tight. ## Overview - **Category:** ai-frontier - **Price:** Free - **Endpoint:** https://edge.vinkius.com/vk_preview_km5X3hHTZMwK6l421yD2vEs8yb47UeU5zl6r4SJ6/ai-agent-connect - **Tags:** llm-gateway, load-balancing, spend-tracking, model-routing, api-key-management, proxy ## Description Managing a fleet of LLM connections usually involves a mess of spreadsheets and manual audits. You have to check different dashboards to see who is spending what, manually rotate keys when they leak, and constantly update your code when a provider goes offline. It is a constant cycle of copy-pasting IDs and hoping you did not miss a budget overage. This Connector changes that by turning your LiteLLM Proxy into a conversational interface. Instead of hunting through configuration files, you can just tell your agent to set up a new routing path or create a team profile with a hard spending cap. It handles the heavy lifting of infrastructure management so you can focus on building. Because Vinkius makes it easy to connect these tools, you get a unified way to oversee your proxy runtime. You can instantly kill leaked keys, inject new deployments from AWS Bedrock or Azure, and see exactly who is burning through your budget in real-time. It moves the complexity of AI Ops into a simple dialogue where you are in the driver's seat of your model gateway. ## Tools ### create_model Add new routing endpoints like Bedrock or Llama 4 to your proxy. This helps you expand your model options without manual config changes. ### create_team Set up new organizational divisions with specific cost limits. This ensures each department stays within its allotted budget. ### create_user Link specific identities to your proxy logs for better tracking. This lets you see exactly which user is making which request. ### delete_key Remove a proxy key immediately to stop unauthorized usage. This is your fastest way to respond to a security breach. ### delete_model Delete a deployment path to prevent errors from reaching your app. Use this to prune broken or deprecated routes. ### generate_key Create a new API key for a specific team or microservice. This allows for easy permissioning without sharing master keys. ### get_key_info Check the budget and configuration for an existing proxy key. Use this to verify that a key has the correct limits. ### get_model_info View the full fallback path for a specific model. This helps you understand your routing logic at a glance. ### get_team_info See the internal logic and user bounds for a team. This provides a bird's-eye view of team-level permissions. ### get_user_info Get the total USD consumed by a specific end-user. This is essential for auditing and individual cost tracking. ## Prompt Examples **Prompt:** ``` Check the spending for the 'Marketing' team. ``` **Response:** ``` I've pulled the latest data for the 'Marketing' team. They have spent a total of **$142.50** this month. Their remaining budget for the period is **$8.50**. Would you like me to generate a new key for them or adjust their limit? ``` **Prompt:** ``` What happens if our primary OpenAI endpoint goes down? ``` **Response:** ``` Your current configuration for the **'gpt-4'** model is set up with the following fallback path: * **Primary:** OpenAI * **Fallback 1:** Anthropic (Claude 3) * **Fallback 2:** Groq (Llama 3) If OpenAI fails, your agent will automatically route requests to Anthropic. ``` **Prompt:** ``` I need a new key for the 'Data-Mining' microservice with a $200 limit. ``` **Response:** ``` Generating your new key now... Done. Here is your new proxy key: `sk-litellm-7x92...`. This key is now associated with the **'Data-Mining'** team and has a hard monthly budget limit of **$200.00**. ``` ## Capabilities ### Create team spending limits Set hard budget caps for different departments to prevent overages. ### Build model fallback paths Define what happens when your primary provider fails automatically. ### Generate isolated API keys Create unique keys for specific microservices or teams to keep access separate. ### Audit real-time spending See exactly how much each user is spending in USD at any moment. ### Inject new routing endpoints Add new providers to your proxy without any downtime. ### Revoke leaked credentials Instantly delete keys to secure your infrastructure from unauthorized use. ## Use Cases ### Adding a new model endpoint A platform engineer needs to add a new Llama 4 endpoint to their Bedrock setup and wants to do it via a chat prompt instead of editing a YAML file. ### Auditing a spending spike An AI Ops lead notices a spike in spending and needs to find out which team is responsible using get_user_info. ### Provisioning dev keys A developer needs a new API key for a test microservice but does not have admin rights; they ask the agent to generate_key for them. ### Emergency key revocation A security lead detects a leaked key and needs to delete_key immediately to prevent a massive bill. ## Benefits - Stop overspending by using create_team to set hard budget caps on specific departments. - Ensure uptime by configuring multi-provider fallback paths with get_model_info. - Secure your infrastructure by instantly revoking compromised credentials with delete_key. - Simplify dev workflows by using generate_key to give teams their own isolated access. - Get full visibility into costs with get_user_info to see exactly who is using the most tokens. - Update your tech stack without downtime by injecting new providers via create_model. ## How It Works The bottom line is you get a conversational control panel for your LLM gateway without leaving your workspace. 1. Subscribe to this Connector on Vinkius. 2. Enter your LiteLLM API URL and Master Key. 3. Start managing your proxy via your AI client. ## Frequently Asked Questions **How does LiteLLM (LLM Proxy & Spend Tracking) help with costs?** It lets you set hard budgets for different teams and users. You can see exactly who is spending what in real-time, making it easy to catch overages before they become a problem. **Can I use LiteLLM (LLM Proxy & Spend Tracking) to manage multiple providers?** Yes, it connects to your proxy to manage fallbacks across providers like OpenAI, Anthropic, and others. You can see your routing paths and add new ones easily. **How do I secure my keys with LiteLLM (LLM Proxy & Spend Tracking)?** You can use the Connector to instantly delete leaked keys or generate new, isolated keys for specific microservices. This keeps your main infrastructure safe from unauthorized access. **Does LiteLLM (LLM Proxy & Spend Tracking) work with Claude or Cursor?** Yes, you can connect it to your preferred AI client to manage your LiteLLM gateway through natural conversation. It works with any MCP-compatible client. **Can I see which models my teams are using most?** Yes, by checking user info, you can see the total spend and which specific models are being consumed the most by different identities. **Is LiteLLM (LLM Proxy & Spend Tracking) good for large organizations?** It is ideal for teams that need organizational isolation. You can create team profiles to track costs and operational boundaries for different divisions. **Can I check the budget and rate limits for a specific proxy key?** Yes. Use the `get_key_info` tool with the specific Key ID. Your agent will retrieve the exact rate limits, budget constraints, and current RPM usage associated with that token. **How do I see the model fallback paths configured in my proxy?** The `get_model_info` tool allows your agent to extract the global model directory. You'll see the exact fallback chains (e.g., if OpenAI fails, use Anthropic) and the physical endpoints assigned to each model name. **Can my agent create a new team to track specific division costs?** Absolutely. Use the `create_team` tool and provide a JSON payload defining the team name and optional budget limits. Your agent will provision the new team identity in LiteLLM, allowing for precise organizational cost tracking.