• LIVE
  • CLEAN
  • AUDIT
  • MANUAL

How to update model routing with your AI

Connect LiteLLM to your agent to manage your proxy endpoints in real time. You can swap out failing models or add new ones without touching a config file.

Ask AI about this page

Short answer

How can I change my model endpoints using an AI?

You tell your agent to add or remove specific models from your LiteLLM proxy. It uses direct tool calls to modify your routing configuration instantly. You'll see the changes reflected in your proxy traffic immediately.

Routing outcomes.

Where your changes land.

What happens when your agent modifies the proxy.

LIVE

New endpoints active

The agent uses create_model to register a new endpoint. Your traffic starts hitting the new model right away.

CLEAN

Broken models removed

When a deployment fails, the agent uses delete_model to pull it from the proxy. This stops requests from hitting dead endpoints.

AUDIT

Current config view

The agent uses get_model_info to check which models are currently active. You get a clear picture of your routing state.

MANUAL

Human oversight

The agent handles the technical execution, but you decide which models are safe to add or remove. You maintain final control over your spend and routing logic.

The workflow

What your AI does when <the thing arrives>.

Your agent acts as a bridge between your chat interface and the LiteLLM proxy configuration.

  1. Inspect current routing

    The agent checks the existing proxy setup to see what models are currently available for use.

    get_model_info

  2. Register new models

    When you want to expand your options, the agent adds the new model details to the proxy.

    create_model

  3. Remove failing endpoints

    If a specific deployment is throwing errors, the agent strips it from the routing table.

    delete_model

  4. Verify the change

    The agent runs a final check to ensure the routing table matches your requested state.

    get_model_info

Try it

Copy these to start.

Use these prompts to begin managing your LiteLLM proxy.

Starting points

These are basic commands. Swap out the model names and provider details for your specific setup.

LiteLLM Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_km5X3hHTZMwK6l421yD2vEs8yb47UeU5zl6r4SJ6/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — LiteLLM capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "litellm-llm-proxy-spend-tracking-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_km5X3hHTZMwK6l421yD2vEs8yb47UeU5zl6r4SJ6/mcp"
    }
  }
}
Copy into chat04

  • Show me all the models currently configured in my LiteLLM proxy.

  • Add a new model endpoint for openai/gpt-4o to my proxy.

  • Remove the failing anthropic/claude-3-opus deployment from my routing.

  • Check the details for the model named 'production-llama-3'.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect LiteLLM once, then ask.

One connection gives your agent full access to your proxy tools. Your credentials stay encrypted and secure.

Connect LiteLLM to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change my API keys?

    No. The agent can only manage model endpoints and routing via the provided tools.

  • 02

    Will the AI delete all my models by mistake?

    No. The agent only deletes specific models that you explicitly tell it to remove.

  • 03

    Does this work with any proxy?

    No. This specific MCP is built specifically for LiteLLM proxy management.

  • 04

    Can the AI monitor my total spend?

    No. While LiteLLM tracks spend, the tools in this MCP are for model routing and management only.

  • 05

    How do I know if a model was successfully added?

    The agent will use the info tool to verify the new model exists in the proxy after it creates it.

  • More questions about LiteLLM? The Connector page answers them. See everything the LiteLLM Connector can do

Connect LiteLLM to Claude, Cursor, ChatGPT & more