How to Use the LiteLLM (LLM Proxy & Spend Tracking) MCP in LlamaIndex

Q: Can I index my proxy configurations into LlamaIndex using LiteLLM (LLM Proxy & Spend Tracking)?

Yes. You can query getkeyinfo and getteaminfo through the MCP server and feed the JSON outputs directly into your LlamaIndex vector store for natural language querying.

Q: How do I dynamically update model endpoints in LlamaIndex using LiteLLM (LLM Proxy & Spend Tracking)?

Your LlamaIndex agent can invoke createmodel to register new endpoints or deletemodel to clean up decommissioned ones. This ensures your index query engine always targets valid API endpoints.

Q: How does LiteLLM (LLM Proxy & Spend Tracking) track RAG user costs in LlamaIndex?

The server provides getuserinfo and createuser tools. Your LlamaIndex pipeline can call these to map vector search queries to specific end-user identities and monitor their exact USD consumption.

Q: Can I manage multi-provider fallback paths within my query engine?

Yes, you can use getmodelinfo to inspect fallback paths. If your primary generator fails, your index query engine can automatically route the prompt to the designated fallback model.

Index and query your LLM gateway budgets, models, and usage logs directly within your LlamaIndex RAG pipelines.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect LiteLLM (LLM Proxy & Spend Tracking) MCP to LlamaIndex

Create your Vinkius account to connect LiteLLM (LLM Proxy & Spend Tracking) to LlamaIndex and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup LiteLLM (LLM Proxy & Spend Tracking) with LlamaIndex

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Semantic search over gateway configurations

Stop guessing which API keys are active. This MCP server lets LlamaIndex index the outputs of `get_key_info` and `get_team_info` directly into your vector store. Your users can then query their current budget limits using plain English. By feeding the configuration data straight into your index, your RAG agent always knows the exact state of your gateway. It uses real-time metadata instead of relying on outdated static files or hardcoded environment variables.

Dynamic model provisioning for RAG

When your LlamaIndex pipeline needs a specialized embedding or generation model, it doesn't need a manual config update. The agent calls `create_model` to register a new endpoint on the fly. If a model becomes deprecated, the agent runs `delete_model` to avoid routing errors. The agent can query `get_model_info` to check fallback paths before executing a retrieval task. This guarantees that your semantic search queries always route to an active, healthy model endpoint.

Indexing spend logs in this MCP Server

Keep a searchable history of how much your RAG queries are costing. Your pipeline can periodically run `get_user_info` and index the precise USD consumption data via this MCP Server. This makes it easy to build a natural language interface over your infrastructure bills. You can also call `create_user` or `create_team` to map end-user identities directly to your index metadata. This maps every vector query directly to the specific user or team that triggered it.

Setup guide

Set up LiteLLM (LLM Proxy & Spend Tracking) MCP in LlamaIndex

Prerequisites

Python 3.10+ installed
llama-index-tools-mcp package
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Run pip install llama-index-tools-mcp llama-index-llms-openai. The MCP tools package provides BasicMCPClient and McpToolSpec.
2

Connect with BasicMCPClient
Point BasicMCPClient to your Vinkius endpoint URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. Supports SSE and Streamable HTTP transports.
3

Convert to LlamaIndex tools
Call mcp_tool_spec.to_tool_list_async() to convert all LiteLLM (LLM Proxy & Spend Tracking) MCP tools into native FunctionTool objects that any LlamaIndex agent can use.
4

Run with any LLM
Create a FunctionAgent with the tools and your preferred LLM. Swap OpenAI for Anthropic, Gemini, or any LlamaIndex-supported provider.

agent.py

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI

# Connect to the MCP
mcp_client = BasicMCPClient(
    "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)
mcp_tool_spec = McpToolSpec(client=mcp_client)

# Convert MCP tools to LlamaIndex tools
tools = await mcp_tool_spec.to_tool_list_async()

# Create and run the agent
agent = FunctionAgent(
    tools=tools,
    llm=OpenAI(model="gpt-4o"),
    system_prompt="You have access to LiteLLM (LLM Proxy & Spend Tracking) tools.",
)
response = await agent.run("List recent LiteLLM (LLM Proxy & Spend Tracking) data")

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by LiteLLM. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect LiteLLM (LLM Proxy & Spend Tracking) now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about LiteLLM (LLM Proxy & Spend Tracking) MCP in LlamaIndex

Yes. You can query `get_key_info` and `get_team_info` through the MCP server and feed the JSON outputs directly into your LlamaIndex vector store for natural language querying.

Your LlamaIndex agent can invoke `create_model` to register new endpoints or `delete_model` to clean up decommissioned ones. This ensures your index query engine always targets valid API endpoints.

The server provides `get_user_info` and `create_user` tools. Your LlamaIndex pipeline can call these to map vector search queries to specific end-user identities and monitor their exact USD consumption.

Yes, you can use `get_model_info` to inspect fallback paths. If your primary generator fails, your index query engine can automatically route the prompt to the designated fallback model.

Your keys and spending logs are handled in an ephemeral V8 isolate on Vinkius using this MCP Server. The server uses a zero-trust model, meaning credentials and usage data are never stored on the platform and only exist in memory during active tool execution.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript