# TrueFoundry MCP for AI Agents AI Agent Connect > TrueFoundry MCP lets you manage your entire LLM infrastructure from a single place. It acts as a unified gateway for your model requests and a deployment hub for your Connector instances. Instead of juggling dozens of API keys and scattered container configs, you get a single point of entry to route prompts, manage embeddings, and monitor your active deployments. It's built for teams that need to scale AI models without the infrastructure headache. ## Overview - **Category:** ai-frontier - **Price:** Free - **Endpoint:** https://edge.vinkius.com/vk_preview_f00nYwHsDWFv4EirgRZbXCnvKYC62empWE9xeumU/ai-agent-connect - **Tags:** llm-gateway, model-orchestration, ml-deployment, ai-infrastructure, api-proxy ## Description Imagine you're tired of managing a dozen different API keys for every different model your team wants to use. You're constantly worrying about which model has the best context window or which one is currently hitting rate limits. This Connector changes that by giving your AI client a single, secure endpoint to talk to. It handles the heavy lifting of routing your requests to the right place, whether you're looking for a high-powered chat model or a fast embedding service. You can see exactly how your models are performing, check their limits, and even spin up new backend containers without getting lost in complex orchestration logs. It's about moving away from a mess of fragmented pipelines and into a single, governed plane. When you use this through the Vinkius catalog, you're essentially getting a command center for your AI infrastructure. You get to focus on building features while the gateway handles the routing, security, and scaling. It strips out the fragmentation of multiple LLM pipelines and multiple MCP tool servers into a single secure plane. ## Tools ### truefoundry_deploy_mcp_server Start a new backend container process using the TrueFoundry service mesh. Use this to spin up new tools on your infrastructure. ### truefoundry_generate_embeddings Create semantic vectors using a single, unified abstraction layer. This keeps your embedding logic consistent across all apps. ### truefoundry_get_deployment_status View detailed metrics and status for your orchestration matrix. It helps you see real-time usage and health. ### truefoundry_get_mcp_server_info Get the specific JSON metadata for a registered TrueFoundry tool. This is useful for inspecting tool schemas. ### truefoundry_list_deployments See all the backend topologies currently running for your team. Use this to keep track of your active services. ### truefoundry_list_gateway_models See every foundation model available through the unified AI gateway. It shows you exactly what's accessible. ### truefoundry_list_mcp_servers Get a full registry mapping of all available MCP tools in your environment. This gives you a bird's eye view of your tools. ### truefoundry_run_gateway_chat Send a chat query through the gateway while keeping your original keys isolated. This handles the routing for you. ## Prompt Examples **Prompt:** ``` What models are available to me through the gateway? ``` **Response:** ``` Here are the foundation models currently accessible via your TrueFoundry gateway: | Model | Provider | Context Limit | Status | | :--- | :--- | :--- | :--- | | **GPT-4o** | OpenAI | 128k | Active | | **Claude 3.5 Sonnet** | Anthropic | 200k | Active | | **Llama 3** | Meta | 8k | Active | *All models are currently routed through the unified proxy.* ``` **Prompt:** ``` I need to deploy a new MCP tool for my database. ``` **Response:** ``` To deploy a new MCP tool, you can use the deployment orchestrator. Here is the plan: 1. **Identify the image**: Choose your database MCP node-image. 2. **Trigger Deployment**: I can initiate the process to spawn a new backend container on your cluster. 3. **Verify Status**: Once triggered, I'll monitor the logs to ensure the topology is mapped correctly. Would you like me to start the deployment for the 'supabase-mcp' image now? ``` **Prompt:** ``` How is my current deployment doing? ``` **Response:** ``` Checking your active deployments... **Current Status:** - **Active Nodes**: 4 - **Average Latency**: 869ms - **Usage Metric**: 74% of cluster capacity - **Isolation Limits**: All bounds are within safe parameters. Everything looks stable. I can provide more detailed metrics on specific nodes if you need. ``` ## Capabilities ### Route prompts to multiple models Send requests to different providers through one endpoint. ### Manage model embeddings Convert text to vectors using a unified channel. ### Discover gateway models Check the limits and context of available models. ### Orchestrate MCP containers Deploy new backend processes onto your infrastructure. ### Monitor active deployments See real-time status and usage metrics for your running services. ### List MCP schemas View the registry of all available tool schemas in your environment. ### Execute chat streams Route live chat contexts without needing individual API keys. ## Use Cases ### Switching providers due to price hikes A Platform Engineer needs to move from one LLM provider to another because of a price hike. They use truefoundry_list_gateway_models to find an alternative and update the route instantly. ### Deploying tools without managing servers A developer wants to deploy a new MCP tool but doesn't want to manage the underlying server. They use truefoundry_deploy_mcp_server to spin up a container on the TrueFoundry mesh. ### Identifying bottlenecks in production A team is hitting rate limits on their primary model. They use truefoundry_get_deployment_status to identify the bottleneck and see which models have remaining capacity. ### Standardizing embeddings across apps An AI Architect wants to ensure all internal apps use the same embedding logic. They point all apps to truefoundry_generate_embeddings to create a single source of truth for vectors. ## Benefits - Stop juggling dozens of API keys by using truefoundry_run_gateway_chat to route requests through a single secure endpoint. - Deploy new backend services instantly with truefoundry_deploy_mcp_server to manage your infrastructure without manual container overhead. - Get a clear picture of your system health using truefoundry_get_deployment_status to see real-time metrics and usage limits. - Simplify your model selection by using truefoundry_list_gateway_models to see every available provider in one list. - Secure your data by using truefoundry_generate_embeddings to handle semantic vectors through a unified, private channel. - Keep your toolset organized by using truefoundry_list_mcp_servers to see every available schema in your registry. ## How It Works The bottom line is you get a single gateway that hides the complexity of multiple model providers and infrastructure setups. 1. Get your personal access token from the TrueFoundry settings. 2. Connect your specific cluster URL to your AI client. 3. Start sending requests to the proxy routes to isolate your code from vendor APIs. ## Frequently Asked Questions **How does TrueFoundry help me manage multiple LLMs?** It provides a unified gateway that routes your requests to different providers. You only need to connect your AI client to one endpoint instead of managing separate keys and configurations for every model. **Can I use TrueFoundry to deploy my own Connector tools?** Yes, the Connector allows you to deploy new backend container processes directly onto your infrastructure. This lets you manage your entire tool topology from one place. **Does this help with LLM rate limits?** Yes, by using the gateway, you can monitor usage metrics and status in real-time. This helps you see exactly how much capacity you have left across your different providers. **Can I use it for embeddings?** Yes, it includes a unified abstraction for generating embeddings. This ensures that your text-to-vector logic stays consistent across all your different applications. **Is TrueFoundry MCP good for production environments?** It's specifically designed for production. It handles the orchestration, security, and monitoring that teams need to run reliable AI infrastructure at scale. **What happens to my API keys when I use this?** Your original vendor keys stay isolated. The Connector routes your requests through the TrueFoundry proxy, so your core codebase doesn't have to handle multiple keys directly. **Can I route conversational streams directly via the AI agent using the Universal Gateway?** Yes! You can orchestrate inferences parsing `run_gateway_chat` providing dedicated string formats mapping natively any enabled model. **Is it possible to monitor crashed services or container states?** Absolutely. Target the instance ID and emit `get_deployment_status` explicitly bounding execution limits and fetching live log matrices. **Are the deployment configuration variables isolated upon server launch?** Yes, using `deploy_mcp_server` dynamically provisions encapsulated boundaries. You stringify environment tokens seamlessly obscuring values into active runtimes only.