# Anyscale MCP for AI Agents MCP > The Anyscale MCP lets your AI client manage entire distributed machine learning environments through natural conversation. You can list models, generate vector embeddings for large text arrays, monitor deployed services, and check complex Ray cluster job statuses—all without opening a terminal or navigating a heavy cloud dashboard. ## Overview - **Category:** ai-frontier - **Price:** Free - **Tags:** distributed-computing, llm-inference, vector-embeddings, cluster-management, scalable-ai ## Description This connector connects your AI agent directly to the Anyscale environment, letting you manage both large-scale LLM queries and underlying backend infrastructure natively. Instead of logging into a clunky web portal just to check if a training job finished, you talk to your agent. It handles the complex background work for you. It provides tools to list active foundational models and run chat completions using specialized Anyscale LLMs. You can also generate semantic vector embeddings from text inputs on the fly. Furthermore, it lets you monitor deployed Ray services and query batch jobs to inspect their recent execution statuses and training metrics via conversation. If you're already using Vinkius for your other APIs, adding this MCP gives you a single point of control over your entire MLOps stack. ## Tools ### list_models Lists all foundational AI models available on your Anyscale Endpoints cluster. ### chat_completion Generates conversational replies by sending structured messages with roles (user, system, assistant) to Anyscale LLMs. ### text_completion Creates text completions using the general Anyscale API when you need foundational, non-conversational text generation. ### generate_embeddings Takes a piece of text and creates its corresponding semantic vector embedding array. ### list_services Retrieves an overview list of all currently deployed services on your Anyscale platform. ### get_service Fetches specific, detailed information about a single designated Anyscale service deployment. ### list_jobs Lists all running or completed batch and training jobs managed by your Ray cluster on Anyscale. ## Prompt Examples **Prompt:** ``` List all the foundational models I have deployed right now. ``` **Response:** ``` **✅ Models Found:** * `meta-llama/Llama-2-70b-chat-hf`: Active. Type: Chat. * `mistralai/Mistral-7B-Instruct-v0.1`: Active. Type: Chat. * `openai/gpt-4o`: Deprecated. Status: Pending Update. ``` **Prompt:** ``` What's the status of my last two training jobs? ``` **Response:** ``` I checked your recent cluster activity. Here are the details: | Job Name | Status | Completion Time | | :--- | :--- | :--- | | daily_retrain_v3 | ✅ Succeeded | 5 hours ago | | experimental_tuning_7b | ❌ Failed | Out of memory (Node 12) | You need to check the service metadata for Node 12. ``` **Prompt:** ``` Can you generate an embedding for this paragraph? ``` **Response:** ``` Sure thing. I passed the text to your configured model. The resulting array has 768 dimensions and is ready to use in your data pipeline. Here's a snippet of the first few floats: `[0.12, -0.55, 0.91, ..., 0.34]`. Let me know if you need it mapped somewhere specific. ``` ## Capabilities ### list_models Lists all foundational AI models available on your Anyscale Endpoints cluster. ### chat_completion Generates conversational replies by sending structured messages with roles (user, system, assistant) to Anyscale LLMs. ### text_completion Creates text completions using the general Anyscale API when you need foundational, non-conversational text generation. ### generate_embeddings Processes arrays of text and generates semantic vector embeddings that can be used for advanced search or RAG systems. ### list_services Retrieves an overview list of all currently deployed services on your Anyscale platform. ### get_service Fetches specific, detailed information about a single designated Anyscale service deployment. ### undefined Lists all running or completed batch and training jobs managed by your Ray cluster on Anyscale. ## Use Cases ### Checking Model Readiness After Deployment An MLOps Engineer needs to validate that a newly trained LLM is live. Instead of logging into the console dashboard and waiting for status lights to turn green, they ask their agent to list models, confirming the exact model ID is available for use. ### Retrieving Training Metrics Mid-Run A Data Scientist notices a job slowing down. Instead of searching through historical logs, they tell their agent to query the latest jobs, immediately seeing if the 'daily_retrain' run completed successfully or failed on specific nodes. ### Building a Search Index for Documentation A developer needs to index hundreds of technical documents. They use the MCP to generate vector embeddings for all text, feeding them directly into their data pipeline rather than running a separate embedding service script. ### Validating Service Health Before Go-Live A Backend Developer needs to ensure a specific microservice is healthy before traffic hits it. They use the agent's ability to retrieve details about a specific service, confirming endpoint configurations and operational status. ## Benefits - You can check the status of large-scale training jobs using the `list_jobs` tool, getting execution metrics without opening a separate terminal window. - Instead of manually checking multiple dashboards, you use the MCP to list all active models (`list_models`) and confirm they are ready for inference immediately. - Generating vectors is fast. The `generate_embeddings` capability processes large text arrays directly, which is critical for building RAG pipelines efficiently. - Debugging service issues is simpler. You just need to use the MCP's `get_service` function to pull up specific endpoint details in a conversation. - The ability to run conversational queries (`chat_completion`) means you interact with complex model outputs using plain language prompts, not API JSON structures. ## How It Works The bottom line is, you get a conversational layer over highly technical ML infrastructure management. 1. Subscribe to this MCP, providing your specific Anyscale API Key and Base URL. 2. Connect your preferred AI client (like Cursor or Claude) to the Vinkius catalog using your credentials. 3. Ask your agent to perform tasks—for example, 'What's the status of my latest training job?' The agent then invokes the necessary tools. ## Frequently Asked Questions **How does the Anyscale MCP help me check my cluster job status?** The Anyscale MCP lets you query your Ray batch jobs directly through conversation. Instead of opening a complex terminal dashboard, simply ask about recent job statuses to see if training succeeded or failed and why. **I need to find out which LLMs are available on my cluster using the Anyscale MCP?** You can use the MCP to list all active foundational models. It gives you a clean rundown of every deployed model, confirming its name and current status before you write a single line of code. **What if my service endpoint is having issues? Can Anyscale MCP help me debug it?** Yes, the MCP allows you to retrieve specific details about your deployed services. This means you can confirm the latest endpoint configurations and check the current health status of a microservice in plain language. **Does Anyscale MCP handle generating embeddings for my documents?** It does. You pass text to the MCP, and it generates semantic vector embeddings using your configured model. This makes preparing data for search or RAG pipelines much easier than running separate scripts. **How do I connect Anyscale MCP to my AI agent?** You subscribe to this MCP in the Vinkius catalog, providing your necessary Anyscale API keys. Your agent then handles all the communication with the cluster tools for you.