# Arize AI MCP for AI Agents AI Agent Connect > Arize AI lets you manage your ML and LLM observability through your AI agent. It handles telemetry ingestion, model monitoring, and automated evaluations. You can track data drift, check for hallucinations, and manage production environments without leaving your primary workspace. It's built for anyone needing to see how their models are actually performing in the wild. ## Overview - **Category:** friends-mcp - **Price:** Free - **Endpoint:** https://edge.vinkius.com/vk_preview_wKJSgGsb2EgJmks5PzM3V0jknb7nFZgbwB0CaRFP/ai-agent-connect - **Tags:** ml-observability, model-monitoring, data-drift, ai-performance, telemetry, troubleshooting ## Description Arize AI lets you manage your ML and LLM observability through your AI agent. It handles telemetry ingestion, model monitoring, and automated evaluations. You can track data drift, check for hallucinations, and manage production environments without leaving your primary workspace. Managing ML models usually means jumping between tabs and staring at complex dashboards to figure out why a prompt failed or why your data started drifting. This Connector changes that by letting your agent handle the heavy lifting of ML Ops. Instead of manually pulling metrics, you can ask your agent to tell you exactly how a specific model is performing in production or check for toxicity in your latest LLM outputs. It connects your Arize AI platform directly to your workflow, so you can push raw logs for analysis or trigger evaluation runs on the fly. Because Vinkius makes it easy to plug these capabilities into your existing setup, you get a dedicated ML Ops engineer living inside your chat interface. You get to stay in your code or your planning docs while your agent keeps an eye on the health of your models. You stay ahead of hallucination spikes and data drift because the information comes to you on demand. It turns your AI client into a constant monitor that understands your specific model configurations and deployment environments. ## Tools ### list_datasets List all static evaluation datasets available to you. This helps you see what benchmarks are ready for testing. ### create_dataset Create a dataset ### get_model Fetch details and metadata for a specific tracked model. You can see the inputs, outputs, and features for any model in your space. ### list_experiments List experiments ### list_projects List projects ### list_spans List spans ## Prompt Examples **Prompt:** ``` Show me the drift metrics for our fraud detection model. ``` **Response:** ``` ### Fraud-Detection-v2 Metrics | Metric | Value | Status | | :--- | :--- | :--- | | **Data Drift** | 0.12 | ✅ Stable | | **Prediction Drift** | 0.08 | ✅ Stable | | **Accuracy** | 98.4% | ✅ Stable | Your model is currently performing within expected parameters. Would you like to see the specific drift components? ``` **Prompt:** ``` Are there any active toxicity evaluations running? ``` **Response:** ``` I've checked your active runs. You have **2 evaluations** currently in progress: * **Toxicity-Check-Batch-42**: In Progress (45% complete) * **PII-Filter-Run-09**: Completed (Passed) Would you like me to list the detailed results for the completed run? ``` **Prompt:** ``` Push these three logs to the customer service bot. ``` **Response:** ``` I've successfully structured those 3 logs and pushed them to the **OpenAI-Customer-Service-Bot** using the `ingest_log` tool. **Summary:** * **Status:** Success * **Payloads:** 3 * **Destination:** Arize Space (Production) They are now available in your telemetry dashboard for drift analysis. ``` ## Capabilities ### Push telemetry logs to Arize Send raw prediction data and inferences directly into your observability platform. ### Fetch real-time model metrics Get immediate data on performance, quality, and prediction drift. ### Trigger automated evaluations Start new scoring loops for toxicity, PII filtering, or hallucinations. ### Browse organizational spaces Navigate between different projects, teams, and telemetry datasets. ### Retrieve evaluation datasets Access specific ground truth data for your testing and benchmarking. ### Fetch model metadata Get detailed schemas and input/output features for any tracked model. ## Use Cases ### Detecting Sudden Drift An engineer notices weird outputs and asks the agent to check `get_metrics` for the production model to see if data drift is the cause. ### Safety and Toxicity Audits A PM asks the agent to `list_evals` to see if the latest batch of prompts passed the toxicity and hallucination checks. ### Automated Log Ingestion A developer wants to push a batch of mocked responses and uses `ingest_log` to send them to Arize without opening a browser. ### Baseline Data Retrieval A data scientist asks the agent to `get_dataset` to find the ground truth for a new evaluation they are about to run. ## Benefits - Stop manual dashboard checking by using `get_metrics` to see performance data instantly. - Automate toxicity checks with `run_eval` to catch PII leaks before they hit production. - Keep your telemetry updated by using `ingest_log` to push data directly from your code. - Organize your work better by using `list_spaces` to switch between different projects. - Verify your deployment health with `list_environments` to see what's live in production. ## How It Works The bottom line is you get a hands-off way to manage ML observability through natural conversation. 1. Subscribe to the Arize AI MCP on Vinkius. 2. Enter your Arize API Key and Space ID Key. 3. Ask your agent to monitor model drift or list your active spaces. ## Frequently Asked Questions **How does Arize AI help with LLM hallucination?** It allows you to run automated evaluations against your logs and static datasets. You can use your agent to trigger these checks and see if your model is producing accurate or hallucinated content. **Can I use Arize AI to monitor data drift?** Yes. You can ask your agent to fetch real-time metrics for any tracked model to see if your data quality or prediction distributions have shifted. **How do I push logs to Arize using an agent?** You can simply tell your agent to push specific telemetry logs or prediction data. It will structure the logs correctly and send them to your Arize platform for analysis. **Does Arize AI support multi-environment tracking?** Yes, it can list and manage different environments like production, training, and verification. This helps you keep your monitoring organized across different deployment stages. **Can my agent run automated evaluations?** Absolutely. You can trigger custom evaluation runs for things like toxicity, PII filtering, and hallucination checks directly through a natural language command. **How do I see which models are currently tracked?** Just ask your agent to list your models. It will pull a list of all ML models and LLMs currently being monitored in your Arize spaces. **Can my AI automatically trigger a hallucination evaluation on a new dataset?** Yes! You can ask your agent to retrieve the specific Ground Truth dataset ID, formulate a testing payload, and invoke the `run_eval` tool natively. Arize will process the asynchronous scoring internally and log the evaluation securely. **How can I quickly check if a production model is experiencing data drift?** Just tell your agent: 'Fetch the primary metrics for model X'. The AI uses the `get_metrics` query to immediately surface latency degradation, prediction drift flags, and incoming data quality indexes without opening the browser. **Is it possible to track telemetry simultaneously for both local development and production environments?** Absolutely. Arize enforces strict separation using Spaces and Environments. You can instruct your AI agent to query the `list_environments` tool, figure out the sandbox ID, and push manual test logs strictly to the sandbox scope during debugging sessions, keeping production metrics clean. **How do I find my Arize API Key?** Log in to your account, navigate to **Settings** > **API**, and generate or copy your unique secret key. **Can I track model drift via AI?** Yes! Use the `list_experiments` tool to retrieve data on active model evaluations and track performance variations programmatically. **How do I retrieve telemetry traces?** Use the `list_spans` tool to retrieve high-fidelity execution spans and traces for your ML projects directly from the platform.