• ANALYSIS
  • MONITORING
  • RETRIEVAL
  • HUMAN

How to track ML experiment performance with your AI

Connect Arize AI to your agent to pull telemetry and metadata. You can then ask questions about model quality and specific experiment runs.

Ask AI about this page

Short answer

How can I track ML experiment performance with my AI?

Your AI client pulls telemetry and metadata from Arize AI to analyze model quality. You can compare experiment results and check for drift using natural language. You'll get a clear view of how your models are behaving.

Where your data lands.

The outcomes of your queries.

The AI uses your Arize AI data to produce these specific results.

ANALYSIS

Experiment comparisons

The AI pulls data from your runs to compare performance. You get a direct look at how different versions performed.

MONITORING

Model quality checks

Your agent reads telemetry to find drift or quality drops. It flags issues in model behavior immediately.

RETRIEVAL

Metadata lookups

The AI fetches specific model details and configuration settings. You can find the exact parameters used in a run.

HUMAN

Manual decisions

The AI identifies anomalies but won't change your model settings. You decide when to roll back or retrain.

The workflow

What your AI does when the data arrives.

The AI uses the Arize AI MCP to bridge your chat interface and your ML telemetry.

  1. Find experiments

    The AI scans your Arize account to find the specific experiment runs you want to discuss.

    list_experiments

  2. Get model info

    Once an experiment is identified, the AI pulls the specific configuration and metadata for that model.

    get_model

  3. Check performance

    The AI uses experiment data to calculate performance trends or identify specific errors.

    list_experiments

  4. Summarize results

    The AI reads model metadata to summarize metrics and highlight deviations from your baseline.

    get_model

Try it

Copy these to start.

Use these prompts to begin interacting with your ML data.

Starting points

Swap the bracketed text for your actual experiment names or model IDs.

Arize AI Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_wKJSgGsb2EgJmks5PzM3V0jknb7nFZgbwB0CaRFP/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Arize AI capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "arize-ai-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_wKJSgGsb2EgJmks5PzM3V0jknb7nFZgbwB0CaRFP/mcp"
    }
  }
}
Copy into chat04

  • List my recent experiments in Arize AI so I can pick one to analyze.

  • Get the details for the model named 'production-v2-classifier'.

  • Compare the performance of the last three experiments I ran.

  • Check the telemetry for my current model to see if there is any significant drift.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Arize AI once, then ask.

One link, login stays encrypted on our side, and the full walkthrough lives on the Connector page.

Connect Arize AI to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change my model settings?

    No. The AI can only read experiment and model data. You must manually update your models in Arize AI.

  • 02

    Can the AI delete my experiments?

    No. This MCP is read-only for telemetry and metadata. It cannot delete or modify any records.

  • 03

    Does the AI see all my telemetry?

    The AI only accesses the specific experiment and model data you ask it to retrieve.

  • 04

    How does the AI find the right model?

    You can name a specific model, or the AI can list your available models to find the right one.

  • 05

    Can I use this with Cursor?

    Yes, you can use this with any MCP-compatible client including Cursor, Claude, or Windsurf.

  • More questions about Arize AI? The Connector page answers them. See everything the Arize AI Connector can do

Connect Arize AI to Claude, Cursor, ChatGPT & more