• DATA
  • ANALYSIS
  • INSIGHT
  • HUMAN

How to compare model performance with your AI

Connect DataRobot to your AI client to pull specific performance data from your active projects. Once connected, you can ask your agent to run comparisons across different model versions.

Ask AI about this page

Short answer

How can I compare model performance with my AI?

Your AI pulls training and validation scores directly from your DataRobot projects. You can then ask it to identify which model has the best metrics for your specific use case. You get a clear breakdown of how each model stacks up.

The results you get

What your comparison yields

The AI handles the data retrieval so you can focus on the decision.

DATA

Raw metric extraction

The AI pulls specific validation scores and training metrics for every model in a project. You get the exact numbers without manual exports.

ANALYSIS

Side-by-side comparison

Your agent organizes the data to show how different models perform against each other. It highlights the differences in accuracy or error rates.

INSIGHT

Performance summaries

Instead of staring at tables, you get a plain English summary of which model is winning. The AI points out the specific metrics that matter most.

HUMAN

Final model selection

The AI provides the data, but you make the final call on which model to deploy. You retain full control over the project decisions.

The workflow

What your AI does when the request arrives.

The AI moves from project identification to specific model data retrieval.

  1. Locate the project

    The AI finds the specific DataRobot project you are working on to ensure it's looking at the right data.

    get_project

  2. Identify models

    Once the project is found, the AI identifies all the different models currently sitting in that project.

    get_model

  3. Extract metrics

    The AI pulls the raw training and validation scores for each identified model.

    get_model

  4. Compare results

    The AI compares the scores and presents the comparison to you in your chat interface.

    get_model

Try it

Copy these to start.

Use these prompts to kick off your analysis.

Starting points

Swap out the project names and model IDs with your actual DataRobot identifiers.

DataRobot Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_WX1YGmPzxHEF3itbazDacfMS9v73zZdwIlCWtf4z/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — DataRobot capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "datarobot-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_WX1YGmPzxHEF3itbazDacfMS9v73zZdwIlCWtf4z/mcp"
    }
  }
}
Copy into chat04

  • Get the metrics for all models in my 'Customer_Churn_v2' project.

  • Compare the validation scores for the 'XGBoost_Model' and the 'RandomForest_Model' in my current project.

  • Which model in my 'Fraud_Detection' project has the best accuracy score?

  • List the training metrics for the model named 'Linear_Regression_01' in my active project.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect DataRobot once, then ask.

Just link your DataRobot account through Vinkius. Your credentials stay encrypted, and you can immediately start asking your AI about your project data.

Connect DataRobot to your AI

FAQ

How this task behaves

  • 01

    Can the AI change my model settings?

    No. The AI can only read metrics and project data. It cannot modify your models or change any project configurations.

  • 02

    Can the AI delete a model from my project?

    No. This MCP is read-only for model data. It cannot delete or overwrite anything in DataRobot.

  • 03

    Do I need to export CSVs first?

    No. The AI pulls the raw data directly from the project using the available tools.

  • 04

    Can I compare models from two different projects?

    Yes, as long as you tell the AI which specific models from which projects you want to compare.

  • 05

    Does the AI see all my DataRobot data?

    The AI only sees the data it needs to answer your specific request based on the project and model you identify.

  • More questions about DataRobot? See everything the DataRobot Connector can do

Connect DataRobot to Claude, Cursor, ChatGPT & more