• INVENTORY
  • DETAILS
  • METRICS
  • HUMAN

How to audit model performance metrics with your AI

Connect H2O.ai to your AI client to pull model data through chat. You get direct access to training configurations and performance stats without leaving your interface.

Ask AI about this page

Short answer

How can I check my model metrics using AI?

Your AI client uses H2O.ai tools to fetch model lists and specific configuration details. You can ask for performance stats or setup info for any trained model in your account. Here is what you can do.

The audit outcomes.

What you get from the AI.

The AI pulls data from H2O.ai and organizes it for you.

INVENTORY

Model list retrieval

The AI identifies all available models in your H2O.ai account. You get a clear view of what's currently trained and ready.

DETAILS

Configuration deep dives

The AI pulls specific parameters and settings for a single model. You can inspect how a model was built without manual searching.

METRICS

Performance evaluation

The AI retrieves performance metrics directly. You can compare accuracy or other stats through natural language questions.

HUMAN

Manual model tuning

The AI only reads and retrieves data. You still need to log into the H2O.ai dashboard if you want to change hyperparameters or retrain a model.

The workflow

What your AI does when you ask for model data.

The AI acts as a bridge between your chat interface and your H2O.ai environment.

  1. Identify models

    The AI scans your H2O.ai account to find all existing trained models.

    list_models

  2. Fetch specific data

    Once you pick a model, the AI pulls its exact configuration and performance stats.

    get_model

  3. Analyze metrics

    The AI interprets the retrieved performance data to answer your specific questions.

    get_model

  4. Report findings

    The AI presents the model details in a readable format within your chat window.

    list_modelsget_model

Try it

Copy these to start.

Use these prompts to begin your audit.

Starting points

Replace the model names with the actual IDs from your H2O.ai account.

H2O.ai Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_joJzJEG0t6WyiAGy91nhlV2uG13UXBw7Ejke9uIr/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — H2O.ai capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "h2oai-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_joJzJEG0t6WyiAGy91nhlV2uG13UXBw7Ejke9uIr/mcp"
    }
  }
}
Copy into chat04

  • List all the models currently in my H2O.ai account.

  • Show me the performance metrics for the 'customer_churn_v2' model.

  • What are the configuration details for my 'fraud_detection_final' model?

  • Compare the performance of the 'sales_forecast' model against its training settings.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect H2O.ai once, then ask.

One connection gives your AI access to your H2O.ai models. Your credentials stay encrypted and secure on our side.

Connect H2O.ai to your AI

FAQ

How the audit behaves

  • 01

    Can the AI retrain my models?

    No. The AI can only retrieve information about existing models. You must perform training tasks within H2O.ai.

  • 02

    Can I delete a model through the chat?

    No. The AI does not have the capability to delete or modify models in your H2O.ai account.

  • 03

    How does the AI find my models?

    It uses the H2O.ai tools to list and fetch data from your specific account.

  • 04

    Does this work with any AI client?

    Yes, it works with any MCP-compatible client like Claude, Cursor, or Windsurf.

  • 05

    Can I see the training history?

    Yes, if the model configuration data includes that history, the AI can retrieve it for you.

  • More questions about H2O.ai? See everything the H2O.ai Connector can do

Connect H2O.ai to Claude, Cursor, ChatGPT & more