• ANALYSIS
  • TRENDS
  • DETECTION
  • DECISION

How to monitor model performance with your AI

Connect Braintrust to your AI client once. From then on, your agent can pull experiment scores and project data to track how your models are actually behaving.

Ask AI about this page

Short answer

How do I check if my model changes improved accuracy?

Your AI queries Braintrust to pull experiment scores and project analytics. It compares these metrics to show you exactly how recent changes impacted behavior. Here is how it works.

The outcomes

What you get from the data

Your AI turns Braintrust metrics into clear answers.

ANALYSIS

Compare experiment scores

The AI pulls data from your latest runs to show how different versions perform against each other.

TRENDS

Track project history

Your agent looks at project analytics to see if performance is trending up or down over time.

DETECTION

Spot behavior shifts

The AI identifies when a specific model change causes a drop in accuracy or a change in output style.

DECISION

Human-led deployment

The AI presents the findings, but you make the final call on which model to push to production.

The workflow

What your AI does when the data arrives.

The AI acts as a bridge between your Braintrust dashboard and your chat interface.

  1. Locate projects

    The AI finds the specific Braintrust projects you want to investigate.

    list_projects

  2. Fetch experiment scores

    It pulls the specific metrics and scores from your recent test runs.

    list_experiments

  3. Compare versions

    The agent looks at different experiment results to find performance deltas.

    list_experiments

  4. Report findings

    Your AI summarizes the project analytics to explain how behavior has changed.

    list_projects

Try it

Copy these to start.

Use these prompts to begin your analysis.

Starting points

Replace the project names and experiment IDs with your actual Braintrust names.

Braintrust Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Braintrust capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "braintrust-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp"
    }
  }
}
Copy into chat04

  • List all my Braintrust projects so I can see what's available.

  • Show me the scores for the 'v2-test-run' experiment in my 'Customer-Bot' project.

  • Compare the accuracy scores between my last three experiments.

  • Did the recent changes in the 'Search-Optimization' project improve my model's performance?

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Braintrust once, then ask.

Connect your account in one step. Your credentials stay encrypted on our side, and you can immediately start asking your AI about your model's performance.

Connect Braintrust to your AI

FAQ

How this task behaves

  • 01

    Can the AI change my experiment settings?

    No. The AI can only read project and experiment data; it cannot modify your Braintrust configurations.

  • 02

    Can the AI delete my test runs?

    No. This MCP is read-only for your experiments and projects.

  • 03

    How does the AI know which project to look at?

    It uses the list_projects tool to see your available projects, then you can tell it which one to focus on.

  • 04

    Does this work with any AI client?

    Yes, it works with any MCP-compatible client like Claude, Cursor, or Windsurf.

  • 05

    Can the AI create new experiments for me?

    No. The AI can only list and query existing experiments.

  • More questions about Braintrust? See everything the Braintrust Connector can do

Connect Braintrust to Claude, Cursor, ChatGPT & more