Braintrust Connector for AI agents.
10 live capabilities
Manage model evaluations and prompt versions without leaving your chat.
Waiting for input…
Why people use Braintrust
Stop Manual Prompt Management with Braintrust
This Connector changes that by letting your agent handle the logistics. You can ask your agent to grab the latest prompt, run a test, and record the results in Braintrust in seconds. You get a single source of truth for your evaluations without the manual overhead.
What Vinkius changes
That your agent becomes a direct interface for your Braintrust evaluation pipeline.
Use it from Claude, ChatGPT, Cursor or another AI client you already have.
One account · 5,900+ Connectors
- Real-world use case 01
Testing a new model
An engineer asks the agent to run a new experiment and compare it against the last version to see if accuracy dropped.
- Real-world use case 02
Updating a prompt
A product manager wants to see the current text of a prompt and asks the agent to pull it to check the tone.
- Real-world use case 03
Expanding a dataset
A data scientist tells the agent to add 50 new rows to a specific dataset for a new edge case.
Complete set · 10capabilities
The complete Braintrust capability set.
These are the exact actions your AI can choose when you ask it to work with Braintrust.
01—04
4 capabilities in this set.
Part of 10 available through Braintrust.
- 01 Capability
List env vars
Check the Braintrust AI Gateway configurations for your model API keys. It's a quick way to probe your current setup.
- 02 Capability
List experiments
Get a list of all evaluation experiments and their associated scores. Use this to monitor your testing progress.
- 03 Capability
Get dataset
Pull a specific dataset that contains the schemas for your LLM outputs. This lets you see exactly what data is being used.
- 04 Capability
Get prompt
Retrieve the literal text and variable contexts for a specific prompt. This is great for auditing your current instructions.
05—07
3 capabilities in this set.
Part of 10 available through Braintrust.
- 05 Capability
Insert dataset row
Add new test cases into a dataset matrix for specific evaluations. It keeps your test data fresh without manual entry.
- 06 Capability
List projects
See every AI evaluation project currently active in your Braintrust account. Use this to get a high level view of your work.
- 07 Capability
List prompts
Find all version-controlled system prompts stored in Braintrust. This helps you manage multiple versions of your instructions.
08—10
3 capabilities in this set.
Part of 10 available through Braintrust.
- 08 Capability
Create experiment
Start a new historical trace to record your LLM pipeline tests. It helps you see how different iterations perform over time.
- 09 Capability
Create project
Set up a new project environment to track your AI evaluations and datasets. Use this to keep different tasks isolated.
- 10 Capability
List datasets
See all isolated Ground Truth text banks used for automated scoring. This helps you find the right data quickly.
Set up in minutes
One URL. Then ask Braintrust to work.
Claude and ChatGPT only need the Connector URL. Copy it once, add it in settings, and use Braintrust from the conversation.
Choose your client
Live previewAdvanced clients IDE · CLI
Claude · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp - Step 01
Open Connectors
In Claude Web or Claude Desktop, open Settings and choose Connectors.
- Step 02
Add the URL
Choose Add custom connector, name it Braintrust, and paste the URL above.
- Step 03
Turn it on in chat
Select +, open Connectors, and enable Braintrust for the conversation.
ChatGPT · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp - Step 01
Open MCP settings
On desktop, open Settings and MCP servers. On web, open your workspace app or connector settings.
- Step 02
Add the URL
Choose Add server with Streamable HTTP, or create a custom MCP app, then paste the Braintrust URL.
- Step 03
Save and start
Save the connection and enable Braintrust in your conversation. Desktop may ask you to restart once.
Cursor · IDE configuration
Advanced setup
{
"mcpServers": {
"braintrust": {
"url": "https://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp"
}
}
}- Step 01
Open MCP Settings
Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows/Linux) → search "MCP Settings"
- Step 02
Add the server config
Paste the JSON configuration above into the mcp.json file that opens
- Step 03
Save the file
Cursor will automatically detect the new Connector
- Step 04
Start using Braintrust
Open Agent mode in chat and ask: "Using Braintrust, help me...". 10 tools available
VS Code Copilot · IDE configuration
Advanced setup
{
"mcpServers": {
"braintrust": {
"url": "https://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp"
}
}
}- Step 01
Create MCP config
Create a .vscode/mcp.json file in your project root
- Step 02
Add the server config
Paste the JSON configuration above
- Step 03
Enable Agent mode
Open GitHub Copilot Chat and switch to Agent mode using the dropdown
- Step 04
Start using Braintrust
Ask Copilot: "Using Braintrust, help me...". 10 tools available
Windsurf · IDE configuration
Advanced setup
{
"mcpServers": {
"braintrust": {
"url": "https://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp"
}
}
}- Step 01
Open MCP Settings
Go to Settings → MCP Configuration or press Cmd+Shift+P and search "MCP"
- Step 02
Add the server
Paste the JSON configuration above into mcp_config.json
- Step 03
Save and reload
Windsurf will detect the new server automatically
- Step 04
Start using Braintrust
Open Cascade and ask: "Using Braintrust, help me...". 10 tools available
Cline · IDE configuration
Advanced setup
{
"mcpServers": {
"braintrust": {
"url": "https://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp"
}
}
}- Step 01
Open Cline MCP Settings
Click the Connectors icon in the Cline sidebar panel
- Step 02
Add remote server
Click "Add Connector" and paste the configuration above
- Step 03
Enable the server
Toggle the server switch to ON
- Step 04
Start using Braintrust
Ask Cline: "Using Braintrust, help me...". 10 tools available
Claude Code · Terminal command
Advanced setup
claude mcp add braintrust --transport http "https://edge.vinkius.com/vk_preview_cr8T7CpPGy155EtBgklGtlDgFsTWLRLYfzKoHIOF/mcp"- Step 01
Install Claude Code
Run npm install -g @anthropic-ai/claude-code if not already installed
- Step 02
Add the Connector
Run the command above in your terminal
- Step 03
Verify the connection
Run claude mcp to list connected servers, or type /mcp inside a session
- Step 04
Start using Braintrust
Ask Claude: "Using Braintrust, show me...". 10 tools are ready
Where the request belongs
Work Braintrust can move forward.
This is for AI developers and ML engineers who are tired of manually checking every single model output or copy-pasting prompts between different testing environments.
ML Engineer
Running regression tests on new model weights to ensure no performance drops occur in production.
AI Developer
Quickly checking the latest prompt version to see if a specific instruction change actually worked.
Data Scientist
Building and querying large matrices of ground truth data for automated scoring and trend analysis.
Build the capability set
Add more capabilities.
Each Connector adds new actions and data without changing how you work.
Browse ConnectorsLangSmith
Observability and evaluation platform for LLM applications. monitor traces, debug agent runs, and track performance metrics across your AI stack.
LangSmith (LLM Observability & Hub)
Monitor LLM apps via LangSmith. track traces, audit prompt templates, and manage evaluation datasets.
Ragas
Equip your AI with Ragas to create datasets, run RAG evaluations, and track experiment metrics directly from your workflow.
Datadog AI (LLM Observability)
Monitor LLM performance via Datadog. track token usage, audit prompts, and monitor AI model metrics directly from any AI agent.
Langfuse (LLM Tracing & Evals)
Monitor LLM apps via Langfuse. track traces, manage prompt templates, and audit evaluation scores.
Aporia
Monitor AI models and validate LLM interactions with guardrails directly from your AI agent to ensure safety and observability.
Bring your own AI
Change the model, client or framework. Keep Braintrust connected.
-
Claude -
ChatGPT -
Gemini -
Cursor -
VS Code -
Windsurf -
ZCode -
Cline -
Zed -
Continue -
Kiro -
Roo Code -
Zencoder -
Goose -
Void -
Augment Code -
Amp -
Qodo -
Tabnine -
Pieces -
Sourcegraph Cody -
JetBrains -
Warp -
Amazon Q -
Antigravity -
BoltAI -
Raycast -
Jan -
LM Studio -
AnythingLLM -
Open WebUI -
Msty -
Cherry Studio -
LibreChat -
TypingMind -
Chorus -
5ire -
n8n -
LangChain -
LlamaIndex -
CrewAI -
Vercel AI SDK
Before you connect
Questions about Braintrust.
The practical details behind the request, access and result.
How does the Braintrust MCP help with my prompt engineering?
It lets you pull the exact, version-controlled text of your prompts directly into your chat. You can check instructions and variable contexts without digging through your code or documentation.
Can I use Braintrust MCP to update my test data?
Yes, you can ask your agent to insert new rows into your datasets. This makes it easy to add new test cases to your evaluation matrices on the fly.
What is the Braintrust MCP for model evaluation?
It connects your agent to your evaluation pipeline so you can run experiments, track scores, and see regressions across different models in one place.
Will Braintrust MCP help me manage my AI projects?
It allows your agent to list all active projects and create new ones. This keeps your different AI testing tasks organized and easy to find.
Can the Braintrust MCP see my model API keys?
The Connector can probe your AI Gateway configurations to check your settings. This helps you manage your model API keys and configurations securely.
Does Braintrust MCP support Ground Truth datasets?
Yes, it can query your specific Ground Truth sets to help your agent understand the schemas and data needed for accurate scoring.
Can I insert new test data dynamically tracking specific limits?
Yes. Utilizing the insert_dataset_row method, you can effortlessly inject exact JSON tracking payload mapping strings directly inside the text corpus evaluating the final results.
Does it pull out original Prompt definitions stored securely?
Certainly. The get_prompt command isolates and returns perfectly version-controlled bounding parameters slicing literal templates natively hosted under the Braintrust database.
How deeply can it inspect test regressions or scoring limits?
Using the robust list_experiments call, you can branch full arrays separating LLM version behaviors over massive iterations tracking the performance anomalies accurately.
One connection away
Give your agent a direct line to Braintrust.
Connect Braintrust once. Keep it beside 5,900+ managed Connectors when the next task needs more.
Explore every Connector No credit card required · Free tier available