Ragas Connector for AI agents.
7 live capabilities
Audit RAG performance and LLM hallucination rates in real-time.
Waiting for input…
Why people use Ragas
Ragas for RAG Evaluation and Hallucination Tracking
This Connector changes that by bringing the evaluation dashboard directly into your chat. You can tell your agent to run a full evaluation on a dataset, and it'll pull back the scores for faithfulness and relevancy immediately. You get a clear view of your progress without the manual data entry, letting you focus on the actual engineering.
What Vinkius changes
You get a way to audit your RAG performance in real-time without switching contexts.
Use it from Claude, ChatGPT, Cursor or another AI client you already have.
One account · 5,900+ Connectors
- Real-world use case 01
Checking for hallucinations
A QA specialist notices a model is making things up.
- Real-world use case 02
Comparing chunking strategies
A dev wants to see if smaller chunks help.
- Real-world use case 03
Organizing test data
A team has 50 different test sets.
Complete set · 7capabilities
The complete Ragas capability set.
These are the exact actions your AI can choose when you ask it to work with Ragas.
01—04
4 capabilities in this set.
Part of 7 available through Ragas.
- 01 Capability
List datasets
See every evaluation dataset you've uploaded. This helps you stay organized when managing multiple test sets for different projects.
- 02 Capability
Get results
See the final scores from a finished experiment. This gives you the hard numbers on your model's performance to help you make data-driven decisions.
- 03 Capability
Get dataset
Get the specific details for one dataset. Use this to check the contents of a particular evaluation group or see its metadata.
- 04 Capability
List experiments
See all the tests associated with a specific dataset. This is the fastest way to find previous runs and compare results.
05—07
3 capabilities in this set.
Part of 7 available through Ragas.
- 05 Capability
Get experiment
Pull up the full details for a single experiment. It's the best way to deep-dive into a specific test run and see the raw data.
- 06 Capability
Run evaluation
Start a new test run on a dataset. This triggers the scoring for your RAG pipeline so you can see how your model performs on real queries.
- 07 Capability
List metrics
See every available scoring metric you can use. Use this to check what's available for your specific needs like faithfulness or relevancy.
Set up in minutes
One URL. Then ask Ragas to work.
Claude and ChatGPT only need the Connector URL. Copy it once, add it in settings, and use Ragas from the conversation.
Choose your client
Live previewAdvanced clients IDE · CLI
Claude · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp - Step 01
Open Connectors
In Claude Web or Claude Desktop, open Settings and choose Connectors.
- Step 02
Add the URL
Choose Add custom connector, name it Ragas, and paste the URL above.
- Step 03
Turn it on in chat
Select +, open Connectors, and enable Ragas for the conversation.
ChatGPT · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp - Step 01
Open MCP settings
On desktop, open Settings and MCP servers. On web, open your workspace app or connector settings.
- Step 02
Add the URL
Choose Add server with Streamable HTTP, or create a custom MCP app, then paste the Ragas URL.
- Step 03
Save and start
Save the connection and enable Ragas in your conversation. Desktop may ask you to restart once.
Cursor · IDE configuration
Advanced setup
{
"mcpServers": {
"ragas": {
"url": "https://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp"
}
}
}- Step 01
Open MCP Settings
Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows/Linux) → search "MCP Settings"
- Step 02
Add the server config
Paste the JSON configuration above into the mcp.json file that opens
- Step 03
Save the file
Cursor will automatically detect the new Connector
- Step 04
Start using Ragas
Open Agent mode in chat and ask: "Using Ragas, help me...". 7 tools available
VS Code Copilot · IDE configuration
Advanced setup
{
"mcpServers": {
"ragas": {
"url": "https://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp"
}
}
}- Step 01
Create MCP config
Create a .vscode/mcp.json file in your project root
- Step 02
Add the server config
Paste the JSON configuration above
- Step 03
Enable Agent mode
Open GitHub Copilot Chat and switch to Agent mode using the dropdown
- Step 04
Start using Ragas
Ask Copilot: "Using Ragas, help me...". 7 tools available
Windsurf · IDE configuration
Advanced setup
{
"mcpServers": {
"ragas": {
"url": "https://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp"
}
}
}- Step 01
Open MCP Settings
Go to Settings → MCP Configuration or press Cmd+Shift+P and search "MCP"
- Step 02
Add the server
Paste the JSON configuration above into mcp_config.json
- Step 03
Save and reload
Windsurf will detect the new server automatically
- Step 04
Start using Ragas
Open Cascade and ask: "Using Ragas, help me...". 7 tools available
Cline · IDE configuration
Advanced setup
{
"mcpServers": {
"ragas": {
"url": "https://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp"
}
}
}- Step 01
Open Cline MCP Settings
Click the Connectors icon in the Cline sidebar panel
- Step 02
Add remote server
Click "Add Connector" and paste the configuration above
- Step 03
Enable the server
Toggle the server switch to ON
- Step 04
Start using Ragas
Ask Cline: "Using Ragas, help me...". 7 tools available
Claude Code · Terminal command
Advanced setup
claude mcp add ragas --transport http "https://edge.vinkius.com/vk_preview_mQUV7klN4rh09rRcp9ld3cwceCP25oL5Lk8res4B/mcp"- Step 01
Install Claude Code
Run npm install -g @anthropic-ai/claude-code if not already installed
- Step 02
Add the Connector
Run the command above in your terminal
- Step 03
Verify the connection
Run claude mcp to list connected servers, or type /mcp inside a session
- Step 04
Start using Ragas
Ask Claude: "Using Ragas, show me...". 7 tools are ready
Where the request belongs
Work Ragas can move forward.
This is for the ML engineers and QA specialists who are tired of manual testing. If you're spent hours running scripts just to see if your RAG system is actually getting better, this is for you.
ML Engineer
Runs evaluations on new chunking strategies on a Tuesday afternoon to check for regressions.
LLM QA Specialist
Benchmarks different model versions against a gold standard dataset to ensure low hallucination rates.
Data Scientist
Compares two different retrieval methods side-by-side using unified Ragas metrics.
Build the capability set
Add more capabilities.
Each Connector adds new actions and data without changing how you work.
Browse ConnectorsBraintrust
Automate AI evaluations with Braintrust. organize projects, test model datasets, run benchmarks, and manage prompts via any AI agent.
Mistral AI (Frontier LLMs & Embeddings)
Manage AI inference via Mistral. execute chat completions, generate RAG embeddings, and audit frontier models.
LangSmith
Observability and evaluation platform for LLM applications. monitor traces, debug agent runs, and track performance metrics across your AI stack.
LangSmith (LLM Observability & Hub)
Monitor LLM apps via LangSmith. track traces, audit prompt templates, and manage evaluation datasets.
Langfuse (LLM Tracing & Evals)
Monitor LLM apps via Langfuse. track traces, manage prompt templates, and audit evaluation scores.
Predibase (LLM Serving & Finetuning)
Deploy and query fine-tuned LLMs via Predibase. run inference, classify text, and monitor deployment metrics directly from your AI agent.
Bring your own AI
Change the model, client or framework. Keep Ragas connected.
-
Claude -
ChatGPT -
Gemini -
Cursor -
VS Code -
Windsurf -
ZCode -
Cline -
Zed -
Continue -
Kiro -
Roo Code -
Zencoder -
Goose -
Void -
Augment Code -
Amp -
Qodo -
Tabnine -
Pieces -
Sourcegraph Cody -
JetBrains -
Warp -
Amazon Q -
Antigravity -
BoltAI -
Raycast -
Jan -
LM Studio -
AnythingLLM -
Open WebUI -
Msty -
Cherry Studio -
LibreChat -
TypingMind -
Chorus -
5ire -
n8n -
LangChain -
LlamaIndex -
CrewAI -
Vercel AI SDK
Before you connect
Questions about Ragas.
The practical details behind the request, access and result.
What is the Ragas MCP used for?
Ragas is used to evaluate RAG systems. It helps you measure how well your AI answers questions based on your data using standard metrics.
How does Ragas help with LLM hallucinations?
It provides specific metrics like faithfulness to help you identify and reduce hallucinations in your RAG pipeline.
Can I use Ragas to track my RAG experiments?
Yes, you can use it to list and view all your previous test runs to see how your model's performance changes over time.
Does Ragas support specific metrics like faithfulness?
Yes, it supports a variety of metrics including faithfulness and answer relevancy to give you a complete picture of performance.
How do I organize my RAG test data with Ragas?
You can organize your evaluation data into specific projects within the Connector to keep your different test sets neatly categorized.
Can my AI agent run evaluations automatically?
Yes, your AI agent can trigger new evaluation runs on your datasets directly from your chat or IDE.
How do I secure an App Token for Ragas?
Log into your provided Ragas dashboard. In your project's settings or dedicated security section, you will find the ability to generate a new Application Token. Copy it immediately, as it may only appear once.
What format is required to upload a dataset?
The capability uses common array formats through the Connector wrapper. When passing data, the AI maps arrays containing question, ground_truth and contexts natively matching Ragas base requirements.
Does the server evaluate prompts automatically during testing?
Yes. When triggering evaluations, Ragas uses its own sophisticated metrics (like Faithfulness, Answer Relevance) running internally. The Connector simply pipes these generated reports back to your chat.
One connection away
Give your agent a direct line to Ragas.
Connect Ragas once. Keep it beside 5,900+ managed Connectors when the next task needs more.
Explore every Connector No credit card required · Free tier available