How do I secure an App Token for Ragas?

Log into your provided Ragas dashboard. In your project's settings or dedicated security section, you will find the ability to generate a new Application Token. Copy it immediately, as it may only appear once.

What format is required to upload a dataset?

The tool uses common array formats through the MCP wrapper. When passing data, the AI maps arrays containing `question`, `ground_truth` and `contexts` natively matching Ragas base requirements.

Does the server evaluate prompts automatically during testing?

Yes. When triggering evaluations, Ragas uses its own sophisticated metrics (like Faithfulness, Answer Relevance) running internally. The MCP server simply pipes these generated reports back to your chat.

Ragas MCP Server for Claude Desktop 7 tools — connect in under 2 minutes

Name: Ragas
Availability: InStock
Author: Vinkius

modelcontextprotocol.io/quickstart/user

Built by Vinkius GDPR 7 Tools IDE

Claude Desktop is Anthropic's native application for interacting with Claude AI models on macOS and Windows. It was the first consumer application to ship with built-in MCP support, making it the reference implementation for the Model Context Protocol standard.

Get MCP Server for AI Agents

ASK AI ABOUT THIS MCP SERVER

Open in ChatGPT Open in Claude Open in Perplexity

Vinkius supports streamable HTTP and SSE.

RecommendedModern Approach — Zero Configuration

Vinkius Desktop App

The modern way to manage MCP Servers — no config files, no terminal commands. Install Ragas and 2,500+ MCP Servers from a single visual interface.

Download Free Open SourceNo signup required

Classic Setup·json

{
  "mcpServers": {
    "ragas": {
      // Your Vinkius token — get it at cloud.vinkius.com
      "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    }
  }
}

Fully ManagedVinkius Servers

60%Token savings

High SecurityEnterprise-grade

IAMAccess control

EU AI ActCompliant

DLPData protection

V8 IsolateSandboxed

Ed25519Audit chain

<40msKill switch

Stream every event to Splunk, Datadog, or your own webhook in real-time

* Every MCP server runs on Vinkius-managed infrastructure inside AWS - a purpose-built runtime with per-request V8 isolates, Ed25519 signed audit chains, and sub-40ms cold starts optimized for native MCP execution. See our infrastructure

About Ragas MCP Server

Integrate Ragas with your AI agent to bring professional grade RAG (Retrieval-Augmented Generation) evaluation and tracking into your chat interface. By subscribing to this server, the AI can seamlessly manage datasets and measure LLM performance on demand.

Claude Desktop is the definitive way to connect Ragas to your AI workflow. Add the Vinkius Edge URL to your config, restart the app, and Claude immediately exposes all 7 tools in the chat interface — ask a question, Claude calls the right tool, and you see the answer. Zero code, zero context switching.

What you can do

Dataset Management — Upload, list, and organize evaluation datasets directly inside your environment.
Run Evaluations — Automatically trigger Ragas evaluations on your RAG pipelines and fetch detailed scoring.
Track Experiments — Monitor and compare iterative improvements by viewing tracked metrics across different agent versions.
Project Organization — Associate evaluations with specific projects within your Ragas dashboard.

The Ragas MCP Server exposes 7 tools through the Vinkius. Connect it to Claude Desktop in under two minutes — no API keys to rotate, no infrastructure to provision, no vendor lock-in. Your configuration, your data, your control.

How to Connect Ragas to Claude Desktop via MCP

Follow these steps to integrate the Ragas MCP Server with Claude Desktop.

Open Claude Desktop Settings

Go to Settings → Developer → Edit Config to open claude_desktop_config.json

Add the MCP Server

Paste the configuration above into the mcpServers section

Restart Claude Desktop

Close and reopen Claude Desktop to load the new server

Start using Ragas

Look for the 🔌 icon in the chat — your 7 tools are now available

Why Use Claude Desktop with the Ragas MCP Server

Claude Desktop by Anthropic provides unique advantages when paired with Ragas through the Model Context Protocol.

Claude Desktop is the reference MCP client — it was designed alongside the protocol itself, ensuring the most complete and stable MCP implementation available

Zero-code configuration: add a server URL to a JSON file and Claude instantly discovers and exposes all available tools in the chat interface

Claude's extended thinking capability lets it reason through multi-step tool usage, chaining multiple API calls to answer complex questions

Enterprise-grade security with local config storage — your tokens never leave your machine, and connections go directly to the Vinkius Edge network

Ragas + Claude Desktop Use Cases

Practical scenarios where Claude Desktop combined with the Ragas MCP Server delivers measurable value.

Interactive data exploration: ask Claude to query DNS records, look up WHOIS data, and cross-reference results in a single conversation

Ad-hoc security audits: type a domain name and let Claude enumerate subdomains, check DNS history, and flag configuration anomalies — all through natural language

Executive briefings: generate comprehensive domain intelligence reports by asking Claude to compile findings into a formatted summary

Learning and training: new team members can explore API capabilities conversationally without needing to read documentation

Ragas MCP Tools for Claude Desktop (7)

These 7 tools become available when you connect Ragas to Claude Desktop via MCP:

get_dataset

Retrieves details for a specific evaluation dataset

get_experiment

Retrieves detailed information for a specific experiment

get_results

Retrieves the results of a completed experiment

list_datasets

Lists available evaluation datasets

list_experiments

Lists experiments associated with a specific dataset

list_metrics

Lists all available evaluation metrics

run_evaluation

g., faithfulness, answer_relevancy). Triggers a new evaluation run for a dataset

Example Prompts for Ragas in Claude Desktop

Ready-to-use prompts you can give your Claude Desktop agent to start working with Ragas immediately.

"List all Ragas datasets available in my project."

"Fetch the metrics and results for the recent experiment 'Support Bot V3'."

"Create a new Ragas project named 'Financial_RAG_Testing'."

Troubleshooting Ragas MCP Server with Claude Desktop

Common issues when connecting Ragas to Claude Desktop through the Vinkius, and how to resolve them.

Server not appearing after restart

Ensure the JSON is valid (no trailing commas). Check the file path: ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\\Claude\\ (Windows).

Authentication error

Verify your Vinkius token is correct. Go to cloud.vinkius.com to regenerate it if needed.

Tools not showing in chat

Click the 🔌 icon at the bottom of the chat input. If it shows 0 tools, the server may still be connecting — wait a few seconds.

Ragas + Claude Desktop FAQ

Common questions about integrating Ragas MCP Server with Claude Desktop.

How does Claude Desktop discover MCP tools?

When Claude Desktop starts, it reads the claude_desktop_config.json file and connects to each configured MCP server. It calls the tools/list endpoint to fetch the schema for every available tool, then surfaces them as clickable options in the chat interface via the 🔌 icon.

What happens if the MCP server is temporarily unavailable?

Claude Desktop handles disconnections gracefully — if the server is unreachable at startup, the tools simply won't appear. Once the server becomes available again, restarting Claude Desktop will re-establish the connection. There is no timeout penalty or error loop.

Can I connect multiple MCP servers simultaneously?

Yes. You can add as many servers as you need in the mcpServers section of the config file. Each server appears as a separate tool provider, and Claude can use tools from multiple servers in a single conversation turn.

Is there a limit on the number of tools per server?

Claude Desktop can handle hundreds of tools per server. However, for optimal LLM performance, Vinkius servers are designed to expose focused, well-documented tool sets rather than overwhelming the model with too many options.

Does Claude Desktop support Streamable HTTP transport?

Yes. Claude Desktop supports both SSE (Server-Sent Events) and the newer Streamable HTTP transport that Vinkius uses. Simply provide the server URL — Claude auto-negotiates the transport protocol.