• DATA
  • BOTS
  • ERRORS
  • HUMAN

How to test bot accuracy with your AI agent

Connect your agent to your bot's data sources once. From there, you can probe its knowledge directly to see if it's hallucinating or sticking to the facts.

Ask AI about this page

Short answer

How can I check if my bot is giving correct answers?

You can use your agent to pull specific information from the bot's configured data sources and compare it to the bot's responses. This lets you catch errors in its logic or knowledge gaps immediately. You'll see exactly where the data matches or fails.

The testing outcomes.

Where your results land.

Running these queries gives you a clear view of how your bot handles real-world questions.

DATA

Verified knowledge

Your agent pulls the raw facts from the source. You get the ground truth to compare against what the bot says.

BOTS

Bot inventory

You can see a list of all your configured bots. This helps you organize which ones need testing.

ERRORS

Identify hallucinations

When the bot's answer contradicts the data your agent finds, you've caught a hallucination. This is your primary way to find bugs.

HUMAN

Manual fixes

The AI identifies the gap, but you decide how to update the source data or the bot's instructions to fix it.

The workflow

What your AI does when you start testing.

The process moves from identifying your bots to pulling specific data for comparison.

  1. Find your bots

    Your agent looks through your account to find every bot you have set up. This gives you a starting point for testing.

    list_bots

  2. Select a target

    Once you pick a bot, your agent pulls its specific configuration and settings. This ensures the test is targeted at the right instance.

    get_bot

  3. Query the data

    Your agent asks the bot's data sources a specific question. It gets the actual information the bot is supposed to know.

    query_bot

  4. Compare results

    Your agent compares the data it just retrieved against the bot's response. It highlights any discrepancies it finds.

    query_bot

Try it

Copy these to start.

Use these prompts to begin your accuracy audit.

Starting points

These are just ideas. Swap out the bot names or specific topics to match your actual setup.

Chatsistant Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_pHKIqUVnpNzysk3EjJJFx7brchTn8atfbhMnxatz/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Chatsistant capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "chatsistant-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_pHKIqUVnpNzysk3EjJJFx7brchTn8atfbhMnxatz/mcp"
    }
  }
}
Copy into chat04

  • List all my bots so I can choose one to test.

  • Get the details for my 'Customer Support' bot.

  • Query the 'Technical Docs' bot about our refund policy.

  • Check the 'Sales Assistant' bot for information on pricing tiers.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Chatsistant once, then ask.

Just link your account with one click. Your credentials stay encrypted and safe, and you can start testing immediately.

Connect Chatsistant to your AI

FAQ

Testing behavior

  • 01

    Can the AI change my bot's settings?

    No. The tools only allow the AI to list, get, and query data. It cannot modify your bot's configuration.

  • 02

    Will this delete any of my data sources?

    No. This MCP is read-only for your data. It cannot delete or overwrite your documents or settings.

  • 03

    Can I test multiple bots at once?

    You can list them all, but you'll need to run queries against them one at a time to get specific answers.

  • 04

    How does the AI know if the answer is wrong?

    The AI compares the answer the bot gives you against the raw data it pulls using the query tool. If they don't match, it flags the error.

  • 05

    Do I need to provide the data manually?

    No. The AI uses the data sources you have already configured in your bot.

  • More questions about Chatsistant? The Connector page answers them. See everything the Chatsistant Connector can do

Connect Chatsistant to Claude, Cursor, ChatGPT & more