• VERIFICATION
  • AUDIT
  • REVISION
  • HUMAN

How to test chatbot responses with your AI

Connect ChatFly to your AI client once. After that, you can use your agent to run live tests on any bot you have deployed.

Ask AI about this page

Short answer

How can I test my chatbot's accuracy using an AI client?

You can use your AI client to send messages directly to your deployed bots. This lets you verify how they handle specific prompts without leaving your usual workspace. You'll see exactly how they respond to different inputs in real time.

Testing outcomes

What you'll find

The results of your testing sessions.

VERIFICATION

Confirm bot accuracy

Your AI sends messages to the bot and reports back the exact text it received. You can see if the bot is following instructions or hallucinating.

AUDIT

Identify edge cases

By testing tricky prompts, you find where the bot breaks. Your AI helps you map out these failures quickly.

REVISION

Refine bot logic

Use the feedback from your chat sessions to tweak your bot's instructions. This closes the loop between testing and deployment.

HUMAN

Manual intervention

If a bot response is completely wrong, you decide how to fix the underlying prompt or logic. The AI identifies the issue, but you own the fix.

The workflow

What your AI does when the test starts.

The AI handles the communication layer so you can focus on the results.

  1. Find your bots

    The AI looks through your ChatFly account to find the specific bots you want to test.

    list_bots

  2. Select a target

    Once the list is ready, you tell your agent which bot needs testing.

    list_bots

  3. Send test messages

    Your agent sends the specific prompts you've written directly to the bot's interface.

    chat

  4. Analyze responses

    The AI captures the bot's reply and presents it to you in your chat window.

    chat

Try it

Copy these to start.

Use these prompts to begin testing immediately.

Starting points

These are ideas to get you moving. Swap out the bot names for your actual bot identifiers.

ChatFly Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_YeceOYq0Sj0dHYZqDFIY8IgiIyBo9f7LRr4xsK2v/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — ChatFly capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "chatfly-alternative-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_YeceOYq0Sj0dHYZqDFIY8IgiIyBo9f7LRr4xsK2v/mcp"
    }
  }
}
Copy into chat04

  • List my bots so I can see which ones are available to test.

  • Send a message to the CustomerSupportBot asking about shipping rates.

  • Test the SalesBot by asking if it can offer a 20% discount.

  • Check the responses from the FAQBot for any incorrect information.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect ChatFly once, then ask.

Just link your account with one click. Your credentials stay encrypted, and you can start testing through your AI client right away.

Connect ChatFly to your AI

FAQ

Common questions

  • 01

    Can the AI change my bot's settings?

    No. The AI can only send messages and list bots. It cannot modify your bot's configuration or settings.

  • 02

    Will the AI delete any of my bots?

    No. There are no tools provided to delete or remove bots from your account.

  • 03

    Can I test bots that aren't deployed yet?

    No. You can only interact with bots that are currently active and accessible via the ChatFly API.

  • 04

    Does the AI see my private chat history?

    The AI only sees the specific messages sent during the testing session to facilitate the chat.

  • 05

    Can I use this with any AI client?

    Yes, as long as the client supports the Model Context Protocol (MCP).

  • More questions about ChatFly? The Connector page answers them. See everything the ChatFly Connector can do

Connect ChatFly to Claude, Cursor, ChatGPT & more