• DATA
  • TUNING
  • ANALYSIS
  • HUMAN

How to automate model feedback with your AI

Connect your agent to Dify to close the loop between user interactions and model improvements. Your AI handles the data submission so you can focus on tuning.

Ask AI about this page

Short answer

How can I send user ratings back to Dify automatically?

Your agent uses the Dify.AI SDK to trigger a feedback submission whenever a user provides a rating or comment. This creates a direct pipeline for RLHF and model tuning. You'll see how the data flows from the chat interface to your Dify dashboard.

Feedback outcomes

Where your data lands.

What happens after your agent captures a rating.

DATA

Logged feedback

The agent sends the rating and text to Dify. This populates your feedback logs for review.

TUNING

RLHF readiness

Captured data becomes available for Reinforcement Learning from Human Feedback. You use this to refine model responses.

ANALYSIS

Performance insights

Aggregated ratings show you exactly where the agent succeeds or fails. You get a clear view of model accuracy.

HUMAN

Manual review

You decide which feedback is high-quality enough to use for fine-tuning. The AI doesn't decide what to keep.

The workflow

What your AI does when the feedback arrives.

The agent acts as the bridge between the user's opinion and your Dify environment.

  1. Capture input

    The agent listens for specific user signals, like a thumbs up or a written critique of a response.

    submit_feedback

  2. Process rating

    The agent formats the user's rating and the associated conversation ID into a structured payload.

    submit_feedback

  3. Submit to Dify

    The agent pushes the formatted feedback directly into your Dify workspace using the provided tool.

    submit_feedback

  4. Confirm success

    The agent verifies the submission was successful and can notify the user that their feedback was received.

    submit_feedback

Try it

Copy these to start.

Use these prompts to trigger the feedback process.

Starting points

Swap out the specific chat IDs or user names to match your current session.

Dify.AI Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_esFwQvD1BTqmueeHwIOlGHPDJ8Oq0FzvLe2J4jkV/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Dify.AI capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "difyai-sdk-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_esFwQvD1BTqmueeHwIOlGHPDJ8Oq0FzvLe2J4jkV/mcp"
    }
  }
}
Copy into chat04

  • Submit a thumbs up feedback for the last response in this chat.

  • Send a rating of 1 to Dify for the response about the quarterly report.

  • Log this user comment as feedback for the current conversation.

  • Submit a negative rating for the last answer and include 'too wordy' as the reason.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Dify.AI SDK once, then ask.

One connection links your agent to the Dify API. You can find the full setup details on the Connector page.

Connect Dify.AI SDK to your AI

FAQ

How this task behaves

  • 01

    Can the AI delete feedback once it is submitted?

    No. The tool only allows for submitting new feedback to Dify.

  • 02

    Does the AI decide which ratings are important?

    No. The AI simply submits whatever feedback the user provides. You must review the data in Dify to decide what to use for tuning.

  • 03

    Can I send feedback for a specific message ID?

    Yes, you can instruct your agent to submit feedback for specific interactions.

  • 04

    Will this automatically retrain my model?

    No. This tool only moves the data to Dify. You still need to initiate the training or tuning process yourself.

  • 05

    Can the agent submit feedback without user input?

    No. The agent requires a user rating or comment to trigger the submission tool.

  • More questions about Dify.AI SDK? The Connector page answers them. See everything the Dify.AI SDK Connector can do

Connect Dify.AI to Claude, Cursor, ChatGPT & more