• VERSIONING
  • SENTIMENT
  • OPTIMIZATION
  • HUMAN

How to analyze prompt versions with your AI

Connect Helicone to your AI client to turn raw logs into actionable insights. You'll see exactly how changes to your instructions impact user satisfaction.

Ask AI about this page

Short answer

How can I compare prompt iterations and user feedback?

Your AI pulls specific prompt versions and their associated user feedback to find patterns in performance. This lets you pinpoint which specific wording drives better responses.

Where your insights land.

The outcomes of your analysis

The AI organizes your Helicone data into these specific categories.

VERSIONING

Prompt history

The AI identifies every iteration of your prompts. You get a clear view of how your instructions have evolved over time.

SENTIMENT

Feedback correlation

The AI maps user feedback directly to the prompt version used. You'll see which versions users liked and which ones they hated.

OPTIMIZATION

Winning instructions

The AI highlights the most successful prompt patterns. This tells you exactly what to keep when you write your next version.

HUMAN

Manual review

The AI flags edge cases where feedback is conflicting. You decide when a prompt needs a total rewrite instead of a small tweak.

The workflow

What your AI does when the data arrives.

The AI handles the heavy lifting of digging through your logs so you don't have to.

  1. Fetch prompt history

    The AI pulls the specific text and metadata for every version of your prompts from your logs.

    get_prompt_versions

  2. Retrieve user feedback

    The AI looks up the specific ratings or comments left by users for those specific runs.

    query_feedback

  3. Cross-reference data

    The AI connects the prompt text to the feedback scores to find the statistical link between them.

    get_prompt_versionsquery_feedback

  4. Summarize findings

    The AI presents a clear comparison of which prompt changes actually moved the needle on user satisfaction.

    query_feedback

Try it

Copy these to start.

Use these prompts to begin your analysis immediately.

Starting points

These are starting points. Swap out the specific prompt names or feedback types to match your actual Helicone setup.

Helicone Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_BY11i3Opi4Qq9KtIJUGQpInAaXPfnwFfbeNDvYuq/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Helicone capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "helicone-llm-observability-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_BY11i3Opi4Qq9KtIJUGQpInAaXPfnwFfbeNDvYuq/mcp"
    }
  }
}
Copy into chat04

  • Compare the 'customer_support_v2' prompt version against the 'customer_support_v1' version using recent user feedback.

  • Show me all prompt versions that resulted in a negative user rating in the last week.

  • Analyze the feedback for the 'creative_writer' prompt to see why users are rating it poorly.

  • Which prompt version has the highest average user rating in my Helicone logs?

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Helicone once, then ask.

Just link your Helicone account. Your credentials stay encrypted, and you can start asking questions about your logs right away.

Connect Helicone to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change my prompts in Helicone?

    No. The AI can only read your prompt versions and feedback. You must manually update your prompts in your own code or the Helicone dashboard.

  • 02

    Can the AI delete old prompt versions?

    No. The AI has no permission to delete or modify any data in your Helicone account.

  • 03

    Does the AI see all my LLM traffic?

    The AI only sees the data it is asked to retrieve through the specific tools provided, such as prompt versions and feedback.

  • 04

    Can I ask about specific user IDs?

    Yes, as long as that user feedback is accessible through the query_feedback tool.

  • 05

    How does the AI know which feedback belongs to which prompt?

    It uses the unique identifiers provided by Helicone to match specific prompt versions to the feedback entries recorded during those runs.

  • More questions about Helicone? The Connector page answers them. See everything the Helicone Connector can do

Connect Helicone to Claude, Cursor, ChatGPT & more