• VERSIONING
  • ANALYSIS
  • AUDIT
  • HUMAN

How to compare prompt versions with your AI

Connect Langfuse to your AI client to access your prompt history. You can then ask your agent to analyze how different versions of a template change the results.

Ask AI about this page

Short answer

How can I compare different versions of my prompts?

Your AI uses the Langfuse connection to pull specific prompt templates and their historical iterations. You can then ask your agent to identify differences in instructions and how those changes affect the output you're seeing.

The outcomes.

What you get from version comparison.

The AI handles the data retrieval so you can focus on the logic.

VERSIONING

Track instruction changes

The AI pulls specific versions of your templates. You'll see exactly what changed between the old and new instructions.

ANALYSIS

Compare output shifts

Your agent looks at how a tweak in the prompt changed the model's response. This helps you catch regressions immediately.

AUDIT

Inspect prompt history

You can query the full history of a template. This gives you a clear timeline of how your prompt engineering has evolved.

HUMAN

Decide on the winner

The AI presents the differences, but you decide which version actually performs better for your specific use case.

The workflow

What your AI does when the request arrives.

The AI acts as a bridge between your chat interface and your Langfuse data.

  1. Fetch templates

    The AI looks up the specific prompt templates you're interested in within Langfuse.

    list_prompts

  2. Retrieve versions

    It pulls the specific historical iterations of those templates to prepare for comparison.

    list_prompts

  3. Identify differences

    Your agent compares the text of the different versions to find instruction changes.

    list_prompts

  4. Summarize impact

    The AI explains how those instruction changes might lead to different model behaviors.

    list_prompts

Try it

Copy these to start.

Use these prompts to begin your comparison.

Starting points

These are starting points. Swap out the template names for the ones you actually use in Langfuse.

Langfuse Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_sTXOynCd5hRTA7WbjY0nBUsTjmjwRIVY989VA7VU/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Langfuse capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "langfuse-llm-tracing-evals-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_sTXOynCd5hRTA7WbjY0nBUsTjmjwRIVY989VA7VU/mcp"
    }
  }
}
Copy into chat04

  • List my prompt templates in Langfuse so I can see what's available.

  • Compare the latest version of the 'customer_support_v2' prompt with its previous version.

  • Show me the history for the 'summarizer_template' and tell me what changed in the last three updates.

  • Find the 'data_extractor' prompt and explain how the new instructions differ from the old ones.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Langfuse once, then ask.

Just provide your Langfuse credentials. The connection is encrypted, and you can start querying your prompt history immediately.

Connect Langfuse to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change my prompts in Langfuse?

    No. The AI can only read and list your existing prompt templates and versions.

  • 02

    Can the AI delete old prompt versions?

    No. The AI has no capability to delete or modify your data in Langfuse.

  • 03

    Do I need to manually export my prompts first?

    No. Once connected, your AI can pull the data directly using the list_prompts tool.

  • 04

    Can the AI run an evaluation on these prompts?

    No. The AI can only retrieve the prompt text and versions for you to inspect.

  • 05

    How does the AI know which version is which?

    It uses the versioning data provided by Langfuse to distinguish between different iterations of the same template.

  • More questions about Langfuse? See everything the Langfuse Connector can do

Connect Langfuse to Claude, Cursor, ChatGPT & more