• LATENCY
  • EVENTS
  • ROOT CAUSE
  • HUMAN

How to debug LLM latency spikes with your AI

Connect New Relic AI to your agent to pinpoint exactly which model calls are slowing down your application.

Ask AI about this page

Short answer

How do I find out why my LLM calls are slow?

Your agent pulls latency metrics and event logs to find the exact calls causing delays. You get a breakdown of performance bottlenecks and the specific event data tied to them.

Investigation outcomes

Where your data lands.

The AI organizes your observability data into actionable categories.

LATENCY

Identify slow calls

The agent uses latency metrics to find outliers. You see which specific model requests are exceeding your thresholds.

EVENTS

Trace specific errors

The agent pulls event data to show what happened during a spike. You get the context needed to understand the failure.

ROOT CAUSE

Pinpoint the bottleneck

The AI correlates timing data with event logs. You find if the delay is in the model, the network, or the payload.

HUMAN

Manual investigation

The AI flags issues that require a deep dive in the New Relic dashboard. You decide when to move from chat to the full UI.

The workflow

What your AI does when the latency spike arrives.

The agent acts as a bridge between your chat interface and your New Relic telemetry.

  1. Scan for delays

    The agent checks your recent telemetry to find spikes in response times.

    query_llm_latency

  2. Inspect the logs

    It pulls the specific event data associated with those slow timestamps.

    query_llm_events

  3. Correlate data

    The agent matches latency numbers against the event details to find patterns.

    query_llm_latencyquery_llm_events

  4. Summarize findings

    The agent presents a clear list of the offending calls and their impact.

    query_llm_latency

Try it

Copy these to start.

Use these prompts to begin your investigation.

Starting points

Replace the bracketed text with your specific service names or time ranges.

New Relic AI Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_oqdaAroeFoXBv9yPI4WsHZZZZZuzqhwVoSn1YCyq/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — New Relic AI capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "new-relic-ai-llm-observability-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_oqdaAroeFoXBv9yPI4WsHZZZZZuzqhwVoSn1YCyq/mcp"
    }
  }
}
Copy into chat04

  • Show me the LLM latency spikes for the production-api service from the last hour.

  • Which specific model calls in the 'chat-completion' module had the highest latency today?

  • Find the event data for the latency spike that happened at 10:15 AM.

  • Compare the latency metrics for the GPT-based calls against the Llama-based calls in my recent logs.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect New Relic AI once, then ask.

One connection gives your agent access to your observability data. Your credentials stay encrypted and secure.

Connect New Relic AI to your AI

FAQ

How this task behaves.

  • 01

    Can the AI fix the latency issues automatically?

    No. The AI only reads latency and event data to tell you what is happening. You have to implement the fix in your code.

  • 02

    Can the AI delete old telemetry data?

    No. The agent only has permission to query data, not to modify or delete anything in New Relic.

  • 03

    Does the AI need my New Relic login?

    No. You connect the MCP through Vinkius, which handles the connection securely.

  • 04

    Can it see all my New Relic data?

    No. It can only access data related to LLM latency and events through the provided tools.

  • 05

    How does it know which calls are slow?

    It uses the latency tool to scan for response times that exceed your normal baseline.

  • More questions about New Relic AI? The Connector page answers them. See everything the New Relic AI Connector can do

Connect New Relic AI to Claude, Cursor, ChatGPT & more