• COST
  • SPEED
  • TRENDS
  • HUMAN

How to monitor LLM token usage with your AI

Connect Datadog AI to your agent. Once connected, you can ask your agent to pull specific telemetry data from your Datadog account.

Ask AI about this page

Short answer

How can I track LLM token usage and latency with my AI?

Your agent uses the Datadog AI connector to query numeric telemetry. You get direct access to token counts and model latency metrics through your chat interface. This breakdown shows you how the agent handles these requests.

The results you get.

Where your data lands.

The agent pulls specific metrics from Datadog and brings them into your conversation.

COST

Token usage tracking

The agent pulls token counts from your telemetry. You'll see exactly how much each model call is costing you.

SPEED

Latency monitoring

The agent retrieves response time data. You can identify which model calls are running slow.

TRENDS

Historical analysis

The agent queries past metrics. You get a view of how your model performance has changed over time.

HUMAN

Manual decisions

The agent provides the numbers. You decide if you need to change your prompts or switch models.

The workflow

What your AI does when the data arrives.

The agent acts as the interface between your chat and your Datadog metrics.

  1. Fetch metrics

    The agent looks up the specific numeric telemetry you requested in Datadog.

    query_metrics

  2. Analyze usage

    The agent processes the token counts to give you a clear total.

    query_metrics

  3. Check latency

    The agent pulls response time data to find delays in your model calls.

    query_metrics

  4. Report data

    The agent delivers the requested numbers directly into your chat window.

    query_metrics

Try it

Copy these to start.

Use these prompts to begin monitoring your models.

Starting points

These are opening lines. Swap out the metric names or time ranges to match your specific Datadog setup.

Datadog Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_bWdhwHEHDioKsLvhV1Lvv7ec4t2gm8k5cIgCZeWx/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Datadog capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "datadog-ai-llm-observability-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_bWdhwHEHDioKsLvhV1Lvv7ec4t2gm8k5cIgCZeWx/mcp"
    }
  }
}
Copy into chat04

  • What was the total token usage for my LLM service over the last 24 hours?

  • Show me the average latency for my production model from this morning.

  • Are there any spikes in token consumption for the agent in the last hour?

  • Compare the latency of my model calls between yesterday and today.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Datadog AI once, then ask.

Connect your account with one click. Your credentials stay encrypted on our side, and you can start querying metrics immediately.

Connect Datadog AI to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change my Datadog alerts?

    No. The agent can only read numeric telemetry. It cannot modify your Datadog settings or alerts.

  • 02

    Can the AI delete my metrics?

    No. The agent is read-only. It has no way to delete or change your data.

  • 03

    Can I monitor non-LLM metrics?

    Yes. If the data is a numeric metric in Datadog, the agent can query it.

  • 04

    Does the AI create Datadog dashboards?

    No. The agent provides data in your chat, but it cannot build or edit dashboards in Datadog.

  • 05

    How does the agent find the right data?

    The agent uses the metric names you provide in your prompt to find the right telemetry.

  • More questions about Datadog AI? See everything the Datadog AI Connector can do

Connect Datadog to Claude, Cursor, ChatGPT & more