• COST
  • SPEED
  • AUDIT
  • HUMAN

How to track token usage and latency with your AI

Connect LangSmith to your agent to turn raw trace data into clear answers about cost and speed.

Ask AI about this page

Short answer

How can I monitor my AI's token consumption and response times?

Your agent uses the LangSmith connector to pull recent traces from your project. This gives you a direct look at the token counts and timing for specific LLM calls. You'll see exactly where your budget and latency are going.

Where your data lands.

What you get from your traces.

The AI processes your LangSmith data to highlight performance trends.

COST

Token usage breakdown

The AI identifies which calls are burning through your token budget. You get a clear view of consumption patterns across different runs.

SPEED

Latency analysis

Find out which specific steps in your chain are slowing things down. The agent flags high-latency calls for you to investigate.

AUDIT

Trace history

The agent pulls a list of recent runs so you can audit specific interactions. This helps you verify that your prompts are behaving as expected.

HUMAN

Manual optimization

The AI identifies the bottlenecks, but you decide how to rewrite prompts or change models to fix them. You stay in control of the actual logic changes.

The workflow

What your AI does when <the thing arrives>.

The agent acts as a bridge between your chat interface and your LangSmith project data.

  1. Fetch traces

    The agent reaches into your LangSmith project to grab the most recent execution data.

    langsmith_list_runs

  2. Calculate costs

    It looks at the token metadata within those runs to sum up usage for specific calls.

    langsmith_list_runs

  3. Measure timing

    The agent compares start and end timestamps to find out how long each step took.

    langsmith_list_runs

  4. Report findings

    The agent summarizes the most expensive or slowest runs so you can act on them.

    langsmith_list_runs

Try it

Copy these to start.

Use these prompts to begin your analysis.

Starting points

Swap out the project names and timeframes to match your actual LangSmith setup.

LangSmith Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_JBueIzhraYvzf2S9aR0UP9Q6bjldfIBgmwxTvRHt/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — LangSmith capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "langsmith-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_JBueIzhraYvzf2S9aR0UP9Q6bjldfIBgmwxTvRHt/mcp"
    }
  }
}
Copy into chat04

  • List the most recent runs from my 'production-agent' project in LangSmith.

  • Which of the recent traces in my 'test-suite' project had the highest token usage?

  • Show me the latency for the last five runs in my 'chatbot-v2' project.

  • Find any runs in my 'eval-project' that took longer than 5 seconds to complete.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect LangSmith once, then ask.

Just provide your LangSmith credentials once. Your connection stays encrypted, and you can immediately start asking your agent about your traces.

Connect LangSmith to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change my LangSmith settings?

    No. The AI can only read trace data. It cannot modify your LangSmith projects or settings.

  • 02

    Does the AI delete any of my traces?

    No. The AI only has permission to list and read runs. It cannot delete or edit your history.

  • 03

    Can I see the exact prompt used in a trace?

    Yes, the agent can pull the input and output data from the traces it retrieves.

  • 04

    How recent is the data the AI sees?

    The AI sees whatever recent runs are available in your LangSmith project based on your request.

  • 05

    Can the AI fix my slow LLM calls?

    No. The AI can identify which calls are slow, but you have to manually adjust your code or prompts to fix them.

  • More questions about LangSmith? The Connector page answers them. See everything the LangSmith Connector can do

Connect LangSmith to Claude, Cursor, ChatGPT & more