# LangSmith MCP for AI Agents AI Agent Connect

> LangSmith MCP connects your AI agent to the LangChain observability platform. It allows you to pull project metrics, browse recent traces, and dig into specific execution details directly from your chat interface. You can monitor latency, token usage, and success rates across your entire LLM stack without switching tabs.

## Overview
- **Category:** friends-mcp
- **Price:** Free
- **Endpoint:** https://edge.vinkius.com/vk_preview_JBueIzhraYvzf2S9aR0UP9Q6bjldfIBgmwxTvRHt/ai-agent-connect
- **Tags:** llm-observability, tracing, evaluation, performance-metrics, ai-debugging, prompt-testing

## Description

When you're shipping LLM applications, you need to know exactly what's happening under the hood. You can't just guess if your agent is hallucinating or why a specific chain is lagging. This Connector lets your agent pull that data directly. Instead of jumping between browser tabs to check your dashboard, you can ask your agent to summarize the health of your production projects or find a specific failed run from last night. It pulls in the hard numbers like median latency, feedback scores, and token costs so you can make decisions based on facts. By adding this to your stack through the Vinkius catalog, you give your agent a window into your production environment. It's about moving from I think it's working to I know it's working because the agent can surface the exact trace data you need to debug a regression or optimize a prompt. You get a clear view of every LLM call, chain execution, and tool action as they happen in the wild.

## Tools

### langsmith_get_run
Pulls detailed info about a specific trace ID to help you figure out why a certain call failed. It shows the full execution path and inputs.

### langsmith_list_projects
Shows all your tracing projects along with high-level stats like total runs and median latency. Use it to get a project overview.

### langsmith_list_runs
Lists recent traces in a project so you can quickly see success rates and token usage. It helps you spot errors in your recent history.

## Prompt Examples

**Prompt:** 
```
Show me the latency for my production project.
```

**Response:** 
```
I've pulled the metrics for your **production-agent** project:

| Project Name | ID | Median Latency | Total Runs |
|---|---|---|---|
| production-agent | `a1b2c3d4` | **340ms** | 12,450 |

Your production environment is currently responding within the expected 340ms range.
```

**Prompt:** 
```
Find the last 5 runs in the staging project.
```

**Response:** 
```
Here are the last 5 runs from the **staging-chatbot** project:

*   **agent_executor** (success) - 1,250 tokens
*   **gpt-4o** (success) - 890 tokens
*   **tool:web_search** (success) - 340ms
*   **agent_executor** (error) - 450 tokens
*   **gpt-4o** (success) - 1,100 tokens
```

**Prompt:** 
```
What happened in run a0b1c2?
```

**Response:** 
```
I've retrieved the details for run `a0b1c2`:

**Status:** Error
**Type:** Chain
**Error Message:** Tool 'web_search' returned timeout after 30s. 

The agent attempted to retry the action 3 times before the process finally failed.
```

## Capabilities

### Summarize project health
Get a high-level look at your tracing projects including total runs and median latency.

### Browse recent traces
Pull a list of the latest runs to see success rates and token consumption.

### Inspect specific run details
Retrieve the full execution trace for a specific ID to see inputs and outputs.

### Monitor token consumption
Track how many tokens your agent is using across different project runs.

### Track execution latency
Identify which parts of your LLM chain are slowing down your response times.

## Use Cases

### Debugging a production failure
An engineer notices a chatbot is acting weird. They ask the agent to find the last 5 failed runs in the production project to see the error messages.

### Cost auditing
A manager wants to know the budget impact. They ask the agent to list all projects and summarize the total token usage for the week.

### Latency checks
A dev wants to see if a new prompt is slower. They ask the agent to compare the median latency of the staging project against production.

### Feedback review
A team wants to see user reactions. They ask the agent to pull projects with the highest feedback scores to see what is landing well.

## Benefits

- Stop tab switching by getting project metrics directly in your chat window for a faster overview of your work.
- Identify production errors faster by pulling specific trace data to see the exact input that caused a failure.
- Track your costs accurately by seeing real-time token usage across all recent runs without manual calculation.
- Spot performance regressions early by checking median latency stats for every project in your account.
- Debug complex agent chains by viewing the full execution path of a failed run instantly to see where it stalled.

## How It Works

The bottom line is you get instant access to your LLM production data without leaving your chat interface.

1. Subscribe to the Connector and grab your LangSmith API key from your account settings.
2. Plug the API key into your AI client configuration to establish the connection.
3. Ask your agent to pull project metrics or debug specific traces in your chat.

## Frequently Asked Questions

**What can I do with LangSmith MCP?**
You can use it to monitor your LLM applications in real-time. It allows your agent to pull project metrics, browse recent traces, and get deep details on specific failed runs.

**How do I see my LLM costs with LangSmith MCP?**
The Connector can pull recent runs which include token consumption data. You can ask your agent to summarize these counts to see how much your project is costing you.

**Can I debug failed agent actions using LangSmith MCP?**
Yes, you can pull the full execution trace for a specific run ID. This shows you the exact inputs and outputs of every step in the chain, making it easy to find where things went wrong.

**Does LangSmith MCP show project health?**
It does by providing aggregate metrics for all your tracing projects. You can quickly see things like median latency and total run counts to gauge overall health.

**Is LangSmith MCP for production monitoring?**
Exactly. It is designed to give you visibility into live LLM applications so you can catch errors, monitor latency, and track costs as they happen.

**How do I get my LangSmith API key?**
You can find your API key in your LangSmith account settings. Once you have it, you just need to add it to your Connector configuration to get started.

**What is LangSmith and why do I need it?**
LangSmith is the 'Datadog for LLM applications'. Without observability, AI agents in production are black boxes — you can't see what they're doing, why they fail, or how much they cost. LangSmith traces every LLM call, chain execution, and tool use, giving you complete visibility into inputs, outputs, latency, token usage, and error rates.

**Does LangSmith work only with LangChain?**
No! While LangSmith is built by the LangChain team and has native LangChain/LangGraph integration, it works with any LLM application. You can trace OpenAI, Anthropic, or any LLM provider directly using the REST API. It also integrates with CrewAI, AutoGen, and other frameworks.

**How much does LangSmith cost?**
LangSmith offers a generous free tier with 5,000 traces per month — no credit card required. The Developer plan is $39/month with 50,000 traces. Enterprise plans include SSO, RBAC, dedicated support, and unlimited traces with volume discounts.