How to Use the LangSmith (LLM Observability & Hub) MCP in VS Code Copilot

Q: How do I track production errors using LangSmith (LLM Observability & Hub) in VS Code Copilot?

You invoke listruns to view recent history and filter for failed requests. Then use getrun to inspect the exact inputs that triggered the error. This helps the whole team debug faster.

Q: Can I manage prompt templates with LangSmith (LLM Observability & Hub) in VS Code Copilot?

Absolutely, the listprompts tool lets you pull and check your templates. You can ensure the team is referencing the correct hub-based prompts. It avoids version drift in your code.

Q: How do I see active evaluation queues in VS Code Copilot using LangSmith (LLM Observability & Hub)?

Use the listannotationqueues tool to get a summary of pending items. This keeps your team updated on what needs human feedback. It integrates evaluation right into your development cycle.

Connect LangSmith (LLM Observability & Hub) to VS Code Copilot to monitor team-wide LLM performance and manage evaluation tasks.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect LangSmith (LLM Observability & Hub) MCP to VS Code Copilot

Create your Vinkius account to connect LangSmith (LLM Observability & Hub) to VS Code Copilot and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup LangSmith (LLM Observability & Hub) with VS Code Copilot

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Monitor team pipelines in VS Code

Use `list_projects` to define the scope of your team's observability. This gives you a clear view of all active pipelines currently monitored in your environment. Once you select a project, `list_runs` allows you to see how your models are performing in production. It gives your entire team a shared view of real-time model behavior.

Audit prompt versions across the team

Keep everyone on the same page with `list_prompts`. This tool pulls your templates directly into the editor, so every team member knows which prompt version is currently live. It eliminates the confusion of local versus production versions. You can verify the exact prompt logic being used in your current codebase with a single command.

Coordinate human-in-the-loop evaluations

Use `list_annotation_queues` to track what is currently pending review. This keeps your team focused on the most critical evaluation tasks that require human input. You can also use `list_datasets` to confirm your testing suites are updated. This ensures that your team is running evaluations against the correct data, keeping your model performance metrics accurate.

Setup guide

Set up LangSmith (LLM Observability & Hub) MCP in VS Code Copilot

Prerequisites

VS Code 1.99 or later with GitHub Copilot extension
Active Vinkius subscription with a valid endpoint token

1

Open MCP configuration

Open the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and run "MCP: Add Server". Select HTTP (Streamable) as the server type. VS Code will create .vscode/mcp.json in your workspace.
2

Add the LangSmith (LLM Observability & Hub) MCP

Paste the JSON snippet shown on the right into your .vscode/mcp.json. Replace [YOUR_TOKEN_HERE] with your endpoint token from cloud.vinkius.com.
3

Switch to Agent mode

Open Copilot Chat (Cmd+Shift+I / Ctrl+Shift+I) and switch to Agent mode using the dropdown. MCP tools are only available in Agent mode — they do not appear in Edit or Ask modes.
4

Verify the connection

In the Copilot Chat input, type # to list available tools. You should see the LangSmith (LLM Observability & Hub) tools listed. Try asking: "List my recent LangSmith (LLM Observability & Hub) transactions" and Copilot will invoke them automatically.

.vscode/mcp.json

{
  "mcpServers": {
    "langsmith-llm-observability-hub-mcp": {
      "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    }
  }
}

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by LangSmith. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect LangSmith (LLM Observability & Hub) now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about LangSmith (LLM Observability & Hub) MCP in VS Code Copilot

Yes, you can include the server setup in your .vscode/mcp.json file. This allows you to share the tool access across your entire team. Everyone gets the same observability capabilities.

You invoke `list_runs` to view recent history and filter for failed requests. Then use `get_run` to inspect the exact inputs that triggered the error. This helps the whole team debug faster.

Absolutely, the `list_prompts` tool lets you pull and check your templates. You can ensure the team is referencing the correct hub-based prompts. It avoids version drift in your code.

Use the `list_annotation_queues` tool to get a summary of pending items. This keeps your team updated on what needs human feedback. It integrates evaluation right into your development cycle.

The tools access your LLM invocation logs, including prompt templates, input variables, and model output traces. You should verify that no sensitive customer information is stored in these traces. Always follow your organization's data handling policies.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript