How to Use the LangSmith (LLM Observability & Hub) MCP in AutoGen

Q: Can AutoGen agents compare multiple pipelines using LangSmith (LLM Observability & Hub)?

Yes, they can call listprojects to get a list of active pipelines. They can then cross-reference metrics between those projects to identify bottlenecks.

Let your AutoGen agents debate observability metrics and optimize pipelines using real-time trace data.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect LangSmith (LLM Observability & Hub) MCP to AutoGen

Create your Vinkius account to connect LangSmith (LLM Observability & Hub) to AutoGen and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup LangSmith (LLM Observability & Hub) with AutoGen

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Debate observability with AutoGen

Enable your agents to challenge each other on performance metrics. One agent can call `list_runs` to present raw latency data, while another evaluates the findings. This creates a feedback loop where agents debate why a specific LLM invocation failed. They converge on a fix based on hard data rather than guesses.

Manage prompts via AutoGen agents

Let your specialized agents inspect and update prompt templates during a conversation. An agent can call `list_prompts` to see the current version before suggesting an improvement. This allows for collaborative prompt engineering. The agent proposing changes can justify them based on the traces it just retrieved.

Coordinate evaluation in AutoGen

Assign an agent to manage your evaluation queues. By using `list_annotation_queues`, the agent can identify high-priority items that need human input. This keeps your evaluation process moving. Your agents act as the orchestrators for your testing workflow, ensuring nothing gets missed.

Setup guide

Set up LangSmith (LLM Observability & Hub) MCP in AutoGen

Prerequisites

Python 3.10+ installed
autogen-ext[mcp] package
Active Vinkius subscription with a valid endpoint token

1

Install AutoGen with MCP
Run pip install "autogen-ext[mcp]" autogen-agentchat. The MCP extension includes mcp_server_tools for stateless tool access.
2

Fetch tools from the MCP
Call mcp_server_tools(SseServerParams(url=...)) with your Vinkius endpoint. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com.
3

Run your agent
Pass the tools to AssistantAgent and call agent.run(). The agent invokes LangSmith (LLM Observability & Hub) tools and returns structured results.

agent.py

from autogen_ext.tools.mcp import SseServerParams, mcp_server_tools
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

server_params = SseServerParams(
    url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)

tools = await mcp_server_tools(server_params)

agent = AssistantAgent(
    name="LangSmith (LLM Observability & Hub)_assistant",
    model_client=OpenAIChatCompletionClient(model="gpt-4o"),
    tools=tools,
)

result = await agent.run("List recent LangSmith (LLM Observability & Hub) data")
print(result.messages[-1].content)

Get your connection token →

Prerequisites

Python 3.10+ installed
autogen-ext[mcp] + autogen-agentchat
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Same packages as above. McpWorkbench is ideal when your agent needs stateful sessions across multiple tool calls.
2

Use McpWorkbench as context manager
Wrap your agent in async with McpWorkbench(...) to maintain shared state and resources. The workbench manages the full MCP session lifecycle.
3

Run with workbench
Pass workbench=workbench to your agent. State is preserved across multiple tool calls within the same session.

agent.py

from autogen_ext.tools.mcp import McpWorkbench, SseServerParams
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

server_params = SseServerParams(
    url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)

async with McpWorkbench(server_params) as workbench:
    agent = AssistantAgent(
        name="LangSmith (LLM Observability & Hub)_assistant",
        model_client=OpenAIChatCompletionClient(model="gpt-4o"),
        workbench=workbench,
    )

    result = await agent.run("List recent LangSmith (LLM Observability & Hub) data")
    print(result.messages[-1].content)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by LangSmith. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect LangSmith (LLM Observability & Hub) now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about LangSmith (LLM Observability & Hub) MCP in AutoGen

Register the MCP server tools with your AssistantAgent. The agents will then determine when to call `list_runs` or `list_projects` during their conversation.

Yes, they can call `list_projects` to get a list of active pipelines. They can then cross-reference metrics between those projects to identify bottlenecks.

It provides access to trace logs, prompt templates, and evaluation datasets. This includes the full payload of LLM inputs and outputs for your agents to analyze.

They use the tools to fetch objective data. By comparing the output of `get_run` for different invocations, they can verify which agent's conclusion is factually correct.

You control the scope of the tools provided. The agents only interact with the data you explicitly expose via the MCP server interface.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python