How to Use the NVIDIA NIM MCP in LangChain

Q: Can I trace NVIDIA NIM tool calls in LangChain using LangSmith?

Yes, every execution of tools like nimlistmodels or nimgetmetrics is recorded as a distinct tool run in your LangSmith dashboard. You get full visibility into the latency, exact input parameters, and raw outputs of your local GPU containers.

Q: How do I pass the model list to a LangChain agent?

You call client.gettools() after initializing the MCP adapter, which registers nimlistmodels as an active tool. The agent executes this tool to discover which local LLMs are active before routing user prompts.

Get raw GPU telemetry and scale local container replicas directly inside your LangChain reasoning loops.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect NVIDIA NIM MCP to LangChain

Create your Vinkius account to connect NVIDIA NIM to LangChain and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup NVIDIA NIM with LangChain

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Run health checks inside LangChain agents

The `nim_check_health_ready` tool exposes the readiness state of your local inference container directly to your active reasoning chain. If a model load fails, the agent intercepts the error and halts the execution pipeline before sending queries. You can chain this with `nim_check_health_live` to confirm host container responsiveness. LangSmith traces every step of this validation flow, showing you exactly when the model initialized and how much latency occurred during the check.

Map GPU memory constraints in LangChain pipelines

The `nim_get_gpu_status` tool outputs real-time VRAM allocation and topological limits directly into your LangChain agent's context window. This prevents your chains from sending high-batch inference requests when the physical hardware is already saturated. By checking `nim_get_metadata` in the same sequence, your chain knows the precise configuration bounds of the active model. The agent uses this data to dynamically route workloads to alternative local nodes or adjust batch sizes on the fly.

Automate replica scaling using this MCP Server

The `nim_scale_replicas` tool lets your LangChain agent adjust active hardware replication assignments based on raw telemetry feedback. When metrics spike, the agent triggers a scale-up action to spin up extra container instances. Your agent pulls real-time load data using `nim_get_metrics` to calculate the exact replica count required. This creates a closed-loop scaling mechanism where the LangChain framework directly manages your local hardware allocations using this MCP Server.

Setup guide

Set up NVIDIA NIM MCP in LangChain

Prerequisites

Python 3.10+ installed
langchain-mcp-adapters + langgraph packages
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Run pip install langchain-mcp-adapters langgraph langchain-openai. The MCP adapters package converts MCP tools into native LangChain BaseTool objects.
2

Connect via HTTP transport
Use MultiServerMCPClient with "transport": "http" pointing to your Vinkius endpoint. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com.
3

Create a ReAct agent
Pass the discovered tools to create_react_agent() from LangGraph. The agent automatically routes NVIDIA NIM tool calls through the MCP protocol.
4

Run with any LLM
Swap ChatOpenAI for ChatAnthropic, ChatGoogleGenerativeAI, or any LangChain-compatible model. The MCP tools work identically across all providers.

agent.py

from langchain_mcp_adapters.client import MultiServerMCPClient
from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI

async with MultiServerMCPClient({
    "nvidia-nim-mcp": {
        "transport": "http",
        "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp",
    }
}) as client:
    tools = client.get_tools()

    agent = create_react_agent(
        ChatOpenAI(model="gpt-4o"),
        tools,
    )
    result = await agent.ainvoke({
        "messages": "List recent NVIDIA NIM transactions"
    })
    print(result["messages"][-1].content)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by NVIDIA NIM. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect NVIDIA NIM now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about NVIDIA NIM MCP in LangChain

The framework uses the MultiServerMCPClient to manage connection states. If the local NIM container drops, your chain catches the connection error and can route queries to a fallback server.

Yes, every execution of tools like nim_list_models or nim_get_metrics is recorded as a distinct tool run in your LangSmith dashboard. You get full visibility into the latency, exact input parameters, and raw outputs of your local GPU containers.

Yes, you can combine this server with other endpoints inside a single MultiServerMCPClient instance. Your LangChain agent can query GPU status with nim_get_gpu_status while simultaneously pulling database records from a separate server.

You call client.get_tools() after initializing the MCP adapter, which registers nim_list_models as an active tool. The agent executes this tool to discover which local LLMs are active before routing user prompts.

No, your raw container logs, telemetry, and VRAM limits remain strictly inside your local setup. Vinkius handles this MCP Server connection securely through an ephemeral sandbox, meaning no physical hardware configuration data ever leaks to external networks.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript