How to Use the NVIDIA Vision MCP in Google ADK

Q: How do I list the available models?

Use the listvisionmodels tool. Your agent can query this to see the active endpoints on the NVIDIA API Catalog before running tasks.

Feed massive batches of visual data to Gemini using the NVIDIA Vision MCP Server integrated with your Google ADK pipelines.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect NVIDIA Vision MCP to Google ADK

Create your Vinkius account to connect NVIDIA Vision to Google ADK and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup NVIDIA Vision with Google ADK

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Query visual data with Google ADK and `visual_question_answering`

The `visual_question_answering` tool analyzes an image from a public URL and answers specific questions about its content. Because Gemini handles massive context windows, your agent can cross-reference these visual answers with corporate data stored in BigQuery. You pass the MCP toolset directly to your LlmAgent during initialization. This lets your agent run complex reasoning loops over hundreds of images without manual pipeline gluing.

Extract precise masks with `image_segmentation`

The `image_segmentation` tool identifies and isolates individual objects within an image. It outputs pixel-level coordinates that your agent can use to catalog inventory or analyze spatial layouts. If you only want to expose specific capabilities, you can use the tool_names filter in the ADK toolset. This restricts access, preventing your agent from invoking heavier tools when it only needs segmentation.

Locate specific objects using `visual_grounding`

The `visual_grounding` tool pinpoints the exact location of a specified object or phrase within an image. It returns bounding boxes that help your agent verify where items are placed in a visual scene. This fits naturally alongside Vertex AI pipelines where spatial coordinates are used for downstream automation. The agent resolves the coordinates and instantly updates your enterprise databases.

Setup guide

Set up NVIDIA Vision MCP in Google ADK

Prerequisites

Python 3.10+ installed
google-adk package (pip install google-adk)
Active Vinkius subscription with a valid endpoint token

1

Install Google ADK

Run pip install google-adk to install the Agent Development Kit. MCP support is included via the McpToolset class.
2

Connect via SSE transport

Use McpToolset.from_server() with SseServerParams pointing to your Vinkius endpoint. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com.
3

Create an LlmAgent

Pass the returned mcp_tools list directly to LlmAgent(tools=mcp_tools). The ADK maps each MCP tool to a native Gemini function call — no manual schema definitions required.
4

Run with any Gemini model

The agent works with any Gemini model (gemini-2.0-flash, gemini-2.5-pro, etc.). Copy the full example on the right to get started with NVIDIA Vision tools in your ADK agent.

agent.py

from google.adk.agents import LlmAgent
from google.adk.tools.mcp_tool.mcp_toolset import McpToolset
from google.adk.tools.mcp_tool.mcp_session_manager import SseServerParams

# Connect to the MCP via SSE
mcp_tools, exit_stack = await McpToolset.from_server(
    connection_params=SseServerParams(
        url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    )
)

# Create your agent with auto-discovered tools
agent = LlmAgent(
    name="NVIDIA Vision_agent",
    model="gemini-2.0-flash",
    instruction="You have access to NVIDIA Vision tools via MCP.",
    tools=mcp_tools,
)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by NVIDIA. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect NVIDIA Vision now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about NVIDIA Vision MCP in Google ADK

Import McpToolset from the SDK and pass your server's HTTP URL. Then, simply provide this toolset to your LlmAgent instance.

Yes, you can pass a list of allowed tool names to the toolset configuration. This prevents the agent from calling tools you want to keep disabled.

Both transports are supported. For hosted cloud environments, the HTTP transport is usually the easiest way to connect your agent.

Use the list_vision_models tool. Your agent can query this to see the active endpoints on the NVIDIA API Catalog before running tasks.

All visual payloads are processed in transit through secure, isolated memory spaces. Vinkius manages the endpoint authentication, so your API keys never leak into agent logs or cloud storage.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript