How to Use the Mapillary MCP in AutoGen

Q: How do AutoGen agents access Mapillary tools?

Use mcpservertools from the AutoGen MCP extension to load the tools. Pass the resulting tool list to your AssistantAgent constructor.

Q: How does AutoGen use Mapillary to resolve asset location conflicts?

One agent calls getmapfeatures to find cataloged signs, while another calls getimagedetections on raw images to verify if those signs actually exist.

Q: What format is required for geographic coordinate searches?

Use standard bounding box format (minlon, minlat, maxlon, maxlat) when calling searchimages or searchsequences.

Build AutoGen agent teams that debate and verify physical road assets using real-time Mapillary data.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Mapillary MCP to AutoGen

Create your Vinkius account to connect Mapillary to AutoGen and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Mapillary with AutoGen

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Verify physical road assets with AutoGen

`get_detection_value` retrieves the specific classification, coordinates, and image associations for any detected object. The tool serves as the ground truth source for your agent discussions. In an AutoGen group chat, a quality-assurance agent uses this tool to challenge another agent's asset classification. They cross-reference coordinates to ensure your asset inventory remains accurate.

Locate street-level images via this MCP Server

`search_images` scans geographic bounding boxes to locate all matching street-level image IDs and metadata. The tool returns precise compass angles and coordinates for every frame. Your coordination agent uses these search results to assign specific image IDs to worker agents. The team then processes these images in parallel to extract localized street data.

Audit physical routes using image sequences

`get_sequence` fetches the ordered timeline of images captured during a single driving run. The tool provides the sequential context needed to trace a vehicle's path. Your auditing agent steps through this sequence to verify that traffic signs are visible from all driving directions. The agent flags obscured signs and logs them for maintenance crews.

Setup guide

Set up Mapillary MCP in AutoGen

Prerequisites

Python 3.10+ installed
autogen-ext[mcp] package
Active Vinkius subscription with a valid endpoint token

1

Install AutoGen with MCP
Run pip install "autogen-ext[mcp]" autogen-agentchat. The MCP extension includes mcp_server_tools for stateless tool access.
2

Fetch tools from the MCP
Call mcp_server_tools(SseServerParams(url=...)) with your Vinkius endpoint. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com.
3

Run your agent
Pass the tools to AssistantAgent and call agent.run(). The agent invokes Mapillary tools and returns structured results.

agent.py

from autogen_ext.tools.mcp import SseServerParams, mcp_server_tools
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

server_params = SseServerParams(
    url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)

tools = await mcp_server_tools(server_params)

agent = AssistantAgent(
    name="Mapillary_assistant",
    model_client=OpenAIChatCompletionClient(model="gpt-4o"),
    tools=tools,
)

result = await agent.run("List recent Mapillary data")
print(result.messages[-1].content)

Get your connection token →

Prerequisites

Python 3.10+ installed
autogen-ext[mcp] + autogen-agentchat
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Same packages as above. McpWorkbench is ideal when your agent needs stateful sessions across multiple tool calls.
2

Use McpWorkbench as context manager
Wrap your agent in async with McpWorkbench(...) to maintain shared state and resources. The workbench manages the full MCP session lifecycle.
3

Run with workbench
Pass workbench=workbench to your agent. State is preserved across multiple tool calls within the same session.

agent.py

from autogen_ext.tools.mcp import McpWorkbench, SseServerParams
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

server_params = SseServerParams(
    url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)

async with McpWorkbench(server_params) as workbench:
    agent = AssistantAgent(
        name="Mapillary_assistant",
        model_client=OpenAIChatCompletionClient(model="gpt-4o"),
        workbench=workbench,
    )

    result = await agent.run("List recent Mapillary data")
    print(result.messages[-1].content)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Mapillary. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Mapillary now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Mapillary MCP in AutoGen

Use mcp_server_tools from the AutoGen MCP extension to load the tools. Pass the resulting tool list to your AssistantAgent constructor.

Yes. The MCP Server handles concurrent requests, allowing different agents to query images and sequences at the same time during a debate.

One agent calls get_map_features to find cataloged signs, while another calls get_image_detections on raw images to verify if those signs actually exist.

Use standard bounding box format (min_lon, min_lat, max_lon, max_lat) when calling search_images or search_sequences.

The MCP server operates on a zero-trust architecture, passing camera angles, sequence IDs, and street-level images directly to your AutoGen runtime. No imagery or sequence IDs are cached or stored externally.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript