How to Use the Firecrawl MCP in Pydantic AI

Q: What happens if a website blocks the scraper?

The scrapepage tool uses built-in proxies and anti-bot bypasses to get the content. If it fails, it returns a clean error that your agent can handle gracefully using standard Python try-except blocks.

Run type-safe web scraping and crawls with strict schema validation in your Pydantic AI workflows.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Firecrawl MCP to Pydantic AI

Create your Vinkius account to connect Firecrawl to Pydantic AI and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Firecrawl with Pydantic AI

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Type-safe page scraping with Pydantic AI

The `scrape_page` tool extracts clean Markdown from any URL while handling anti-bot systems and dynamic content. This Firecrawl MCP Server integrates directly with Pydantic AI to ensure the returned web data matches your exact structural schemas at runtime. You eliminate the risk of silent parsing failures by validating the output immediately. If a page layout changes, your Pydantic AI agent catches the schema mismatch rather than passing corrupted data down the chain.

Mapping sites with this MCP Server

The `map_site` tool discovers all URLs on a website and returns them as a structured list. It allows your Pydantic AI agent to validate site architecture before initiating deep scraping operations. By feeding this sitemap into a validated Pydantic model, you guarantee your agent only processes URLs that match your specific domain constraints. This keeps your data pipelines clean and predictable.

Verified web search queries

The `search_web` tool combines search engine queries with automatic page scraping to return clean Markdown from top results. It gives your Pydantic AI agent a reliable way to fetch real-time information without writing custom scraping logic. The tool's output is structured so that your agent can easily parse search results. This makes it simple to validate facts and ingest current web data while maintaining strict type safety.

Setup guide

Set up Firecrawl MCP in Pydantic AI

Prerequisites

Python 3.10+ installed
pydantic-ai-slim[fastmcp] package
Active Vinkius subscription with a valid endpoint token

1

Install Pydantic AI with FastMCP
Run pip install "pydantic-ai-slim[fastmcp]". The FastMCP toolset replaces the deprecated MCPServerHTTP class with full protocol support.
2

Configure the FastMCPToolset
Pass a JSON-style config dict to FastMCPToolset with your Vinkius URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. Supports Streamable HTTP, SSE, and Stdio transports.
3

Create and run your agent
Pass the toolset to Agent(toolsets=[toolset]) and call agent.run(). Swap openai:gpt-4o for any supported model — Anthropic, Google, Mistral, or Groq.

agent.py

from pydantic_ai import Agent
from pydantic_ai.toolsets.fastmcp import FastMCPToolset

toolset = FastMCPToolset({
    "mcpServers": {
        "firecrawl-mcp": {
            "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
        }
    }
})

agent = Agent(
    "openai:gpt-4o",
    toolsets=[toolset],
    system_prompt="You have access to Firecrawl tools.",
)

result = await agent.run("List recent Firecrawl transactions")
print(result.output)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Firecrawl. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Firecrawl now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Firecrawl MCP in Pydantic AI

You pass the Vinkius MCP Server URL to the toolset constructor. The Pydantic AI agent automatically registers the scraping tools and enforces type validation on their inputs and outputs.

Yes, the MCP Server works natively with Pydantic AI's async runtime. You can run multiple `scrape_page` calls concurrently without blocking your main event loop.

Absolutely. You can define a Pydantic model for the search output, and Pydantic AI will validate the scraped Markdown and metadata returned by `search_web` before your agent processes it.

The `scrape_page` tool uses built-in proxies and anti-bot bypasses to get the content. If it fails, it returns a clean error that your agent can handle gracefully using standard Python try-except blocks.

All scraped URLs and Markdown are processed in transit through zero-trust V8 isolates. Vinkius never caches or logs the actual content of your scrapes, meaning your data remains strictly between your Pydantic AI client and the target website.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript