How to Use the Crawlbase MCP in Pydantic AI

Q: How does Pydantic AI ensure the data from Crawlbase is correct?

Pydantic AI doesn't check if the data is factually correct, it checks if it's structurally correct. You define a Pydantic model, and if the JSON from a tool like scrapejsonformat doesn't match your model's schema, it raises a validation error.

Q: What's the process for adding the Crawlbase tools to my Pydantic AI agent?

Install with pip install "pydantic-ai-slimmcp". Then, just create an MCPToolset instance with your Vinkius server URL and pass it into the toolsets list when you initialize your agent. The tools are discovered from there.

Get web data you can trust. This Crawlbase server gives your Pydantic AI agent type-safe, validated results.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Crawlbase MCP to Pydantic AI

Create your Vinkius account to connect Crawlbase to Pydantic AI and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Crawlbase with Pydantic AI

Ask AI about this MCP

ChatGPT

Claude

Perplexity

No More Silent Data Corruption

The `scrape_json_format` tool is designed for Pydantic AI. It performs structural extraction based on the fields you need. Your agent gets back a JSON object, and Pydantic AI immediately validates it against your model. If a website changes its structure or the scraper returns an unexpected field, your agent will raise a `ValidationError` instantly. This is the whole point. You build systems that stop and alert you to a problem, instead of polluting your database with bad data.

Build Data Pipelines That Don't Break

Use tools like `scrape_amazon` and `scrape_linkedin` to pull structured data into your Pydantic models. You define the exact schema you expect for a product or a profile. The agent makes the call, Crawlbase gets the data, and Pydantic AI verifies it. This approach makes your agent model-agnostic. Whether you're using GPT-4, Claude, or a local Llama model, the data validation step is consistent. The Crawlbase MCP Server provides the data, and Pydantic AI enforces the contract.

Validate Every MCP Server Response

Even for simple tools like `scrape_html`, you can add a layer of safety. Define a Pydantic model that expects a string of HTML. If the tool were to return an error object or null by mistake, your agent would fail loudly instead of processing invalid input. The same goes for `get_screenshot_link`. You can create a model that expects a response containing a valid URL. It's a simple but effective way to ensure every tool your Pydantic AI agent uses is behaving exactly as you expect, every single time.

Setup guide

Set up Crawlbase MCP in Pydantic AI

Prerequisites

Python 3.10+ installed
pydantic-ai-slim[fastmcp] package
Active Vinkius subscription with a valid endpoint token

1

Install Pydantic AI with FastMCP
Run pip install "pydantic-ai-slim[fastmcp]". The FastMCP toolset replaces the deprecated MCPServerHTTP class with full protocol support.
2

Configure the FastMCPToolset
Pass a JSON-style config dict to FastMCPToolset with your Vinkius URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. Supports Streamable HTTP, SSE, and Stdio transports.
3

Create and run your agent
Pass the toolset to Agent(toolsets=[toolset]) and call agent.run(). Swap openai:gpt-4o for any supported model — Anthropic, Google, Mistral, or Groq.

agent.py

from pydantic_ai import Agent
from pydantic_ai.toolsets.fastmcp import FastMCPToolset

toolset = FastMCPToolset({
    "mcpServers": {
        "crawlbase-mcp": {
            "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
        }
    }
})

agent = Agent(
    "openai:gpt-4o",
    toolsets=[toolset],
    system_prompt="You have access to Crawlbase tools.",
)

result = await agent.run("List recent Crawlbase transactions")
print(result.output)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Crawlbase. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Crawlbase now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Crawlbase MCP in Pydantic AI

Pydantic AI doesn't check if the data is *factually* correct, it checks if it's *structurally* correct. You define a Pydantic model, and if the JSON from a tool like `scrape_json_format` doesn't match your model's schema, it raises a validation error.

Yes. Pydantic AI is model-agnostic, and the Crawlbase server is just an HTTP endpoint. As long as your agent can make a web request, it can use these tools, regardless of which LLM is powering its reasoning.

Install with `pip install "pydantic-ai-slim[mcp]"`. Then, just create an `MCPToolset` instance with your Vinkius server URL and pass it into the `toolsets` list when you initialize your agent. The tools are discovered from there.

Crawlbase will attempt to scrape the page. If the change breaks the expected structure, the returned JSON might be different. Pydantic AI will then see the mismatch against your model and raise an error, preventing the malformed data from being used.

The Crawlbase server is passed the URL to be scraped and nothing else. It retrieves the page's raw HTML or JSON content. Each request is handled in a dedicated, ephemeral Vinkius sandbox that's destroyed afterward, and authentication is managed by a separate token system, not the server itself.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript