How to Use the WebScrapingAPI MCP in Pydantic AI

Q: How do I search for information using WebScrapingAPI with Pydantic AI?

Use searchgoogleserp to get structured results. The resulting data from the SERP is passed through your agent and immediately validated by a Pydantic model, guaranteeing predictable Python objects.

Q: Can I use WebScrapingAPI for complex e-commerce data with Pydantic AI?

Yes. The scrapeecommerceproduct tool provides the raw JSON, and your agent enforces that this data conforms to a strict schema before proceeding. This prevents silent corruption.

Q: Does WebScrapingAPI support scraping multiple search engines with Pydantic AI?

Absolutely. You can use searchbingserp, searchgoogleserp, and searchyandexserp. The benefit here is that the data from every source, regardless of origin, adheres to your defined Python types.

Q: What happens if I need to scrape content that requires a mobile view using WebScrapingAPI with Pydantic AI?

Use scrapeasmobile. It emulates the device and captures the data, which your agent then validates. This means you get both the right source and the guaranteed correct type.

WebScrapingAPI with Pydantic AI: Guarantee accurate, typed data from any scraped source.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect WebScrapingAPI MCP to Pydantic AI

Create your Vinkius account to connect WebScrapingAPI to Pydantic AI and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup WebScrapingAPI with Pydantic AI

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Extract structured product details using MCP Server

The `scrape_ecommerce_product` tool returns price, title, and reviews as clean JSON. When your agent uses this via Pydantic AI, every piece of data is validated against a strict schema at runtime. You won't get unexpected fields. This guarantees data correctness before the model even sees it. If the API messes up, your agent fails loudly—you know exactly what went wrong.

Capture full page content with Pydantic AI

For sites that rely on JavaScript rendering, use `scrape_js_rendered`. This slower method captures the full state of a site, and then Pydantic validates the resulting HTML structure against your defined model. It's reliable. If you just need raw source code for speed, `scrape_static_html` pulls out the basic HTML, which is still typed and validated by your agent.

Execute advanced scraping tasks with MCP Server

The `custom_api_scrape` tool lets you run scrapes using custom parameters for country, session management, and wait times. Your Pydantic AI setup ensures that even these complex scraped operations yield data matching your expected Python types. It's about making the output reliable, no matter how complicated the input site is.

Setup guide

Set up WebScrapingAPI MCP in Pydantic AI

Prerequisites

Python 3.10+ installed
pydantic-ai-slim[fastmcp] package
Active Vinkius subscription with a valid endpoint token

1

Install Pydantic AI with FastMCP
Run pip install "pydantic-ai-slim[fastmcp]". The FastMCP toolset replaces the deprecated MCPServerHTTP class with full protocol support.
2

Configure the FastMCPToolset
Pass a JSON-style config dict to FastMCPToolset with your Vinkius URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. Supports Streamable HTTP, SSE, and Stdio transports.
3

Create and run your agent
Pass the toolset to Agent(toolsets=[toolset]) and call agent.run(). Swap openai:gpt-4o for any supported model — Anthropic, Google, Mistral, or Groq.

agent.py

from pydantic_ai import Agent
from pydantic_ai.toolsets.fastmcp import FastMCPToolset

toolset = FastMCPToolset({
    "mcpServers": {
        "webscrapingapi-mcp": {
            "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
        }
    }
})

agent = Agent(
    "openai:gpt-4o",
    toolsets=[toolset],
    system_prompt="You have access to WebScrapingAPI tools.",
)

result = await agent.run("List recent WebScrapingAPI transactions")
print(result.output)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by WebScrapingAPI. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect WebScrapingAPI now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about WebScrapingAPI MCP in Pydantic AI

Use `search_google_serp` to get structured results. The resulting data from the SERP is passed through your agent and immediately validated by a Pydantic model, guaranteeing predictable Python objects.

Yes. The `scrape_ecommerce_product` tool provides the raw JSON, and your agent enforces that this data conforms to a strict schema before proceeding. This prevents silent corruption.

Absolutely. You can use `search_bing_serp`, `search_google_serp`, and `search_yandex_serp`. The benefit here is that the data from every source, regardless of origin, adheres to your defined Python types.

Use `scrape_as_mobile`. It emulates the device and captures the data, which your agent then validates. This means you get both the right source *and* the guaranteed correct type.

The server handles publicly visible content like raw HTML, structured search engine results (SERP), and product metadata. Pydantic AI ensures that even if the source is sensitive, the structure of what your agent processes remains predictable.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript