How to Use the WebScrapingAPI MCP in OpenAI Agents SDK

Q: How do I search for information using WebScrapingAPI with OpenAI Agents SDK?

Use searchgoogleserp to get structured results from Google. You just feed it a query string, and your agent gets the necessary data immediately. It's far better than scraping basic search result HTML.

Q: Can I use WebScrapingAPI for complex e-commerce data with OpenAI Agents SDK?

Absolutely. The scrapeecommerceproduct tool is built to return structured JSON containing price, title, and reviews from major stores. Your agent can ingest this clean data right away.

Q: Does WebScrapingAPI support scraping multiple search engines with OpenAI Agents SDK?

Yes. You've got searchgoogleserp, searchbingserp, and searchyandexserp. Your agent can pick the right one based on where you need the data.

Q: What happens if I need to scrape content that requires a mobile view using WebScrapingAPI with OpenAI Agents SDK?

Just use scrapeasmobile. This tool emulates scraping from a mobile device, which is critical because many sites show different layouts or data based on the user agent.

WebScrapingAPI with OpenAI Agents SDK: Get structured data from any webpage.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect WebScrapingAPI MCP to OpenAI Agents SDK

Create your Vinkius account to connect WebScrapingAPI to OpenAI Agents SDK and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup WebScrapingAPI with OpenAI Agents SDK

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Extract structured product details using MCP Server

The `scrape_ecommerce_product` tool gets you price, title, and reviews in clean JSON. You pass it a URL, and your agent spits out exactly what you need for Amazon or Walmart listings. Don't waste time writing custom parsers. Just let the MCP Server handle the extraction so your OpenAI Agents SDK can move on to the next task.

Execute advanced scraping tasks with OpenAPI Agents SDK

Need specific parameters? The `custom_api_scrape` tool lets you run scrapes using custom settings for country, session management, and wait times. This gives your agent precise control over complex web interactions. You can use this to hit niche sites that require particular headers or geographical constraints. It's way more flexible than standard scraping tools.

Capture full page content with MCP Server

Sometimes the data is locked behind JavaScript execution. The `scrape_js_rendered` tool runs a headless browser to capture the entire rendered state of a site, which is slower but necessary for accurate reads. This ensures your agent sees what a real user's browser does. If you just need raw source code, `scrape_static_html` passes the full URL and pulls out the basic HTML structure without rendering overhead.

Setup guide

Set up WebScrapingAPI MCP in OpenAI Agents SDK

Prerequisites

Python 3.10+ installed
openai-agents package (pip install openai-agents)
Active Vinkius subscription with a valid endpoint token

1

Install the SDK

Run pip install openai-agents to install the OpenAI Agents SDK. The MCP integration is built-in — no extra dependencies needed.
2

Connect via SSE transport

Use MCPServerSse with your Vinkius endpoint URL. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. The SDK auto-discovers all WebScrapingAPI tools at runtime.
3

Create your Agent

Pass the MCP to Agent(mcp_servers=[server]). The agent receives WebScrapingAPI tools as native definitions — JSON schemas resolve automatically.
4

Run the agent

Call Runner.run(agent, prompt) to execute. The agent invokes the appropriate WebScrapingAPI tools and returns structured results. Copy the full example on the right to get started.

agent.py

import asyncio
from agents import Agent, Runner
from agents.mcp import MCPServerSse

async def main():
    async with MCPServerSse(
        url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    ) as server:
        agent = Agent(
            name="WebScrapingAPI Agent",
            instructions="You have access to WebScrapingAPI tools.",
            mcp_servers=[server],
        )
        result = await Runner.run(agent, "List recent transactions")
        print(result.final_output)

asyncio.run(main())

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by WebScrapingAPI. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect WebScrapingAPI now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about WebScrapingAPI MCP in OpenAI Agents SDK

Use `search_google_serp` to get structured results from Google. You just feed it a query string, and your agent gets the necessary data immediately. It's far better than scraping basic search result HTML.

Absolutely. The `scrape_ecommerce_product` tool is built to return structured JSON containing price, title, and reviews from major stores. Your agent can ingest this clean data right away.

Yes. You've got `search_google_serp`, `search_bing_serp`, and `search_yandex_serp`. Your agent can pick the right one based on where you need the data.

Just use `scrape_as_mobile`. This tool emulates scraping from a mobile device, which is critical because many sites show different layouts or data based on the user agent.

The server handles publicly accessible web content in the form of raw HTML, structured search engine results (SERP), and product metadata. It doesn't access private user accounts or credentials.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript