How to Use the Diffbot MCP in CrewAI

Q: Can Diffbot help CrewAI agents research?

It is the perfect fit. Your research agents can use extractarticle to gather information from hundreds of sources for the rest of the crew.

Q: How do I filter tools for a specific CrewAI agent?

Use the toolfilter option in your agent setup. You can limit each CrewAI agent to only the tools they need to perform their specific role.

Equip your CrewAI agents with web superpowers. Extract structured data from any URL and let your team of agents collaborate on it.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Diffbot MCP to CrewAI

Create your Vinkius account to connect Diffbot to CrewAI and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Diffbot with CrewAI

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Web research for CrewAI teams

Assign an agent to use `analyze_page` for gathering context. The tool automatically maps site layout to JSON, giving your CrewAI team a clean starting point. Your research agent finds the data, and the analyst agent processes it. No manual parsing required.

Targeted data extraction

Give your agents `extract_video` or `extract_image` to pull specific media metadata. This allows your CrewAI team to work with visual content as easily as text. Specialized agents use these tools to build deep insights. The structure is consistent, so your agents don't get confused by layout differences.

Autonomous data gathering

Let your CrewAI crew operate without human input. By using the server, your agents can crawl and extract data autonomously based on their specific roles. This creates a self-sustaining loop. The agents identify the need, call the tools, and complete the task.

Setup guide

Set up Diffbot MCP in CrewAI

Prerequisites

Python 3.10+ installed
crewai package (pip install crewai)
Active Vinkius subscription with a valid endpoint token

1

Install CrewAI
Run pip install crewai to install the framework. MCP support is built-in via the mcps parameter.
2

Add the MCP URL to your agent
Pass your Vinkius endpoint directly to the mcps list. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. CrewAI handles tool discovery and caching automatically.
3

Kick off your crew
Create a Crew with your agent and tasks. Call crew.kickoff() — the agent will automatically invoke Diffbot tools as needed.

crew.py

from crewai import Agent, Task, Crew

agent = Agent(
    role="Diffbot Analyst",
    goal="Access and analyze Diffbot data via MCP.",
    backstory="Expert analyst with direct Diffbot access.",
    mcps=[
        "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    ],
)

task = Task(
    description="List recent Diffbot transactions",
    agent=agent,
    expected_output="A summary of recent activity",
)

crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
print(result)

Get your connection token →

Prerequisites

Python 3.10+ installed
crewai + crewai-tools packages
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Run pip install crewai crewai-tools. The MCPServerAdapter handles lifecycle management and tool conversion.
2

Connect with MCPServerAdapter
Use MCPServerAdapter as a context manager with SseServerParameters pointing to your Vinkius endpoint. The adapter automatically manages connection lifecycle.
3

Assign tools and run
Pass the returned mcp_tools to your agent's tools parameter. The adapter converts MCP tools to native BaseTool objects compatible with all CrewAI agents.

crew.py

from crewai import Agent, Task, Crew
from crewai_tools import MCPServerAdapter
from mcp import SseServerParameters

server_params = SseServerParameters(
    url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)

with MCPServerAdapter(server_params) as mcp_tools:
    agent = Agent(
        role="Diffbot Analyst",
        goal="Access and analyze Diffbot data via MCP.",
        backstory="Expert analyst with direct Diffbot access.",
        tools=mcp_tools,
    )

    task = Task(
        description="List recent Diffbot transactions",
        agent=agent,
        expected_output="A summary of recent activity",
    )

    crew = Crew(agents=[agent], tasks=[task])
    result = crew.kickoff()
    print(result)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Diffbot. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Diffbot now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Diffbot MCP in CrewAI

Simply include the endpoint in your agent's MCP configuration. Your CrewAI agents will immediately see the full suite of extraction tools.

It is the perfect fit. Your research agents can use `extract_article` to gather information from hundreds of sources for the rest of the crew.

Yes. Every agent in your CrewAI hierarchy can call these tools, allowing for specialized extraction at every level of your team.

Use the tool_filter option in your agent setup. You can limit each CrewAI agent to only the tools they need to perform their specific role.

We use a zero-trust architecture for all requests. Your CrewAI agents only receive the final structured data, keeping the underlying infrastructure isolated.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript