How to Use the HTML DOM Query Engine MCP in CrewAI

Q: How do I add the HTML DOM Query Engine to a CrewAI agent?

First, run pip install crewai. You can then pass the server URL directly into your Agent's mcps list, allowing CrewAI to auto-configure the MCP Server tool for that specific agent.

Q: Does the HTML DOM Query Engine support selective tool exposure in CrewAI?

You can use the MCPServerHTTP class from crewai.mcp to filter which tools are exposed. This allows you to restrict the MCP tool to your dedicated researcher agent while keeping other agents clean.

Equip your CrewAI agent teams with fast, deterministic HTML DOM parsing tools.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect HTML DOM Query Engine MCP to CrewAI

Create your Vinkius account to connect HTML DOM Query Engine to CrewAI and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup HTML DOM Query Engine with CrewAI

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Multi-Agent Scraping Teams in CrewAI

The `query_dom` tool extracts specific text or attributes from raw HTML strings using standard CSS selectors. When deployed in CrewAI, you can assign this tool to a specialized MCP scraping agent. That agent focuses solely on pulling clean data from raw markup, while other agents analyze the results. This division of labor prevents your primary reasoning agents from getting bogged down in raw text processing. Your crew collaborates efficiently by passing clean, structured data between tasks.

Shared Memory DOM Extraction

The `query_dom` tool feeds parsed data directly into CrewAI's shared memory system. Once your scraping agent extracts the target attributes, that information becomes instantly accessible to the entire crew. This ensures that downstream agents always work with the most up-to-date DOM extractions. You do not need to manually pass data variables between agents. The framework manages the state, allowing your team to collaborate on complex web analysis tasks.

Hierarchical Parsing Operations

The `query_dom` tool can be managed by a CrewAI manager agent who delegates parsing tasks to sub-agents. If a complex HTML page has multiple sections, the manager assigns different CSS selectors to individual agents. They run the queries in parallel, speeding up the overall extraction process. This structure is perfect for scraping large directories or e-commerce listings. You get a coordinated effort that handles massive payloads without bottlenecks.

Setup guide

Set up HTML DOM Query Engine MCP in CrewAI

Prerequisites

Python 3.10+ installed
crewai package (pip install crewai)
Active Vinkius subscription with a valid endpoint token

1

Install CrewAI
Run pip install crewai to install the framework. MCP support is built-in via the mcps parameter.
2

Add the MCP URL to your agent
Pass your Vinkius endpoint directly to the mcps list. Replace [YOUR_TOKEN_HERE] with your token from cloud.vinkius.com. CrewAI handles tool discovery and caching automatically.
3

Kick off your crew
Create a Crew with your agent and tasks. Call crew.kickoff() — the agent will automatically invoke HTML DOM Query Engine tools as needed.

crew.py

from crewai import Agent, Task, Crew

agent = Agent(
    role="HTML DOM Query Engine Analyst",
    goal="Access and analyze HTML DOM Query Engine data via MCP.",
    backstory="Expert analyst with direct HTML DOM Query Engine access.",
    mcps=[
        "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    ],
)

task = Task(
    description="List recent HTML DOM Query Engine transactions",
    agent=agent,
    expected_output="A summary of recent activity",
)

crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
print(result)

Get your connection token →

Prerequisites

Python 3.10+ installed
crewai + crewai-tools packages
Active Vinkius subscription with a valid endpoint token

1

Install dependencies
Run pip install crewai crewai-tools. The MCPServerAdapter handles lifecycle management and tool conversion.
2

Connect with MCPServerAdapter
Use MCPServerAdapter as a context manager with SseServerParameters pointing to your Vinkius endpoint. The adapter automatically manages connection lifecycle.
3

Assign tools and run
Pass the returned mcp_tools to your agent's tools parameter. The adapter converts MCP tools to native BaseTool objects compatible with all CrewAI agents.

crew.py

from crewai import Agent, Task, Crew
from crewai_tools import MCPServerAdapter
from mcp import SseServerParameters

server_params = SseServerParameters(
    url="https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
)

with MCPServerAdapter(server_params) as mcp_tools:
    agent = Agent(
        role="HTML DOM Query Engine Analyst",
        goal="Access and analyze HTML DOM Query Engine data via MCP.",
        backstory="Expert analyst with direct HTML DOM Query Engine access.",
        tools=mcp_tools,
    )

    task = Task(
        description="List recent HTML DOM Query Engine transactions",
        agent=agent,
        expected_output="A summary of recent activity",
    )

    crew = Crew(agents=[agent], tasks=[task])
    result = crew.kickoff()
    print(result)

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Cheerio DOM. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect HTML DOM Query Engine now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about HTML DOM Query Engine MCP in CrewAI

First, run `pip install crewai`. You can then pass the server URL directly into your Agent's `mcps` list, allowing CrewAI to auto-configure the MCP Server tool for that specific agent.

Yes, the server scales to handle concurrent requests from multiple agents. When your crew runs parallel tasks, each agent can invoke the `query_dom` tool independently without blocking other operations.

You can use the `MCPServerHTTP` class from `crewai.mcp` to filter which tools are exposed. This allows you to restrict the MCP tool to your dedicated researcher agent while keeping other agents clean.

The tool returns clean JSON containing the matched text content or attributes. This structured output is easy for your agents to parse, preventing formatting errors during downstream tasks.

Processing occurs entirely within a secure V8 Isolate sandbox managed by Vinkius. Your CSS selectors and raw HTML streams never touch a shared database or persistent logs. The execution environment is completely destroyed immediately after the tool returns your data.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript