How to Use the Web Scraper MCP in Cursor

Q: Is Web Scraper reliable for grabbing titles and descriptions in Cursor?

Absolutely. Using extract ensures you get clean metadata like title, description, and author information, which is structured data perfect for use in code variables or comments.

Q: Can Web Scraper help me find all links across a project using Cursor?

While it can't scan your local drive, the listlinks tool finds all internal links on a single web page. Your agent can then use that data to inform where you need to write code.

Inject Live Web Data into Your Codebase Using Cursor's AI Agent Mode.

See Vinkius in Action

Works with every AI agent you already use

…and any MCP-compatible client

MCP Servers - Free for Subscribers

Connect Web Scraper MCP to Cursor

Create your Vinkius account to connect Web Scraper to Cursor and route execution through our secure gateway. The platform manages server hosting, runtime updates, and security layers. Configuration requires no manual server provisioning.

GDPR Free for Subscribers

Setup Web Scraper with Cursor

Ask AI about this MCP

ChatGPT

Claude

Perplexity

Get Clean Articles with `read` in Cursor

Writing code that needs external documentation? The `read` tool grabs the full, clean Markdown content from any public web page. Your agent can use this live data to inform function definitions or comments right where you're typing. It’s a huge time saver. Instead of manually copying text into your file for context, you just let the AI call `read` and inject it directly into the code block.

Validate Source Info with MCP Server

Before committing to an API endpoint or a data structure, check the source. The `extract` tool gets structured metadata—things like OG tags, canonical URLs, and authors—from a link. This helps your agent validate if the page is authoritative. This capability lets you write code that accounts for potential source quality issues by checking these key details first.

Map Out Dependencies with `list_links`

If you're building a site generator or an internal tool, you need to know what links are available. The `list_links` tool pulls all internal hyperlinks from a given page into the code context. Your agent can then use this list to generate required boilerplate or file structures. This is better than guesswork; it's actual data pulled directly into your editor environment.

Setup guide

Set up Web Scraper MCP in Cursor

Prerequisites

Cursor installed (macOS, Windows, or Linux)
Active Vinkius subscription with a valid endpoint token

1

Open MCP Settings

Go to Cursor Settings → MCP or open the Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and search for "MCP: Add Server".
2

Add the Web Scraper MCP

Cursor will create or open .cursor/mcp.json in your project root. Paste the JSON snippet on the right. Replace [YOUR_TOKEN_HERE] with your endpoint token from cloud.vinkius.com.
3

Enable Agent mode

Open Composer (Cmd+I / Ctrl+I) and switch to Agent mode using the dropdown at the top. MCP tools are only available in Agent mode.
4

Verify the connection

Ask Cursor something like "List my recent Web Scraper transactions." If the MCP tools are loaded correctly, Cursor will call the Web Scraper tools automatically. You can also check Settings → MCP for a green status indicator.

.cursor/mcp.json

{
  "mcpServers": {
    "web-scraper-mcp": {
      "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    }
  }
}

Get your connection token →

Independent Platform Disclaimer: Vinkius is an independent platform and is not affiliated with, endorsed by, sponsored by, verified by, or otherwise authorized by Web Scraper. All third-party trademarks, logos, and brand names are the property of their respective owners. Their use on this website is strictly for informational purposes to identify service compatibility and interoperability.

Why Choose Vinkius

Vinkius connects your tools to AI with real-time monitoring and automatic cost savings — all from one dashboard.

Connect Web Scraper now

Real-time monitoring

Live

visibility into every interaction

Connect your favorite tools to your AI and see exactly what's happening — every request, every response, in real time.

Built-in savings

60%

lower AI costs

Vinkius compresses data between your apps and your AI automatically. Lower bills every month — no configuration required.

Single dashboard

One

place for every integration

Every tool your AI connects to, managed from a single screen. One account, complete control.

Common questions about Web Scraper MCP in Cursor

When you activate Agent mode, the AI sees this MCP Server as a live tool. If it needs web context, it automatically calls functions like `read` or `extract`, and the output is treated as real data for your code.

Yes, if you need to gather multiple related documentation pages. The `crawl` tool systematically visits a site starting from one URL, providing the content of up to 10 pages for your agent to consume.

Absolutely. Using `extract` ensures you get clean metadata like `title`, `description`, and author information, which is structured data perfect for use in code variables or comments.

While it can't scan your local drive, the `list_links` tool finds all internal links *on* a single web page. Your agent can then use that data to inform where you need to write code.

This server processes public text (Markdown) and structured metadata, including `title`, `description`, and `lang`. It only accesses publicly visible web content.

Use it with your favorite AI tools

Connect this server to Cursor, Claude, VS Code, and more.

OpenAI Agents SDK sdk-python

Google ADK sdk-python

Pydantic AI sdk-python

Vercel AI SDK sdk-typescript