Web Scraper Connector for AI agents.
7 live capabilities
Fetch live web content and crawl documentation for your agent.
Waiting for input…
Why people use Web Scraper
Web Scraper for documentation and content extraction
This Connector changes the game by letting your agent do the heavy lifting. You just drop a link, and the capability pulls the core content into a clean Markdown format. You get the actual facts without the noise of ads or navigation bars.
What Vinkius changes
Your agent gets real-time web access without you having to copy-paste a single paragraph.
Use it from Claude, ChatGPT, Cursor or another AI client you already have.
One account · 5,900+ Connectors
- Real-world use case 01
Learning a new library
A developer points the agent to a new library's docs and uses crawl to let it learn the full API structure before writing code.
- Real-world use case 02
Synthesizing research
A researcher provides 5 Wikipedia links and uses batch_read to get a synthesized summary of all of them at once.
- Real-world use case 03
SEO link auditing
An SEO specialist uses extract to grab meta tags and list_links to see where a competitor's site points.
Complete set · 7capabilities
The complete Web Scraper capability set.
These are the exact actions your AI can choose when you ask it to work with Web Scraper.
01—04
4 capabilities in this set.
Part of 7 available through Web Scraper.
- 01 Capability
Batch read
Pulls up to 10 different URLs in parallel. Use this to compare multiple articles or summarize several sources at once.
- 02 Capability
Read
Fetches a public web page and returns its full content as clean Markdown using Mozilla Readability logic. This is perfect for getting the core text from blogs or Wikipedia without the junk.
- 03 Capability
Extract
Pulls structured metadata like titles, OG tags, and descriptions from a page. It's the fastest way to audit a site's SEO without downloading the full body.
- 04 Capability
List links
Grabs every hyperlink from a webpage. Use this to map out a site's structure or audit outbound links.
05—07
3 capabilities in this set.
Part of 7 available through Web Scraper.
- 05 Capability
Read json
Fetches and parses JSON data from a URL. This lets your agent interact with live API responses directly.
- 06 Capability
Extract jsonld
Pulls JSON-LD blocks from a webpage. It's great for getting structured schema data from the web.
- 07 Capability
Crawl
Navigates a website starting from a specific URL. It goes up to 10 pages deep to gather a broader picture of a documentation hub.
Set up in minutes
One URL. Then ask Web Scraper to work.
Claude and ChatGPT only need the Connector URL. Copy it once, add it in settings, and use Web Scraper from the conversation.
Choose your client
Live previewAdvanced clients IDE · CLI
Claude · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp - Step 01
Open Connectors
In Claude Web or Claude Desktop, open Settings and choose Connectors.
- Step 02
Add the URL
Choose Add custom connector, name it Web Scraper, and paste the URL above.
- Step 03
Turn it on in chat
Select +, open Connectors, and enable Web Scraper for the conversation.
ChatGPT · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp - Step 01
Open MCP settings
On desktop, open Settings and MCP servers. On web, open your workspace app or connector settings.
- Step 02
Add the URL
Choose Add server with Streamable HTTP, or create a custom MCP app, then paste the Web Scraper URL.
- Step 03
Save and start
Save the connection and enable Web Scraper in your conversation. Desktop may ask you to restart once.
Cursor · IDE configuration
Advanced setup
{
"mcpServers": {
"web-scraper": {
"url": "https://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp"
}
}
}- Step 01
Open MCP Settings
Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows/Linux) → search "MCP Settings"
- Step 02
Add the server config
Paste the JSON configuration above into the mcp.json file that opens
- Step 03
Save the file
Cursor will automatically detect the new Connector
- Step 04
Start using Web Scraper
Open Agent mode in chat and ask: "Using Web Scraper, help me...". 7 tools available
VS Code Copilot · IDE configuration
Advanced setup
{
"mcpServers": {
"web-scraper": {
"url": "https://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp"
}
}
}- Step 01
Create MCP config
Create a .vscode/mcp.json file in your project root
- Step 02
Add the server config
Paste the JSON configuration above
- Step 03
Enable Agent mode
Open GitHub Copilot Chat and switch to Agent mode using the dropdown
- Step 04
Start using Web Scraper
Ask Copilot: "Using Web Scraper, help me...". 7 tools available
Windsurf · IDE configuration
Advanced setup
{
"mcpServers": {
"web-scraper": {
"url": "https://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp"
}
}
}- Step 01
Open MCP Settings
Go to Settings → MCP Configuration or press Cmd+Shift+P and search "MCP"
- Step 02
Add the server
Paste the JSON configuration above into mcp_config.json
- Step 03
Save and reload
Windsurf will detect the new server automatically
- Step 04
Start using Web Scraper
Open Cascade and ask: "Using Web Scraper, help me...". 7 tools available
Cline · IDE configuration
Advanced setup
{
"mcpServers": {
"web-scraper": {
"url": "https://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp"
}
}
}- Step 01
Open Cline MCP Settings
Click the Connectors icon in the Cline sidebar panel
- Step 02
Add remote server
Click "Add Connector" and paste the configuration above
- Step 03
Enable the server
Toggle the server switch to ON
- Step 04
Start using Web Scraper
Ask Cline: "Using Web Scraper, help me...". 7 tools available
Claude Code · Terminal command
Advanced setup
claude mcp add web-scraper --transport http "https://edge.vinkius.com/vk_preview_YmTaAx3VdeRzHJqckaOgMTVnBzECnxTL8z7QeMgb/mcp"- Step 01
Install Claude Code
Run npm install -g @anthropic-ai/claude-code if not already installed
- Step 02
Add the Connector
Run the command above in your terminal
- Step 03
Verify the connection
Run claude mcp to list connected servers, or type /mcp inside a session
- Step 04
Start using Web Scraper
Ask Claude: "Using Web Scraper, show me...". 7 tools are ready
Where the request belongs
Work Web Scraper can move forward.
This is for developers who need the latest API specs, researchers hunting for specific data points across multiple sites, and SEOs who need to audit link structures on the fly.
Software Engineer
Reads new library documentation to write accurate code using the latest syntax.
Content Researcher
Summarizes multiple Wikipedia pages into a single report to synthesize a topic.
SEO Analyst
Audits a competitor's metadata and outbound links to check search visibility.
Build the capability set
Add more capabilities.
Each Connector adds new actions and data without changing how you work.
Browse ConnectorsFirecrawl
Turn any website into clean, LLM-ready Markdown with a single API call. scrape, crawl, search, and map the entire web for your AI agent.
Browse AI
Extract data from any website without code using trained robots that monitor pages and deliver structured results automatically.
Spider
High-performance Rust-powered web scraping and crawling. scrape, crawl, and search up to 100K+ pages/second with built-in anti-bot protection.
ScrapingAnt
Extract web data reliably with rotating proxies, headless Chrome rendering, and CAPTCHA solving built into every request.
Bring your own AI
Change the model, client or framework. Keep Web Scraper connected.
-
Claude -
ChatGPT -
Gemini -
Cursor -
VS Code -
Windsurf -
ZCode -
Cline -
Zed -
Continue -
Kiro -
Roo Code -
Zencoder -
Goose -
Void -
Augment Code -
Amp -
Qodo -
Tabnine -
Pieces -
Sourcegraph Cody -
JetBrains -
Warp -
Amazon Q -
Antigravity -
BoltAI -
Raycast -
Jan -
LM Studio -
AnythingLLM -
Open WebUI -
Msty -
Cherry Studio -
LibreChat -
TypingMind -
Chorus -
5ire -
n8n -
LangChain -
LlamaIndex -
CrewAI -
Vercel AI SDK
Before you connect
Questions about Web Scraper.
The practical details behind the request, access and result.
Can the Web Scraper MCP read paywalled content?
No, this capability only accesses publicly available web pages. It cannot bypass logins, paywalls, or subscription screens.
How many pages can the Web Scraper MCP crawl at once?
The crawl capability allows your agent to navigate up to 10 pages deep from a starting URL, which is usually enough to capture a full documentation set.
Does the Web Scraper MCP require an API key?
No, there are no API keys or authentication requirements needed to use this Connector.
Can I use the Web Scraper MCP to get clean text from a blog?
Yes, it uses Mozilla Readability logic to strip out ads, navbars, and boilerplate, giving your agent just the core article content.
Can the Web Scraper MCP handle JSON data?
Yes, the read_json capability allows your agent to fetch and parse structured JSON data from any URL that returns it.
How does the Web Scraper MCP handle messy websites?
It automatically cleans the content by converting it to Markdown and removing common web clutter like sidebars and navigation menus.
Can it read documentation sites that are split into multiple pages?
Yes! You can use the crawl capability. For example: 'Crawl the getting started guide at https://example.com/docs'. The agent will fetch the starting page and automatically follow inner links to gather up to 10 pages of context.
How does it handle ads and cluttered websites?
The read capability uses the same underlying technology as Firefox's 'Reader View' (@mozilla/readability). It intelligently strips out standard website boilerplate—like navbars, sidebars, footers, and ads—leaving only the title and the clean main article text converted to Markdown.
Is there a limit on how many URLs I can batch process?
Yes, to ensure conversational AI latency remains reasonable, the batch_read capability accepts a maximum of 10 URLs in a single request. All 10 URLs are fetched simultaneously in parallel for maximum speed.
One connection away
Give your agent a direct line to Web Scraper.
Connect Web Scraper once. Keep it beside 5,900+ managed Connectors when the next task needs more.
Explore every Connector No credit card required · Free tier available