Can it read documentation sites that are split into multiple pages?

Yes! You can use the `crawl` tool. For example: 'Crawl the getting started guide at https://example.com/docs'. The agent will fetch the starting page and automatically follow inner links to gather up to 10 pages of context.

How does it handle ads and cluttered websites?

The `read` tool uses the same underlying technology as Firefox's 'Reader View' (@mozilla/readability). It intelligently strips out standard website boilerplate—like navbars, sidebars, footers, and ads—leaving only the title and the clean main article text converted to Markdown.

Is there a limit on how many URLs I can batch process?

Yes, to ensure conversational AI latency remains reasonable, the `batch_read` tool accepts a maximum of 10 URLs in a single request. All 10 URLs are fetched simultaneously in parallel for maximum speed.

Web Scraper MCP Server for VS Code Copilot 5 tools — connect in under 2 minutes

Name: Web Scraper
Availability: InStock
Author: Vinkius

code.visualstudio.com/docs/copilot/chat/mcp-servers

Built by Vinkius GDPR 5 Tools IDE

GitHub Copilot in VS Code is the most widely adopted AI coding assistant, embedded directly into the world's most popular code editor. With MCP support in Agent mode, Copilot can access external data and APIs to generate context-aware code grounded in real-time information.

Get MCP Server for AI Agents

ASK AI ABOUT THIS MCP SERVER

Open in ChatGPT Open in Claude Open in Perplexity

Vinkius supports streamable HTTP and SSE.

RecommendedModern Approach — Zero Configuration

Vinkius Desktop App

The modern way to manage MCP Servers — no config files, no terminal commands. Install Web Scraper and 2,500+ MCP Servers from a single visual interface.

Download Free Open SourceNo signup required

Classic Setup·json

{
  "mcpServers": {
    "web-scraper": {
      "url": "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp"
    }
  }
}

Fully ManagedVinkius Servers

60%Token savings

High SecurityEnterprise-grade

IAMAccess control

EU AI ActCompliant

DLPData protection

V8 IsolateSandboxed

Ed25519Audit chain

<40msKill switch

Stream every event to Splunk, Datadog, or your own webhook in real-time

* Every MCP server runs on Vinkius-managed infrastructure inside AWS - a purpose-built runtime with per-request V8 isolates, Ed25519 signed audit chains, and sub-40ms cold starts optimized for native MCP execution. See our infrastructure

About Web Scraper MCP Server

Connect the Web Scraper utility to any AI agent to give it direct access to the public internet. Instead of letting the AI hallucinate facts, allow it to read real-time articles, parse documentation, and fetch clean text from any URL you provide.

GitHub Copilot Agent mode brings Web Scraper data directly into your VS Code workflow. With a project-scoped config, the entire team shares access to 5 tools. Copilot queries live data, generates typed code, and writes tests from actual API responses, all without leaving the editor.

What you can do

Reader View — Convert any cluttered webpage into pristine, readable Markdown by stripping out ads, navbars, and boilerplate using Mozilla Readability logic
Site Crawling — Instruct the AI to crawl a starting URL (like a documentation hub or wiki) up to 10 pages deep automatically
Batch Processing — Fetch up to 10 different URLs in parallel to compare articles or summarize multiple sources at once
Metadata Extraction — Quickly pull SEO titles, descriptions, OG tags, canonical links, and all outbound hyperlinks without downloading the entire page body

The Web Scraper MCP Server exposes 5 tools through the Vinkius. Connect it to VS Code Copilot in under two minutes — no API keys to rotate, no infrastructure to provision, no vendor lock-in. Your configuration, your data, your control.

How to Connect Web Scraper to VS Code Copilot via MCP

Follow these steps to integrate the Web Scraper MCP Server with VS Code Copilot.

Create MCP config

Create a .vscode/mcp.json file in your project root

Add the server config

Paste the JSON configuration above

Enable Agent mode

Open GitHub Copilot Chat and switch to Agent mode using the dropdown

Start using Web Scraper

Ask Copilot: "Using Web Scraper, help me...". 5 tools available

Why Use VS Code Copilot with the Web Scraper MCP Server

GitHub Copilot for Visual Studio Code provides unique advantages when paired with Web Scraper through the Model Context Protocol.

VS Code is used by over 70% of developers. adding MCP tools to Copilot means your team can leverage external data without leaving their primary editor

Project-scoped MCP configs (`.vscode/mcp.json`) let you commit server configurations to your repository, ensuring the entire team shares the same tool access

Copilot's Agent mode integrates MCP tools seamlessly with file editing, terminal commands, and workspace search in a single agentic loop

GitHub's enterprise compliance and audit features extend to MCP tool usage, providing visibility into how AI interacts with external services

Web Scraper + VS Code Copilot Use Cases

Practical scenarios where VS Code Copilot combined with the Web Scraper MCP Server delivers measurable value.

Live API integration: Copilot can query an MCP server, inspect the response schema, and generate typed API client code in the same step

DevSecOps workflows: security teams can give developers access to domain intelligence tools directly in their editor for real-time vulnerability assessment during code review

Data pipeline development: Copilot fetches sample data via MCP and generates transformation scripts, validators, and test fixtures from actual API responses

Documentation generation: Copilot queries available tools and auto-generates README sections, API reference docs, and usage examples

Web Scraper MCP Tools for VS Code Copilot (5)

These 5 tools become available when you connect Web Scraper to VS Code Copilot via MCP:

batch_read

All URLs are fetched in parallel. Maximum 10 URLs per batch. Fetch multiple web pages in parallel

crawl

Maximum 10 pages to keep response size manageable. Crawl a website starting from a URL

extract

Returns: title, description, OG tags, lang, author, robots, canonical, link count. For the full page content, use the read tool instead. Extract structured metadata from a web page: title, description, OG tags, and more

list_links

Internal links share the same hostname as the source page. Extract all hyperlinks from a web page

read

Uses @mozilla/readability (Firefox Reader View) to extract the main article content, then converts to Markdown. Works best for articles, docs, blogs, and Wikipedia. Fetch any public web page and return its full content as clean Markdown

Example Prompts for Web Scraper in VS Code Copilot

Ready-to-use prompts you can give your VS Code Copilot agent to start working with Web Scraper immediately.

"Read https://en.wikipedia.org/wiki/Artificial_intelligence and summarize its history."

"Extract the links from https://news.ycombinator.com/"

"Compare these two links: url1.com and url2.com"

Troubleshooting Web Scraper MCP Server with VS Code Copilot

Common issues when connecting Web Scraper to VS Code Copilot through the Vinkius, and how to resolve them.

MCP tools not available

Ensure you are in Agent mode in Copilot Chat. MCP tools only appear in Agent mode.

Web Scraper + VS Code Copilot FAQ

Common questions about integrating Web Scraper MCP Server with VS Code Copilot.

Which VS Code version supports MCP?

MCP support requires VS Code 1.99 or later with the GitHub Copilot extension. Ensure both are updated to the latest version. Older versions of Copilot may not expose the Agent mode toggle.

How do I switch to Agent mode?

Open the Copilot Chat panel and look for two mode options: "Ask" and "Agent". Click "Agent" to enable autonomous tool calling. In Ask mode, Copilot provides conversational answers but cannot invoke MCP tools.

Can I restrict which MCP tools Copilot can access?

Yes. VS Code shows a tool consent dialog before any MCP tool is invoked for the first time. You can also configure tool access policies at the organization level through GitHub Copilot settings.

Does MCP work in VS Code Remote or Codespaces?

Yes. MCP servers configured via .vscode/mcp.json work in Remote SSH, WSL, and GitHub Codespaces environments. The MCP connection is established from the remote host, so ensure the server URL is accessible from that environment.