Web Scraper MCP Server for Vercel AI SDK 5 tools — connect in under 2 minutes
The Vercel AI SDK is the TypeScript toolkit for building AI-powered applications. Connect Web Scraper through Vinkius and every tool is available as a typed function. ready for React Server Components, API routes, or any Node.js backend.
ASK AI ABOUT THIS MCP SERVER
Vinkius supports streamable HTTP and SSE.
import { createMCPClient } from "@ai-sdk/mcp";
import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
async function main() {
const mcpClient = await createMCPClient({
transport: {
type: "http",
// Your Vinkius token. get it at cloud.vinkius.com
url: "https://edge.vinkius.com/[YOUR_TOKEN_HERE]/mcp",
},
});
try {
const tools = await mcpClient.tools();
const { text } = await generateText({
model: openai("gpt-4o"),
tools,
prompt: "Using Web Scraper, list all available capabilities.",
});
console.log(text);
} finally {
await mcpClient.close();
}
}
main();
* Every MCP server runs on Vinkius-managed infrastructure inside AWS - a purpose-built runtime with per-request V8 isolates, Ed25519 signed audit chains, and sub-40ms cold starts optimized for native MCP execution. See our infrastructure
About Web Scraper MCP Server
Connect the Web Scraper utility to any AI agent to give it direct access to the public internet. Instead of letting the AI hallucinate facts, allow it to read real-time articles, parse documentation, and fetch clean text from any URL you provide.
The Vercel AI SDK gives every Web Scraper tool full TypeScript type inference, IDE autocomplete, and compile-time error checking. Connect 5 tools through Vinkius and stream results progressively to React, Svelte, or Vue components. works on Edge Functions, Cloudflare Workers, and any Node.js runtime.
What you can do
- Reader View — Convert any cluttered webpage into pristine, readable Markdown by stripping out ads, navbars, and boilerplate using Mozilla Readability logic
- Site Crawling — Instruct the AI to crawl a starting URL (like a documentation hub or wiki) up to 10 pages deep automatically
- Batch Processing — Fetch up to 10 different URLs in parallel to compare articles or summarize multiple sources at once
- Metadata Extraction — Quickly pull SEO titles, descriptions, OG tags, canonical links, and all outbound hyperlinks without downloading the entire page body
The Web Scraper MCP Server exposes 5 tools through the Vinkius. Connect it to Vercel AI SDK in under two minutes — no API keys to rotate, no infrastructure to provision, no vendor lock-in. Your configuration, your data, your control.
How to Connect Web Scraper to Vercel AI SDK via MCP
Follow these steps to integrate the Web Scraper MCP Server with Vercel AI SDK.
Install dependencies
Run npm install @ai-sdk/mcp ai @ai-sdk/openai
Replace the token
Replace [YOUR_TOKEN_HERE] with your Vinkius token
Run the script
Save to agent.ts and run with npx tsx agent.ts
Explore tools
The SDK discovers 5 tools from Web Scraper and passes them to the LLM
Why Use Vercel AI SDK with the Web Scraper MCP Server
Vercel AI SDK provides unique advantages when paired with Web Scraper through the Model Context Protocol.
TypeScript-first: every MCP tool gets full type inference, IDE autocomplete, and compile-time error checking out of the box
Framework-agnostic core works with Next.js, Nuxt, SvelteKit, or any Node.js runtime. same Web Scraper integration everywhere
Built-in streaming UI primitives let you display Web Scraper tool results progressively in React, Svelte, or Vue components
Edge-compatible: the AI SDK runs on Vercel Edge Functions, Cloudflare Workers, and other edge runtimes for minimal latency
Web Scraper + Vercel AI SDK Use Cases
Practical scenarios where Vercel AI SDK combined with the Web Scraper MCP Server delivers measurable value.
AI-powered web apps: build dashboards that query Web Scraper in real-time and stream results to the UI with zero loading states
API backends: create serverless endpoints that orchestrate Web Scraper tools and return structured JSON responses to any frontend
Chatbots with tool use: embed Web Scraper capabilities into conversational interfaces with streaming responses and tool call visibility
Internal tools: build admin panels where team members interact with Web Scraper through natural language queries
Web Scraper MCP Tools for Vercel AI SDK (5)
These 5 tools become available when you connect Web Scraper to Vercel AI SDK via MCP:
batch_read
All URLs are fetched in parallel. Maximum 10 URLs per batch. Fetch multiple web pages in parallel
crawl
Maximum 10 pages to keep response size manageable. Crawl a website starting from a URL
extract
Returns: title, description, OG tags, lang, author, robots, canonical, link count. For the full page content, use the read tool instead. Extract structured metadata from a web page: title, description, OG tags, and more
list_links
Internal links share the same hostname as the source page. Extract all hyperlinks from a web page
read
Uses @mozilla/readability (Firefox Reader View) to extract the main article content, then converts to Markdown. Works best for articles, docs, blogs, and Wikipedia. Fetch any public web page and return its full content as clean Markdown
Example Prompts for Web Scraper in Vercel AI SDK
Ready-to-use prompts you can give your Vercel AI SDK agent to start working with Web Scraper immediately.
"Read https://en.wikipedia.org/wiki/Artificial_intelligence and summarize its history."
"Extract the links from https://news.ycombinator.com/"
"Compare these two links: url1.com and url2.com"
Troubleshooting Web Scraper MCP Server with Vercel AI SDK
Common issues when connecting Web Scraper to Vercel AI SDK through the Vinkius, and how to resolve them.
createMCPClient is not a function
npm install @ai-sdk/mcpWeb Scraper + Vercel AI SDK FAQ
Common questions about integrating Web Scraper MCP Server with Vercel AI SDK.
How does the Vercel AI SDK connect to MCP servers?
createMCPClient from @ai-sdk/mcp and pass the server URL. The SDK discovers all tools and provides typed TypeScript interfaces for each one.Can I use MCP tools in Edge Functions?
Does it support streaming tool results?
useChat and streamText that handle tool calls and display results progressively in the UI.Connect Web Scraper with your favorite client
Step-by-step setup guides for every MCP-compatible client and framework:
Anthropic's native desktop app for Claude with built-in MCP support.
AI-first code editor with integrated LLM-powered coding assistance.
GitHub Copilot in VS Code with Agent mode and MCP support.
Purpose-built IDE for agentic AI coding workflows.
Autonomous AI coding agent that runs inside VS Code.
Anthropic's agentic CLI for terminal-first development.
Python SDK for building production-grade OpenAI agent workflows.
Google's framework for building production AI agents.
Type-safe agent development for Python with first-class MCP support.
TypeScript toolkit for building AI-powered web applications.
TypeScript-native agent framework for modern web stacks.
Python framework for orchestrating collaborative AI agent crews.
Leading Python framework for composable LLM applications.
Data-aware AI agent framework for structured and unstructured sources.
Microsoft's framework for multi-agent collaborative conversations.
Connect Web Scraper to Vercel AI SDK
Get your token, paste the configuration, and start using 5 tools in under 2 minutes. No API key management needed.