Crawlbase Connector for AI agents.
10 live capabilities
Automate web scraping and data extraction from Amazon, LinkedIn, and Google.
Waiting for input…
Why people use Crawlbase
Crawlbase Web Scraping and Data Extraction Overview
With this Connector, you just tell your agent what you want. It handles the headless browsing, the JS execution, and the proxy rotation behind the scenes. You get the final data you need without ever seeing a 403 Forbidden error.
What Vinkius changes
You get structured web data through natural language instead of manual coding.
Use it from Claude, ChatGPT, Cursor or another AI client you already have.
One account · 5,900+ Connectors
- Real-world use case 01
Tracking competitor pricing
A growth lead wants to track Amazon prices.
- Real-world use case 02
Gathering social proof
A researcher needs social proof.
- Real-world use case 03
Debugging web extraction
A developer is debugging a scraper.
Complete set · 10capabilities
The complete Crawlbase capability set.
These are the exact actions your AI can choose when you ask it to work with Crawlbase.
01—04
4 capabilities in this set.
Part of 10 available through Crawlbase.
- 01 Capability
Scrape html
Pulls HTML content from specific areas of a site using datacenter proxies. It identifies bounded routing spaces for clean extraction.
- 02 Capability
Get screenshot link
Creates a URL for a snapshot of a web page. Use it to get visual validation of a rendered site.
- 03 Capability
Scrape amazon
Navigates Amazon's internal arrays to get product details. It mitigates common e-commerce constraints.
- 04 Capability
Scrape linkedin
Matches specific structural blueprints to extract LinkedIn profiles. It verifies constraints for accurate data.
05—07
3 capabilities in this set.
Part of 10 available through Crawlbase.
- 05 Capability
Scrape facebook
Exports active social pages based on your defined rules. It enumerates attached rules for easy export.
- 06 Capability
Scrape google serp
Parses Google search results while bypassing CAPTCHAs. It identifies active arrays for search data.
- 07 Capability
Scrape twitter
Fetches mapped limits from Twitter (X) profiles. It uses Crawlbase X extraction for social data.
08—10
3 capabilities in this set.
Part of 10 available through Crawlbase.
- 08 Capability
Custom scrape
Sets up custom proxies with unique headers and crawling logic. It provisions highly-available request payloads.
- 09 Capability
Scrape js rendered
Retrieves data from pages that require JavaScript execution. It tracks payload IDs to get dynamic content.
- 10 Capability
Scrape json format
Forces raw web outputs into a strictly structured JSON format. It handles the structural extraction for you.
Set up in minutes
One URL. Then ask Crawlbase to work.
Claude and ChatGPT only need the Connector URL. Copy it once, add it in settings, and use Crawlbase from the conversation.
Choose your client
Live previewAdvanced clients IDE · CLI
Claude · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp - Step 01
Open Connectors
In Claude Web or Claude Desktop, open Settings and choose Connectors.
- Step 02
Add the URL
Choose Add custom connector, name it Crawlbase, and paste the URL above.
- Step 03
Turn it on in chat
Select +, open Connectors, and enable Crawlbase for the conversation.
ChatGPT · Web + desktop
Connector URL · ready to paste
Streamable HTTPhttps://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp - Step 01
Open MCP settings
On desktop, open Settings and MCP servers. On web, open your workspace app or connector settings.
- Step 02
Add the URL
Choose Add server with Streamable HTTP, or create a custom MCP app, then paste the Crawlbase URL.
- Step 03
Save and start
Save the connection and enable Crawlbase in your conversation. Desktop may ask you to restart once.
Cursor · IDE configuration
Advanced setup
{
"mcpServers": {
"crawlbase": {
"url": "https://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp"
}
}
}- Step 01
Open MCP Settings
Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows/Linux) → search "MCP Settings"
- Step 02
Add the server config
Paste the JSON configuration above into the mcp.json file that opens
- Step 03
Save the file
Cursor will automatically detect the new Connector
- Step 04
Start using Crawlbase
Open Agent mode in chat and ask: "Using Crawlbase, help me...". 10 tools available
VS Code Copilot · IDE configuration
Advanced setup
{
"mcpServers": {
"crawlbase": {
"url": "https://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp"
}
}
}- Step 01
Create MCP config
Create a .vscode/mcp.json file in your project root
- Step 02
Add the server config
Paste the JSON configuration above
- Step 03
Enable Agent mode
Open GitHub Copilot Chat and switch to Agent mode using the dropdown
- Step 04
Start using Crawlbase
Ask Copilot: "Using Crawlbase, help me...". 10 tools available
Windsurf · IDE configuration
Advanced setup
{
"mcpServers": {
"crawlbase": {
"url": "https://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp"
}
}
}- Step 01
Open MCP Settings
Go to Settings → MCP Configuration or press Cmd+Shift+P and search "MCP"
- Step 02
Add the server
Paste the JSON configuration above into mcp_config.json
- Step 03
Save and reload
Windsurf will detect the new server automatically
- Step 04
Start using Crawlbase
Open Cascade and ask: "Using Crawlbase, help me...". 10 tools available
Cline · IDE configuration
Advanced setup
{
"mcpServers": {
"crawlbase": {
"url": "https://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp"
}
}
}- Step 01
Open Cline MCP Settings
Click the Connectors icon in the Cline sidebar panel
- Step 02
Add remote server
Click "Add Connector" and paste the configuration above
- Step 03
Enable the server
Toggle the server switch to ON
- Step 04
Start using Crawlbase
Ask Cline: "Using Crawlbase, help me...". 10 tools available
Claude Code · Terminal command
Advanced setup
claude mcp add crawlbase --transport http "https://edge.vinkius.com/vk_preview_AcN0JE7lQv9RjbUnrxQnYlI9bigE4gxGmDTPSZBI/mcp"- Step 01
Install Claude Code
Run npm install -g @anthropic-ai/claude-code if not already installed
- Step 02
Add the Connector
Run the command above in your terminal
- Step 03
Verify the connection
Run claude mcp to list connected servers, or type /mcp inside a session
- Step 04
Start using Crawlbase
Ask Claude: "Using Crawlbase, show me...". 10 tools are ready
Where the request belongs
Work Crawlbase can move forward.
This is for the data professional who is tired of their scraping scripts breaking every time a website changes its layout or blocks their IP.
Data Analyst
Pulling clean datasets from complex websites for reporting without writing a single line of Python.
Growth Hacker
Monitoring competitor pricing on Amazon or tracking social feeds on LinkedIn in real-time.
Developer
Testing extraction logic and JS rendering behavior through natural conversation.
Market Researcher
Gathering snapshots and deep web crawls for offline analysis and trend reporting.
Build the capability set
Add more capabilities.
Each Connector adds new actions and data without changing how you work.
Browse ConnectorsSpider
High-performance Rust-powered web scraping and crawling. scrape, crawl, and search up to 100K+ pages/second with built-in anti-bot protection.
ScrapingBee
Powerful web scraping with JS rendering, premium proxies, and AI-powered data extraction directly from your AI agent.
ScraperAPI
Equip your AI agent with proxy rotation and headless browsers to extract HTML, Google SERPs, and Amazon data at scale.
Nimbleway
Web data collection and scraping via Nimbleway. extract content and search the web directly from your AI agent.
Scrapfly
Scrape web data at scale with a managed API that handles proxies, browser rendering, and anti-bot bypassing automatically.
ScrapingAnt
Extract web data reliably with rotating proxies, headless Chrome rendering, and CAPTCHA solving built into every request.
Bring your own AI
Change the model, client or framework. Keep Crawlbase connected.
-
Claude -
ChatGPT -
Gemini -
Cursor -
VS Code -
Windsurf -
ZCode -
Cline -
Zed -
Continue -
Kiro -
Roo Code -
Zencoder -
Goose -
Void -
Augment Code -
Amp -
Qodo -
Tabnine -
Pieces -
Sourcegraph Cody -
JetBrains -
Warp -
Amazon Q -
Antigravity -
BoltAI -
Raycast -
Jan -
LM Studio -
AnythingLLM -
Open WebUI -
Msty -
Cherry Studio -
LibreChat -
TypingMind -
Chorus -
5ire -
n8n -
LangChain -
LlamaIndex -
CrewAI -
Vercel AI SDK
Before you connect
Questions about Crawlbase.
The practical details behind the request, access and result.
Can Crawlbase MCP scrape LinkedIn profiles?
Yes, it includes specific capabilities to retrieve LinkedIn profile data while matching structural blueprints to ensure accuracy.
Does Crawlbase MCP handle CAPTCHAs?
Yes, it is designed to bypass CAPTCHAs automatically, especially when parsing search engine results or high-traffic sites.
Can I use Crawlbase MCP for Amazon products?
Absolutely. It has a dedicated capability to navigate Amazon's internal arrays and extract product information while avoiding blocks.
How does Crawlbase MCP handle JavaScript?
It uses a headless engine to render JavaScript, allowing your agent to see and extract content from dynamic web applications.
Can Crawlbase MCP export data in JSON?
Yes, you can force the Connector to return data in a strictly structured JSON format, making it easy to use the data in other workflows.
Does Crawlbase MCP work with Cursor and Claude?
Yes, because it's an Connector, it connects directly to any compatible client like Claude, Cursor, Windsurf, and VS Code.
When should I use the JavaScript (JS) Token versus the Normal Token?
Use the Normal Token for fast, static HTML extraction. Switch to the JavaScript Token when the target site uses frameworks like React or Angular, where content is rendered dynamically in the browser. The 'scrape_js_rendered' capability requires the JS Token to function.
Can my agent bypass CAPTCHAs while scraping Google or LinkedIn?
Yes. Crawlbase is built to handle CAPTCHAs and blocks natively. When you use specialized capabilities like 'scrape_google_serp' or 'scrape_linkedin', the agent routes your requests through Crawlbase's advanced proxy infrastructure to ensure successful data extraction.
How do I get a structured JSON response instead of raw HTML?
Use the 'scrape_json_format' capability or the specialized scraper capabilities (Amazon, LinkedIn, etc.). These trigger Crawlbase's auto-extraction pipelines, which analyze the page structure and return specific data fields in a clean JSON format.
One connection away
Give your agent a direct line to Crawlbase.
Connect Crawlbase once. Keep it beside 5,900+ managed Connectors when the next task needs more.
Explore every Connector No credit card required · Free tier available