# Firecrawl MCP for AI Agents AI Agent Connect

> Firecrawl. Scrape and crawl the web to turn complex websites into LLM-ready markdown. Audit site structures and extract clean data without the headache of manual scraping or messy HTML. Use it to build knowledge bases, monitor competitors, or map out site hierarchies instantly. It provides clean, structured data that your agent can actually use for analysis or content generation.

## Overview
- **Category:** friends-mcp
- **Price:** Free
- **Endpoint:** https://edge.vinkius.com/vk_preview_fc9lRjafM7OLZXZoNzbjhXgbJ3RhOnI5LpERnMdC/ai-agent-connect
- **Tags:** data-extraction, markdown-conversion, rag-pipelines, recursive-crawling, llm-ready, web-data

## Description

You spend way too much time trying to get clean data out of messy websites. Firecrawl changes that by letting your AI agent handle the scraping for you. Instead of copy-pasting or writing custom scripts to parse HTML, you just tell your agent which URL you need. It handles the heavy lifting, turning the site into clean markdown that your agent can actually understand and use. You can grab a single page or tell it to crawl an entire site to build a knowledge base. It's one of the most reliable ways to get real-time web data into your workflow, and you can find it easily in the Vinkius catalog. You get structured data, not a mess of tags and scripts. It works by identifying the core content of a page and stripping away the noise like ads, headers, and navigation menus. This means your agent can focus on the actual information you want, whether that's a product description, a blog post, or a set of pricing details. You don't have to worry about site layouts changing or different formatting styles breaking your process. It just works, giving you a consistent output every time you need to pull information from the web. It's built for speed and accuracy, making it a go-to for anyone who needs to turn the internet into usable data.

## Tools

### get_api_usage
Check Firecrawl credit usage

### cancel_active_crawl
Stop an ongoing crawl

### start_crawl
Returns a job ID.

Recursively crawl a website

### map_website_structure
Discover all URLs on a site

### get_crawl_status
Check if a crawl job is still running or finished. This helps you know when the data is ready for your agent to use.

### scrape_url
Get the markdown content from a single URL. It's perfect for when you just need one page of data quickly.

## Prompt Examples

**Prompt:** 
```
Scrape https://vinkius.com/about and give me the markdown content.
```

**Response:** 
```
I've scraped that page for you. Here is the clean markdown content regarding Vinkius's mission and team. It's ready for you to use in your project.
```

**Prompt:** 
```
Start a crawl job for https://docs.firecrawl.dev with a limit of 10 pages.
```

**Response:** 
```
Crawl job started! The Job ID is `job_12345`. I'm monitoring the progress now. So far, it's identified the main documentation categories. I'll let you know when it's finished.
```

**Prompt:** 
```
Map the website structure of https://github.com.
```

**Response:** 
```
I've mapped out the hierarchy for GitHub. It includes major sections like /features, /pricing, and /trending. Here is the full list of identified subdirectories:

* /features
* /pricing
* /trending
* /explore
```

## Capabilities

### Extract clean markdown from URLs
Your agent pulls the core content of a page while stripping away useless HTML tags.

### Crawl entire websites
Start multi-page jobs to gather data from a whole site in one go.

### Monitor crawl progress
Check the status of running jobs to know exactly when your data is ready.

### Map website hierarchies
Get a full visual layout of a site's structure and page distribution.

### Manage crawl history
List and delete past jobs to keep your scraping workspace organized.

## Use Cases

### Competitor Site Audits
A marketing lead asks the agent to map a competitor's site to see how they organize their product pages and content.

### Knowledge Base Building
A developer tells the agent to crawl a documentation site to create a local reference guide in markdown format.

### Real-time Price Tracking
A growth engineer uses the agent to pull data from a specific URL every morning to check for pricing changes.

### SEO Hierarchy Analysis
An SEO pro asks the agent to map out a site's structure to find missing meta tags or deep navigation issues.

## Benefits

- Stop fighting with messy HTML. By getting clean markdown from a single URL, your agent won't get confused by site code.
- Handle massive data collection. You can start multi-page crawls to grab hundreds of pages without manual intervention.
- See the big picture instantly. Mapping out a website's hierarchy lets you understand site depth in seconds.
- Track your progress easily. Checking the status of running jobs keeps you updated on long-running tasks.
- Keep your workspace tidy. Removing old crawl jobs from your history lets you stay organized and focused.

## How It Works

The bottom line is you get structured web data without writing a single line of scraping code.

1. Get your Firecrawl API key from their official dashboard.
2. Connect the Firecrawl MCP to your AI client via the Vinkius catalog.
3. Ask your agent to scrape a URL or start a site crawl.

## Frequently Asked Questions

**How does Firecrawl help me get data from websites?**
It turns messy web pages into clean markdown that your AI client can actually read. This means you don't have to deal with HTML tags or scripts.

**Can I use Firecrawl to crawl an entire website?**
Yes, you can start multi-page crawl jobs. Your agent will handle the heavy lifting and gather data from across the whole site.

**What is the difference between scraping and crawling with Firecrawl?**
Scraping gets you data from a single URL, while crawling gathers data from multiple pages across a whole site.

**Can Firecrawl help with SEO audits?**
It can map out a website's entire hierarchy. This helps you see the site structure and page distribution without clicking every link.

**How do I know when my web crawl is finished?**
You can ask your agent to check the status of a specific job. It'll tell you if it's still running or if the data is ready.

**Is the data from Firecrawl easy for an AI to use?**
Yes, because it returns markdown. This format is optimized for your AI client to process without getting confused by site code.

**How do I find my Firecrawl API Key?**
Log in to your [**Firecrawl.dev dashboard**](https://www.firecrawl.dev/app/settings/api-keys), and you will find your API Key under the settings. Copy and paste it below.

**Can the agent crawl multiple pages at once?**
Yes. Use the `crawl_url` tool providing the base URL. Firecrawl will start a job to extract all subpages, and you can monitor the status via `get_crawl_status`.

**Is it possible to see the website structure before scraping?**
Yes. The `map_website` tool allows your agent to retrieve a hierarchy of the site, giving you an audit of the structure before performing a full scrape or crawl.

**Can I scrape content excluding headers and footers?**
Yes! The `scrape_url` tool includes an `onlyMainContent` parameter. When set to true, Firecrawl uses AI to extract only the core article or page content.

**How long does a recursive crawl take?**
Crawl time depends on the site size and depth. Use the `get_crawl_status` tool to monitor progress and retrieve results once the job is complete.