• TEXT
  • CONTEXT
  • DATA
  • HUMAN

How to scrape web pages to markdown with your AI

Connect Firecrawl to your AI client to turn any URL into clean Markdown. Once connected, your agent can read live web data directly.

Ask AI about this page

Short answer

How can I turn a website into Markdown for my AI?

Use the scrape_page tool to pull the main content from a URL. Your AI receives a clean Markdown version of the page, stripped of junk code and ads. This makes it easy to feed web data into your prompts without hitting token limits with HTML bloat.

The results.

Where your data lands.

The output of the scraping process lands in these formats.

TEXT

Clean Markdown

The tool converts the page into structured Markdown. Your AI gets readable text with headers and lists preserved.

CONTEXT

Prompt Input

The scraped content becomes part of your conversation. You can immediately ask questions about the specific site content.

DATA

Structured Content

The AI receives organized data instead of raw HTML. This helps your agent follow instructions more accurately.

HUMAN

Manual Review

You check the final Markdown output. If the site has weird layouts, you decide if the data is clean enough to use.

The workflow

What your AI does when a URL is provided.

The AI handles the heavy lifting of navigating and cleaning the web page.

  1. Fetch the page

    Your agent sends the URL to the scraper to access the live site.

    scrape_page

  2. Render dynamic content

    The tool waits for JavaScript to load so you don't miss content hidden behind menus or popups.

    scrape_page

  3. Strip HTML noise

    The scraper removes scripts, styles, and ads that would otherwise confuse your agent.

    scrape_page

  4. Format as Markdown

    The final output is converted into a clean Markdown string ready for your next prompt.

    scrape_page

Try it

Copy these to start.

Use these prompts to trigger the scraping process.

Starting points

These are starting points. Swap out the URLs and topics for your specific needs.

Firecrawl Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_H3lslwPhMfSWLHyTzOylzN5WvZ9uNMEMQ6eYZqT4/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Firecrawl capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "firecrawl-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_H3lslwPhMfSWLHyTzOylzN5WvZ9uNMEMQ6eYZqT4/mcp"
    }
  }
}
Copy into chat04

  • Scrape https://example.com and summarize the main article in Markdown.

  • Use the scrape_page tool on this URL: https://docs.example.com and list the installation steps.

  • Read the content at https://blog.example.com and tell me who the author is.

  • Convert the text from https://product-page.com into a clean Markdown list of features.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Firecrawl once, then ask.

Click the link to connect your account. Your credentials stay encrypted, and you can start scraping immediately in your preferred AI client.

Connect Firecrawl to your AI

FAQ

How the scraping works

  • 01

    Can the AI scrape pages that require a login?

    No. The scrape_page tool accesses public URLs. It cannot bypass authentication or log into your personal accounts.

  • 02

    Does it work on sites that use a lot of JavaScript?

    Yes. The tool is designed to handle dynamic content by rendering the page before extracting the text.

  • 03

    Can the AI change the content on the website?

    No. This tool is read-only. It can only pull data from the web, not modify or post content to any site.

  • 04

    Will the AI save the Markdown files to my computer?

    No. The tool provides the Markdown text directly to your AI client. You must manually save the text if you want a file.

  • 05

    What happens if a website blocks scrapers?

    If a site has aggressive anti-bot protections that block the tool, the scrape_page command will fail and your agent will report the error.

  • More questions about Firecrawl? The Connector page answers them. See everything the Firecrawl Connector can do

Connect Firecrawl to Claude, Cursor, ChatGPT & more