• CLEAN DATA
  • RAW TEXT
  • HUMAN REVIEW
  • CONTEXT

How to scrape web pages to Markdown with your AI

Connect Firecrawl to your AI client once. From then on, your agent can pull clean text from any website to use in your chats.

Ask AI about this page

Short answer

How can I turn a website into Markdown for my AI?

Your AI uses the scrape_url tool to fetch a URL and convert the HTML into structured Markdown. This removes the clutter like nav bars and ads, leaving only the text your agent needs to work. You'll get clean data ready for reasoning or summarization.

The results of a scrape.

What your AI produces.

The output changes based on how you ask your agent to process the data.

CLEAN DATA

Structured Markdown

The tool strips HTML tags and leaves behind clean Markdown. Your AI gets the core content without the noise.

RAW TEXT

Plain text extraction

If you don't need formatting, your agent can just grab the raw text. This is useful for simple pattern matching.

HUMAN REVIEW

Manual verification

You decide if the scraped content is accurate. The AI handles the heavy lifting of extraction, but you keep the final say.

CONTEXT

RAG readiness

The Markdown output is perfect for feeding into your local knowledge base. It makes searching through web data much easier.

The workflow

What your AI does when a URL arrives.

The AI handles the messy part of web navigation so you don't have to.

  1. Fetch the page

    Your agent sends the URL to the connector to grab the live HTML content.

    scrape_url

  2. Strip the noise

    The tool removes scripts, styles, and ads that usually confuse LLMs.

    scrape_url

  3. Convert to Markdown

    The HTML is transformed into a clean Markdown format that preserves headings and links.

    scrape_url

  4. Deliver content

    The structured text is handed back to your agent for your next instruction.

    scrape_url

Try it

Copy these to start.

Use these prompts to trigger the scraping process immediately.

Starting points

Swap out the website addresses for the ones you actually want to read.

Firecrawl Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_VUpOJRFVTRGTwbzAMeVqIv4t7KoCrITMFS6MmDGP/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Firecrawl capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "firecrawl-extended-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_VUpOJRFVTRGTwbzAMeVqIv4t7KoCrITMFS6MmDGP/mcp"
    }
  }
}
Copy into chat04

  • Scrape this URL and give me the Markdown: https://example.com/article

  • Read the content at https://docs.example.com and summarize the main points in Markdown.

  • Convert the text from https://blog.example.com into a clean Markdown list of all headers.

  • Look at https://news.example.com and extract the main article text as Markdown.

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Firecrawl once, then ask.

Just click the link to connect. Your credentials stay encrypted, and you can start scraping immediately in your preferred client.

Connect Firecrawl to your AI

FAQ

How the scraping works

  • 01

    Can the AI scrape content behind a login?

    No. The scrape_url tool only accesses publicly available URLs.

  • 02

    Will the AI change the original website?

    No. The tool only reads the data. It cannot modify, delete, or interact with the website itself.

  • 03

    Does it work on JavaScript-heavy sites?

    Yes. The connector is designed to handle modern web pages that rely on JavaScript.

  • 04

    Can I ask the AI to save the Markdown to a file?

    The tool provides the Markdown to your agent. Whether that gets saved to a file depends on the capabilities of your specific AI client.

  • 05

    What happens if a site blocks scrapers?

    If the site blocks the request, the tool will return an error to your agent, and the scrape will fail.

  • More questions about Firecrawl? The Connector page answers them. See everything the Firecrawl Connector can do

Connect Firecrawl to Claude, Cursor, ChatGPT & more