• PRODUCT
  • ARTICLE
  • FORUM
  • HUMAN

How to extract structured web content with your AI

Connect Diffbot to your AI client once. From there, your agent handles the parsing of raw HTML into clean data.

Ask AI about this page

Short answer

How can I turn web pages into structured data with AI?

Your agent uses Diffbot to identify page types and pull specific fields like product specs or forum posts. You get organized data instead of raw HTML. This breakdown shows where that data lands.

Data destinations

Where your data lands.

The AI maps raw web elements to these specific outcomes.

PRODUCT

Clean product catalogs

The AI identifies e-commerce pages and pulls prices, names, and specs. You get a structured list ready for a spreadsheet.

ARTICLE

Organized news feeds

It strips away ads and sidebars to grab the headline, author, and body text. This turns messy blogs into clean text blocks.

FORUM

Structured discussions

It parses forum threads into individual posts and user metadata. This makes it easy to analyze conversations.

HUMAN

Manual review needed

If a page doesn't match known patterns, the AI flags it for you. You decide how to handle edge cases that don't fit standard schemas.

The workflow

What your AI does when the URL arrives.

The AI moves from raw HTML to structured intelligence through these steps.

  1. Identify the page

    The AI first looks at the HTML to figure out if it's looking at a store, a blog, or a discussion board.

    analyze_page_type

  2. Pull product details

    If it's a shopping page, the AI grabs the specific attributes like price and SKU.

    extract_product_data

  3. Parse articles

    For news or blogs, the AI isolates the core content from the surrounding clutter.

    extract_article_data

  4. Map forum threads

    It organizes community discussions into a logical sequence of posts and replies.

    extract_forum_thread

Try it

Copy these to start.

Use these prompts once you've connected Diffbot.

Starting points

Swap out the bracketed info for your actual links or specific data needs.

Diffbot Connector

You're all set. Choose your MCP client and follow the setup instructions.

Connector linkhttps://edge.vinkius.com/vk_preview_5HEsQ3kHnUpmnDG46R4fceuP7Qbytx3PkQCCCPhX/mcp

Claude Desktop

Follow the steps below to connect in seconds.

  1. 1In Claude Desktop, open Settings → Connectors.
  2. 2Click “Add custom connector” and paste the connector link above as the remote MCP server URL.
  3. 3Click Add and start a new chat — Diffbot capabilities are ready to use.
Configuration · claude_desktop_config.jsonCopy
{
  "mcpServers": {
    "diffbot-alternative-mcp": {
      "url": "https://edge.vinkius.com/vk_preview_5HEsQ3kHnUpmnDG46R4fceuP7Qbytx3PkQCCCPhX/mcp"
    }
  }
}
Copy into chat04

  • Extract all product details from this URL: [URL]

  • Summarize the main points from this article: [URL]

  • Get a list of all posts in this forum thread: [URL]

  • Analyze the pricing data on this product page: [URL]

  • Claude
  • ChatGPT
  • Cursor
  • VS Code
  • Windsurf
  • Claude Code
  • JetBrains
  • Cline

Start here

Connect Diffbot once, then ask.

Just follow the link to connect your account. Your credentials stay encrypted, and you can start prompting immediately.

Connect Diffbot to your AI

FAQ

How this task behaves.

  • 01

    Can the AI change the content on the original website?

    No. The AI only reads and extracts data. It cannot modify or delete anything on the web pages it visits.

  • 02

    Does the AI automatically save the data to my database?

    No. The AI extracts the data and presents it to you in the chat. You must then copy that data or use another tool to save it.

  • 03

    What happens if the website has a login wall?

    The AI can only access pages that are publicly available on the web.

  • 04

    Can it extract data from images?

    No. The tools are designed to parse HTML structure, not perform OCR on images.

  • 05

    Can the AI browse the web on its own?

    No. The AI only accesses specific URLs that you provide in your prompts.

  • More questions about Diffbot? The Connector page answers them. See everything the Diffbot Connector can do

Connect Diffbot to Claude, Cursor, ChatGPT & more