# Olostep MCP for AI Agents AI Agent Connect > Olostep lets you scrape web pages at scale using a headless browser API that renders JavaScript and returns clean, structured data instantly. It handles the complexity of modern web environments so your agent can focus on the actual data. Whether you need to pull content from complex apps or run large-scale batch extractions, Olostep turns messy HTML into usable information without the manual overhead. ## Overview - **Category:** developer-tools - **Price:** Free - **Endpoint:** https://edge.vinkius.com/vk_preview_seIYmeX5wOsif72vmko8hRP1y8NfQ4pfMrD0gu0x/ai-agent-connect - **Tags:** headless-browser, data-extraction, web-automation, scraping-batches, structured-data, api-integration ## Description Olostep lets you give your agent the ability to navigate complex websites and pull out exactly what you need without worrying about JavaScript rendering or messy HTML. This Connector takes the grunt work out of web scraping. Instead of building custom scrapers for every new site, you can just tell your agent to go grab the data. It handles the headless browser stuff in the background and hands back clean results. It's a big shift in how you manage data pipelines. You can oversee entire batches of URLs, monitor how many pages have finished, and check your API usage without ever leaving your chat window. By adding Olostep to your toolkit via the Vinkius catalog, you turn your agent into a high-speed data coordinator. It’s about moving from manual clicking and script maintenance to just asking for the data and getting it. ## Tools ### get_agent See the details of a specific scraping agent. Use this to check the configuration of your custom scrapers. ### get_batch_results Pull the final data from a completed batch of URLs. This helps you get the structured results you need. ### get_batch Check the status and details of a specific scraping batch. Use this to see how many pages are finished. ### get_usage View your API stats to see how many pages you have scraped. This helps you stay on top of your credits. ### list_agents See all the scraping agents currently in your account. Use this to manage your collection of data sources. ### list_batches Get a list of all the scraping batches you have created. This is great for keeping track of your history. ### scrape_url Scrape a single page in markdown, html, or text format. This is your primary tool for getting web data. ### check_olostep_status Confirm your API connection is active and working. Use this to verify your setup before starting a job. ### create_agent Set up a new scraping agent for a specific site. This lets you automate extractions for favorite domains. ### create_batch Start a new batch scrape by providing a list of URLs. This is the best way to handle large scale scraping. ## Prompt Examples **Prompt:** ``` Scrape the homepage of example.com as markdown. ``` **Response:** ``` Page scraped successfully. The content is 2,400 words with 15 images and 8 links. Here's the markdown output. Would you like to save it or scrape additional pages? ``` **Prompt:** ``` Create a batch scrape for 5 competitor product pages. ``` **Response:** ``` Batch created with 5 URLs. Job ID: BATCH-7291. Processing has started. 2 of 5 pages are already completed. Would you like me to check back when all are done? ``` **Prompt:** ``` Show my Olostep API usage this month. ``` **Response:** ``` This month: 1,245 pages scraped, 82 batch jobs, 340 MB bandwidth used. You have 8,755 credits remaining on your plan. Would you like to see an agent-level breakdown? ``` ## Capabilities ### Scrape web pages in multiple formats Get website content back as markdown, html, or plain text depending on your needs. ### Manage large scraping batches Create and track multiple URL extractions at once to handle large scale data collection. ### Monitor real-time scraping status Check the progress of active jobs to see exactly how many pages have been completed. ### Retrieve structured batch results Pull the finished data from a completed batch of URLs into your current workspace. ### Access scraping agents and schedules View your directory of research agents and automated schedules for organized data ingestion. ### Check API connectivity and usage Verify your account status and monitor your scraping volume directly through your agent. ### Create dedicated scraping agents Set up new agents to handle specific websites and recurring extraction tasks. ## Use Cases ### Competitor Price Tracking A researcher asks the agent to scrape 50 competitor product pages. The agent uses create_batch to start the job and get_batch_results to summarize the price differences. ### RAG Pipeline Ingestion A developer needs to feed a news site into a database. They use scrape_url to get clean markdown and feed it directly into their RAG pipeline. ### Daily Data Auditing An engineer uses list_batches and get_batch to see if a daily scraping job failed or succeeded, ensuring no data gaps occur in the warehouse. ### Real Estate Lead Generation A team uses create_agent to set up a permanent scraper for a local real estate site and checks get_usage to monitor monthly costs. ## Benefits - Get structured data from JavaScript-heavy sites because Olostep handles the headless browser rendering automatically. - Manage huge scraping jobs with create_batch and list_batches to see everything in one place without manual tracking. - Save time on formatting by using scrape_url to get clean markdown ready for your LLM immediately. - Keep your data pipelines running smoothly by using check_olostep_status to verify your connection is live. - Track your costs and limits easily with get_usage to stay on top of your scraping volume. - Organize your data sources by using create_agent to build dedicated scrapers for different websites. ## How It Works The bottom line is you get a way to turn any website into a structured data source through simple conversation. 1. Subscribe to the Olostep MCP on Vinkius. 2. Get your API Key from the Olostep dashboard. 3. Connect the key to your agent and start asking for data. ## Frequently Asked Questions **Can Olostep scrape websites that use JavaScript?** Yes, it uses a headless browser to render JavaScript automatically. This means it can handle modern web apps that simple scrapers often miss. **How do I get the data from a batch scrape?** Once your batch is finished, your agent can pull the results for you. You can ask it to summarize the findings or save them to a specific file. **Can I get data in different formats like Markdown?** Yes, you can choose between markdown, html, or plain text. Markdown is usually the best choice if you plan to feed the data into an LLM. **How do I check if my API is working?** You can ask your agent to check the connection status. It will verify that your API key is active and the service is reachable. **Is Olostep good for RAG pipelines?** It is a great fit for RAG because it provides clean, structured data. This makes it much easier to index your information without extra cleaning steps. **Can I see how many pages I've scraped?** Yes, you can ask the agent to show your usage stats. It will provide a breakdown of your pages scraped and your remaining credits. **How do I scrape a web page via AI?** Use the `scrape_url` tool with the target URL and optional format (markdown, html, or text). The content is extracted and returned instantly. **Can I scrape multiple URLs at once?** Yes. Use `create_batch` with comma-separated URLs to submit a batch job. Track progress with `get_batch` and retrieve results with `get_batch_results`. **What are scraping agents and how do I use them?** Agents are reusable scraping configurations with custom extraction rules. Use `create_agent` to set one up and `list_agents` to manage them.