MCP Catalogs
← HomeScrapling screenshot

Scrapling

by D4Vinci·★ 50,346·Score 62

Web scraping framework with adaptive parsing and anti-bot capabilities, now integrated as an MCP server.

web-scrapingai-llmdeveloper-tools
4,765
Forks
5
Open issues
this month
Last commit
2d ago
Indexed

Overview

Scrapling is a comprehensive web scraping framework that has been extended to function as an MCP server, allowing it to be integrated into AI agent workflows. It features adaptive parsing that can learn from website changes and automatically relocate elements, advanced fetchers that can bypass anti-bot systems like Cloudflare Turnstile, and a scalable spider framework for concurrent, multi-session crawls. The MCP integration enables users to leverage these web scraping capabilities directly within AI agent environments through standardized tool calling.

Try asking AI

After installing, here are 5 things you can ask your AI assistant:

you:Extracting structured data from dynamic websites with changing layouts
you:Large-scale web crawling with automatic proxy rotation and anti-bot bypass
you:Integrating web scraping capabilities into AI agent workflows
you:Does Scrapling support JavaScript-rendered websites?
you:How does the adaptive parsing work?

When to choose this

Choose Scrapling for adaptive web scraping when you need to handle frequently changing websites or require robust anti-bot bypass capabilities within your AI agent workflows.

When NOT to choose this

Don't choose Scrapling if you need to scrape websites that explicitly prohibit scraping or if you require support for JavaScript-heavy applications with complex client-side rendering.

Tools this server exposes

5 tools extracted from the README
  • fetch

    Fetch website content using different fetcher types

  • css

    Parse HTML content using CSS selectors

  • start

    Start a web scraping spider

  • adaptive

    Enable adaptive parsing for element selection

  • auto_save

    Automatically save scraped data

Note: Tool names inferred from Python code examples and documentation, but no explicit MCP tool section was found

Comparable tools

browserless-mcppuppeteer-mcpplaywright-mcpbeautifulsouprequests-html

Installation

Installing Scrapling MCP Server

  1. Install the Scrapling package via pip:

``bash pip install scrapling ``

  1. Configure your MCP client to use the Scrapling server. For Claude Desktop, add to claude_desktop_config.json:

``json { "mcpServers": { "scrapling": { "command": "python", "args": ["-m", "scrapling", "mcp"] } } } ``

FAQ

Does Scrapling support JavaScript-rendered websites?
Yes, through its DynamicFetcher and StealthyFetcher which can handle JavaScript execution and modern anti-bot measures.
How does the adaptive parsing work?
Scrapling's parser learns from website changes and can automatically relocate elements when page structures update, maintaining your scraping selectors.

On Hacker News

Recent discussion from the developer community.

Compare Scrapling with

Scrapling vs ultimate_mcp_serverScrapling vs mcp-server-chartScrapling vs everythingScrapling vs filesystemScrapling vs time
GitHub →

Last updated · Auto-generated from public README + GitHub signals.