crawl4ai-mcp-server

概述

Crawl4AI MCP Server 是一个专门实现的 MCP 服务器，通过模型上下文协议提供网页抓取和爬取功能。它提供四个主要工具：抓取工具用于单页内容提取，爬取工具用于多页广度优先爬取和自适应停止，网站爬取工具用于全面的网站爬取，以及站点地图爬取工具。该服务器是自托管且免费的，为 Firecrawl API 提供替代方案，同时提供类似功能。它支持多种 AI 开发环境，包括 OpenAI Agents SDK、Cursor 和 Claude Code。

试试问 AI

装完之后，这里有 5 个你可以让 AI 做的事：

• scrapescrape(url, output_dir, crawler, browser, script, timeout_sec)

  Fetch a single URL and return markdown content.

  试试:Scrape the content from https://example.com试试:Extract markdown content from https://news.example.com/article试试:Get the full text content of https://blog.example.com/post
• crawlcrawl(seed_url, max_depth, max_pages, same_domain_only, include_patterns, exclude_patterns, adaptive, output_dir, crawler, browser, script, timeout_sec)

  Multi-page breadth-first crawling with filtering and adaptive stopping.

  试试:Crawl https://docs.example.com with adaptive stopping试试:Get content from up to 10 pages starting from https://blog.example.com试试:Crawl https://news.example.com but only from technology sections
• crawl_sitecrawl_site(entry_url, output_dir, max_depth, max_pages, ...)

  Comprehensive site crawling with persistence.

  试试:Crawl the entire website at https://example.com试试:Download all pages from https://docs.example.com试试:Archive content from https://blog.example.com to local storage
• crawl_sitemapcrawl_sitemap(sitemap_url, output_dir, max_entries, ...)

  Sitemap-based crawling with persistence.

  试试:Crawl all URLs in https://example.com/sitemap.xml试试:Process the sitemap at https://blog.example.com/sitemap.xml试试:Extract content from all pages listed in https://news.example.com/sitemap.xml

可对比工具

安装

# 拉取预构建镜像
docker pull uysalsadi/crawl4ai-mcp-server:latest

# 测试安装
python test-config.py

git clone https://github.com/uysalsadi/crawl4ai-mcp-server.git
cd crawl4ai-mcp-server
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python -m playwright install chromium

{
  "mcpServers": {
    "crawl4ai-mcp": {
      "command": "docker",
      "args": [
        "run", "--rm", "-i",
        "--volume", "/tmp/crawl4ai-crawls:/app/crawls",
        "uysalsadi/crawl4ai-mcp-server:latest"
      ],
      "env": {
        "CRAWL4AI_MCP_LOG": "INFO"
      }
    }
  }
}

FAQ

这个服务器与 Firecrawl 相似吗？

是的，它提供与 Firecrawl API 类似的功能，但它是自托管的且免费的，无需 API 密钥。

服务器实现了哪些安全功能？

服务器默认阻止内部网络、本地主机和私有 IP，以防止抓取敏感或私密内容。

crawl4ai-mcp-server 对比