crw

Overview

fastCRW is a high-performance web scraping and crawling API designed for AI agents. Built in Rust, it offers a Firecrawl-compatible API with significantly better performance metrics, consuming only 6MB of RAM while being up to 5.5x faster than Firecrawl. The project features both self-hosted and cloud deployment options, with built-in MCP server functionality allowing easy integration with AI agents through stdio and HTTP transports.

Try asking AI

After installing, here are 3 things you can ask your AI assistant:

• scrape

  Convert any URL to markdown, HTML, JSON, or links

  Try:Scrape https://example.com for markdown contentTry:Get the HTML content from https://example.comTry:Extract structured data from https://example.com
• crawl

  Async BFS website crawler with rate limiting

  Try:Crawl https://docs.example.com with depth 2 and max 50 pagesTry:Extract content from all pages of https://example.comTry:Scrape an entire website starting from https://example.com
• map

  Discover all URLs on a site instantly

  Try:Map all URLs on https://example.comTry:Find all pages on https://example.comTry:Get the site structure of https://example.com
• search

  Web search + content scraping with built-in SearXNG

  Try:Search for 'open source web scraper 2026'Try:Find recent news about AI web scraping toolsTry:Search the web for content about Rust web scrapers

Comparable tools

Installation

curl -fsSL https://raw.githubusercontent.com/us/crw/main/install.sh | CRW_BINARY=crw sh
crw setup

pip install crw

brew install us/crw/crw

# Local mode
claude mcp add crw -- npx crw-mcp

# Cloud mode
claude mcp add -e CRW_API_URL=https://fastcrw.com/api -e CRW_API_KEY=your-key crw -- npx crw-mcp

npx crw-mcp

Compare crw with