crawlbase-mcp

Overview

Crawlbase MCP is a Model Context Protocol server that bridges AI agents and the live web, enabling LLMs to fetch fresh, structured content instead of relying on outdated training data. It handles the complexity of web scraping by providing JavaScript rendering for modern web apps, proxy rotation for anti-bot evasion, and structured outputs in HTML, Markdown, or screenshot formats. The server powers integrations with popular AI clients like Claude, Cursor, and Windsurf.

Try asking AI

After installing, here are 5 things you can ask your AI assistant:

• crawl

  Fetch raw HTML from a webpage

  Try:Crawl Hacker News and return the raw HTMLTry:Get the HTML content of TechCrunch homepage
• crawl_markdown

  Extract clean Markdown from a webpage

  Try:Crawl Hacker News and return top stories in markdownTry:Convert TechCrunch homepage to markdown format
• crawl_screenshot

  Capture screenshots of a webpage

  Try:Take a screenshot of TechCrunch homepageTry:Capture an image of the current weather page

Comparable tools

Installation

{
  "mcpServers": {
    "crawlbase": {
      "type": "stdio",
      "command": "npx",
      "args": ["@crawlbase/mcp@latest"],
      "env": {
        "CRAWLBASE_TOKEN": "your_token_here",
        "CRAWLBASE_JS_TOKEN": "your_js_token_here"
      }
    }
  }
}

git clone https://github.com/crawlbase/crawlbase-mcp.git
cd crawlbase-mcp
npm install

# Start HTTP server
CRAWLBASE_TOKEN=your_token CRAWLBASE_JS_TOKEN=your_js_token npm run start:http

FAQ

What does Crawlbase MCP do?

Crawlbase MCP connects AI agents to live web data through structured scraping, JavaScript rendering, and anti-bot protection, enabling fresh data in AI workflows.

What commands are available in Crawlbase MCP?

The server provides three main commands: 'crawl' (fetch raw HTML), 'crawl_markdown' (extract clean Markdown), and 'crawl_screenshot' (capture screenshots).

Compare crawlbase-mcp with