webfetch-mcp

Overview

WebFetch.MCP is a self-hosted MCP server that bridges the gap between local LLMs and the live web. It offers both web search functionality and URL content fetching capabilities without requiring API keys. The server uses SearxNG as a metasearch engine and Mozilla Readability for content extraction, with support for 70+ search engines including specialized ones like Google Scholar, arXiv, and GitHub. The implementation includes smart rate limiting (12 calls per 5 minutes) and configurable options for source weighting, content filtering, and domain restrictions.

Try asking AI

After installing, here are 6 things you can ask your AI assistant:

• web_search

  Search the web with 70+ configurable engines for fresh, real-time data

  Try:Search for recent AI research on transformer architecturesTry:Find Python asyncio docs site:python.orgTry:Search for recent climate data from government sources
• web_fetch

  Extract clean content from a specific URL using Mozilla Readability

  Try:Extract content from https://example.com/articleTry:Get main text from https://arxiv.org/abs/2305.12345Try:Fetch and summarize https://news.example.com/latest

Comparable tools

Installation

git clone https://github.com/searxng/searxng-docker.git
cd searxng-docker
sed -i "s|ultrasecretkey|$(openssl rand -hex 32)|g" searxng/settings.yml
docker compose up -d

git clone https://github.com/manull/webfetch-mcp.git
cd webfetch-mcp
npm install
node server.mjs

{
  "mcpServers": {
    "webfetch": {
      "command": "node",
      "args": ["/full/path/to/webfetch-mcp/server.mjs"],
      "env": {
        "SEARXNG_BASE": "http://localhost:8080"
      }
    }
  }
}

FAQ

Does WebFetch.MCP require API keys?

No, it uses SearxNG as a metasearch engine and doesn't require any API keys for search functionality.

What rate limits are imposed on the server?

WebFetch.MCP implements a smart rate limit of 12 calls per 5 minutes with 8 calls per 30-second burst to prevent spam while allowing reasonable research workflows.

Compare webfetch-mcp with