mcp-server-webscan

概述

MCP Webscan Server 是一个基于 TypeScript 的实现，提供网页抓取、分析和信息提取工具。它具有页面抓取并转换为 Markdown、链接提取和过滤、递归网站爬行和深度控制、断开链接检查、URL 模式匹配和 XML 站点地图生成等功能。该服务器运行在 stdio 传输上，与 Claude Desktop 等 MCP 客户端兼容。代码结构良好，服务、工具和类型之间有清晰的分离。

试试问 AI

装完之后，这里有 5 个你可以让 AI 做的事：

• fetch-pageurl: string, selector?: string

  Fetches a web page and converts it to Markdown

  试试:Fetch the content from https://example.com and convert it to Markdown试试:Get the main content from https://news.ycombinator.com using the .post CSS selector
• extract-linksurl: string, baseUrl?: string, limit?: number

  Extracts all links from a web page with their text

  试试:Extract all links from https://example.com试试:Get links from https://example.com but only those from example.com domain, limit to 50
• crawl-siteurl: string, maxDepth?: number

  Recursively crawls a website up to a specified depth

  试试:Crawl https://example.com with a maximum depth of 2试试:Perform a deep crawl of https://blog.example.com up to depth 3
• check-linksurl: string

  Checks for broken links on a page

  试试:Check all links on https://example.com for broken ones试试:Verify if there are any dead links on my website https://myblog.com
• find-patternsurl: string, pattern: string

  Finds URLs matching a specific pattern

  试试:Find all product URLs matching the pattern /product/[0-9]+ on https://shop.example.com试试:Search for all email addresses on https://example.com using pattern [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
• generate-site-mapurl: string, maxDepth?: number, limit?: number

  Generates a simple XML sitemap by crawling

  试试:Generate a sitemap for https://example.com with maximum depth of 2试试:Create an XML sitemap for https://blog.example.com with a limit of 100 URLs

可对比工具

安装

npx -y @smithery/cli install mcp-server-webscan --client claude

# 克隆仓库
git clone <repository-url>
cd mcp-server-webscan

# 安装依赖
npm install

# 构建项目
npm run build

{
  "mcpServers": {
    "webscan": {
      "command": "node",
      "args": ["path/to/mcp-server-webscan/build/index.js"],
      "env": {
        "NODE_ENV": "development",
        "LOG_LEVEL": "info"
      }
    }
  }
}

FAQ

网站爬行的最大深度是多少？

默认最大深度为 2，但可以在 0-5 之间配置爬行站点和生成站点地图工具。

可以按基础 URL 过滤链接吗？

是的，extract-links 工具接受可选的 baseUrl 参数来过滤链接。

mcp-server-webscan 对比