mcpbeat Sign in

Web Scraping MCP Server

by thenextgennexus Your server? Claim it
answering

Web Scraping MCP Server is answering right now. Last checked 14 min ago. It exposes 3 tools.

Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.

Uptime history 47 days of history
47 days agonow
100.0%
Uptime 24h
91 of 91 checks
3
Tools
read from the server
59 ms
Response time
average over 24h
open, no key
Access
streamable-http

Nothing serious here today

Today is the operative word: we check Web Scraping MCP Server every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.

Three servers free · no card

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 14 min ago.

run in your terminal
claude mcp add web-scraping-mcp-server --transport http https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp
~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "web-scraping-mcp-server": {
      "url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
    }
  }
}
~/.codex/config.toml
[mcp_servers.web-scraping-mcp-server]
url = "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
.cursor/mcp.json
{
  "mcpServers": {
    "web-scraping-mcp-server": {
      "url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
    }
  }
}
.vscode/mcp.json
{
  "mcpServers": {
    "web-scraping-mcp-server": {
      "url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
    }
  }
}

Available tools 3

Read directly from the server with tools/list, grouped by what they act on. If a tool disappears, we record the date.

crawl
crawl_website
Crawl a website and extract its content as structured data. Args: url: Website URL to crawl (e.g. 'https://example.com') max_pages: Max pages to crawl (default 5)
google
google_search
Search Google and return organic results with titles, URLs, snippets. Args: query: Search query (e.g. 'best python libraries 2026') max_results: Max results (default 10)
wikipedia
wikipedia_lookup
Look up a Wikipedia article and return its content. Args: topic: Topic to look up (e.g. 'Artificial intelligence')

Endpoints

URLTransportStateLatencyChecked
https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp streamable-http answering 79 ms 14 min ago

Alternatives to Web Scraping MCP Server

same job, measured the same way
Web Scraper
by ryudi84

MCP server for web scraping: URLs, links, metadata, CSS extraction.

32 installs/wk local only
rasterly
by rasterly-dev

Web toolkit for AI agents: read any URL as Markdown, screenshot, extract data, or crawl a site.

33 installs/wk local only
rasterly
by jhonychain

Web toolkit for AI agents: read any URL as Markdown, screenshot, extract data, or crawl a site.

local only
Brainiall Web
by brainiall

Web search, URL content extraction to Markdown, site mapping, and recursive web crawler.

5 tools answering
Firecrawl
by mcparmory

Scrape, crawl, and extract structured data from websites at scale

239 installs/wk local only
LinkedIn Company Data Scraper - No Cookies
by ranovam

Extract detailed LinkedIn company data without login cookies.

answering
Apify MCP Server
by apify

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

answering
Robotstxt AI
by sharozdawa

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

29 installs/wk local only

Web Scraping MCP Server — questions

Answers built from our own checks of this server.

What can Web Scraping MCP Server do?
It exposes 3 tools, read directly from the server on our last check. Among them: crawl_website, google_search, wikipedia_lookup. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README. How MCP servers expose tools in the first place →
Is Web Scraping MCP Server working right now?
We send a real MCP handshake every 15 minutes. Over the last 24 hours 91 of 91 checks got a reply (100.0%), average response time 59 ms. The bar chart above shows every period we have measured.
How do I connect Web Scraping MCP Server?
Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.
Does Web Scraping MCP Server need an API key?
No. Web Scraping MCP Server completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 3 of them are readable on this page. This is what we observed, not what the docs claim.
How fast is Web Scraping MCP Server?
It answers our handshake in 59 ms on average, which is faster than 87% of all working MCP servers we measure. That puts it in the quick quarter of the ecosystem. The comparison comes from our own checks across the whole registry, every 15 minutes.