Web Scraping MCP Server

by thenextgennexus Your server? Claim it

answering

Web Scraping MCP Server is answering right now. Last checked 5 min ago. It exposes 3 tools.

Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.

Uptime history 47 days of history

47 days agonow

100.0%

Uptime 24h

92 of 92 checks

Tools

read from the server

59 ms

Response time

average over 24h

open, no key

Access

streamable-http

Nothing serious here today

Today is the operative word: we check Web Scraping MCP Server every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 5 min ago.

Claude Code Claude Desktop Codex Cursor VS Code

run in your terminal

claude mcp add web-scraping-mcp-server --transport http https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "web-scraping-mcp-server": {
      "url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
    }
  }
}

~/.codex/config.toml

[mcp_servers.web-scraping-mcp-server]
url = "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"

.cursor/mcp.json

{
  "mcpServers": {
    "web-scraping-mcp-server": {
      "url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
    }
  }
}

.vscode/mcp.json

{
  "mcpServers": {
    "web-scraping-mcp-server": {
      "url": "https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp"
    }
  }
}

Available tools 3

Read directly from the server with tools/list, grouped by what they act on. If a tool disappears, we record the date.

crawl

crawl_website

Crawl a website and extract its content as structured data. Args: url: Website URL to crawl (e.g. 'https://example.com') max_pages: Max pages to crawl (default 5)

google

google_search

Search Google and return organic results with titles, URLs, snippets. Args: query: Search query (e.g. 'best python libraries 2026') max_results: Max results (default 10)

wikipedia

wikipedia_lookup

Look up a Wikipedia article and return its content. Args: topic: Topic to look up (e.g. 'Artificial intelligence')

Endpoints

URL	Transport	State	Latency	Checked
https://nexgendata-mcp-proxy.steve-corbeil.workers.dev/web-scraping-mcp-server/mcp	streamable-http	answering	50 ms	5 min ago

Alternatives to Web Scraping MCP Server

same job, measured the same way

Web Scraper

by ryudi84

MCP server for web scraping: URLs, links, metadata, CSS extraction.

32 installs/wk local only

rasterly

by rasterly-dev

Web toolkit for AI agents: read any URL as Markdown, screenshot, extract data, or crawl a site.

33 installs/wk local only

rasterly

by jhonychain

Web toolkit for AI agents: read any URL as Markdown, screenshot, extract data, or crawl a site.

local only

Brainiall Web

by brainiall

Web search, URL content extraction to Markdown, site mapping, and recursive web crawler.

5 tools answering

Firecrawl

by mcparmory

Scrape, crawl, and extract structured data from websites at scale

239 installs/wk local only

LinkedIn Company Data Scraper - No Cookies

by ranovam

Extract detailed LinkedIn company data without login cookies.

answering

Apify MCP Server

by apify

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

answering

Robotstxt AI

by sharozdawa

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

29 installs/wk local only

Web Scraping MCP Server — questions

Answers built from our own checks of this server.

What can Web Scraping MCP Server do?

It exposes 3 tools, read directly from the server on our last check. Among them: crawl_website, google_search, wikipedia_lookup. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README. How MCP servers expose tools in the first place →

Is Web Scraping MCP Server working right now?

We send a real MCP handshake every 15 minutes. Over the last 24 hours 92 of 92 checks got a reply (100.0%), average response time 59 ms. The bar chart above shows every period we have measured.

How do I connect Web Scraping MCP Server?

Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.

Does Web Scraping MCP Server need an API key?

No. Web Scraping MCP Server completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 3 of them are readable on this page. This is what we observed, not what the docs claim.

How fast is Web Scraping MCP Server?

It answers our handshake in 59 ms on average, which is faster than 87% of all working MCP servers we measure. That puts it in the quick quarter of the ecosystem. The comparison comes from our own checks across the whole registry, every 15 minutes.