mcpbeat Sign in

Web Crawler MCP by InsightfulPipe MCP Server

by insightfulpipe Your server? Claim it
answering

Web Crawler MCP by InsightfulPipe is answering right now. Last checked 2 min ago. Last commit 10 Oct 2026.

Hosted Web Crawler MCP server for Claude, ChatGPT and Cursor, with each action switched per account.

Uptime history 6 days of history · worst day 99%
6 days agonow
100.0%
Uptime 24h
92 of 92 checks
—
Tools
hidden behind auth
94 ms
Response time
average over 24h
0
Stars
last commit 10 Oct 2026

Web Crawler MCP by InsightfulPipe missed one check this week

Everything else answered, so this is steady rather than shaky. We check every 15 minutes, which is how a one-off gets told apart from the start of a pattern, and how you hear about the next one within the hour instead of from your users.

Three servers free · no card

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 2 min ago.

run in your terminal
claude mcp add crawler --transport http https://crawler.insightfulmcp.com/
Settings → Connectors → Add custom connector, then paste this address
https://crawler.insightfulmcp.com/
~/.codex/config.toml
[mcp_servers.crawler]
url = "https://crawler.insightfulmcp.com/"
.cursor/mcp.json
{
  "mcpServers": {
    "crawler": {
      "url": "https://crawler.insightfulmcp.com/"
    }
  }
}
.vscode/mcp.json
{
  "servers": {
    "crawler": {
      "type": "http",
      "url": "https://crawler.insightfulmcp.com/"
    }
  }
}

This endpoint answered with an authorization challenge. The server is running, and it signs you in through your browser: there is no API key to paste.

Endpoints

URLTransportStateLatencyChecked
https://crawler.insightfulmcp.com/ streamable-http sign-in 86 ms 2 min ago

Alternatives to Web Crawler MCP by InsightfulPipe

same job, measured the same way
Enrichment Crawlers MCP by InsightfulPipe
by insightfulpipe

Hosted Enrichment Crawlers MCP server, with each action switched on or off per account.

answering
Evocrawl MCP Server
by superaihuman

MCP server for Evocrawl — search, scrape, and interact with the web.

15 installs/wk local only
O
Olostep MCP Server
by olostep

Search, scrape, and crawl the web for AI agents. Batch scraping and answers with citations.

194 installs/wk answering
DeepSFetch
by ermultimidia

Web scraping, extraction and search for AI agents. Hosted MCP with workspace API keys.

answering
mcp-my-site
by pournasserian

Search and fetch tools for one website, served from an index you build with the mcpmysite crawler.

2 tools answering
SEO Perception
by seoperception

Search Console, crawl and SEO to-dos, read-only inside Cursor and Claude.

answering
SearXNG MCP Server
by bumbarasch

Self-hosted SearXNG metasearch for MCP clients: 13 tools, citation refs, deep search and crawling.

309 installs/wk local only
Firecrawl MCP Server
by firecrawl

Firecrawl, the context API for AI agents: search, scrape, interact with the web, plus paper search.

128 628 installs/wk 3 tools answering

Web Crawler MCP by InsightfulPipe — questions

Answers built from our own checks of this server.

Why is there no tool list for Web Crawler MCP by InsightfulPipe?
The server answered our handshake with an authorization challenge, so it is running — but it will not describe its tools to an anonymous client. To see them you need to connect with your own credentials. We record it as alive, not as broken: 100.0% of checks in the last 24 hours got a reply.
Is Web Crawler MCP by InsightfulPipe working right now?
We send a real MCP handshake every 15 minutes. Over the last 24 hours 92 of 92 checks got a reply (100.0%), average response time 94 ms. The bar chart above shows every period we have measured.
How do I connect Web Crawler MCP by InsightfulPipe?
Copy the ready config from this page. We generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each in the form that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address. You sign in through your browser once and the client keeps the session: no key to copy, and it refuses anonymous clients.
How do you sign in to Web Crawler MCP by InsightfulPipe?
No API key is involved. Web Crawler MCP by InsightfulPipe answers our knock with an OAuth challenge, so you authorise it once in your browser and the client keeps the session. That is also why we see no tool list: it will not describe itself to an anonymous client, and that is the server working as intended, not a fault.
How fast is Web Crawler MCP by InsightfulPipe?
It answers our handshake in 94 ms on average, which is faster than 97% of all working MCP servers we measure. That puts it in the quick quarter of the ecosystem. The comparison comes from our own checks across the whole registry, every 15 minutes.
Is Web Crawler MCP by InsightfulPipe open source?
The code is open source: it is published under the MIT licence and 0 stars on GitHub. The licence covers the code in the repository, not the hosted service at crawler.insightfulmcp.com, which runs on its operator's own terms.