mcpbeat Sign in

Octocrawl MCP Server

answering

Octocrawl is answering right now. Last checked 13 min ago. 531 installs a week from npm. It exposes 2 tools. Last commit 11 Oct 2026.

Web scraper for agents: blocked, empty and wrong pages reported as such, with an Evidence Record.

Installs per day peak 187 · avg 18
a month agotoday
Uptime history 64 hours of history
64 hours agonow
100.0%
Uptime 24h
91 of 91 checks
2
Tools
read from the server
494 ms
Response time
average over 24h
531
Installs / week
npm and PyPI

What changed 7

Every tool that appeared, vanished or quietly changed what it asks for. Recorded since 9 October 2026. No other catalogue keeps this.

11 Oct a tool disappeared scrape_product
11 Oct a tool changed the parameters it asks for scrape
10 Oct a tool changed version2 times that day
9 Oct a tool changed version2 times that day
9 Oct a tool changed the parameters it asks for scrape
and 2 more, back to 9 October 2026

What the code does

We read the source, 1 h ago · tools taken from the live server · rules ccca72095570

Evidence

Things with no honest explanation: a promise that contradicts the code, code that runs at install time while hiding what it does, data leaving the machine.

    const response = await this.fetcher('http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token', {
Capabilities

What this server is able to do. For an MCP server this is often the job itself — a terminal server runs commands because that is what it is for. Listed so you know what you are plugging in, not as an accusation.

    spill = pool(sx(from), sy(top - 2 * ART_CELL), sx(to), sy(top + SPILL_DROP + ART_CELL), cell,
Sends data to a third-party sink research/gate3-https-delivery.generated.json:6
  "httpsEndpoint": "https://happens-task-production-treating.trycloudflare.com/webhook",
Reads files and sends them to the network packages/mcp/src/hostedApi.ts:194
    try { entries = await readdir(base) } catch { continue }

Is this your server and something here is wrong? Tell us — corrections are free and do not require a plan.

We found things in this code

Code changes quietly between releases, and nobody reads the diff of a dependency. We do, on every release — watch Octocrawl and you get told the day something new turns up.

Three servers free · no card

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 13 min ago.

run in your terminal
claude mcp add octocrawl --transport http https://mcp.octocrawl.dev/mcp
~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "octocrawl": {
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}
~/.codex/config.toml
[mcp_servers.octocrawl]
url = "https://mcp.octocrawl.dev/mcp"
.cursor/mcp.json
{
  "mcpServers": {
    "octocrawl": {
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}
.vscode/mcp.json
{
  "mcpServers": {
    "octocrawl": {
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

This one needs environment variables set before it will start: W2L_API_URL (The Octocrawl API the server calls: npx octocrawl serve on this computer (default http://127.0.0.1:8787), or https://api.octocrawl.dev), W2L_API_TOKEN (A bearer token for a hosted or self-hosted API; none for a local one). The author declared them in the registry entry; get the values from the project itself.

Available tools 2

Read directly from the server with tools/list, grouped by what they act on. If a tool disappears, we record the date.

map
map
List a site's URLs without fetching each page: the start URL, the links on its page (read on the http lane alone; no browser) and the entries of the sitemaps the site declares (robots.txt Sitemap: lines, else /sitemap.xml), inside one deadline. Every URL is in the crawl's scope (the start host and its www twin, the start URL's path subtree, assets left out, similar URLs folded) and allowed by its host's robots.txt unless ignoreRobotsTxt is set; what was left out is counted. A title is never fetched: the start page's own, an anchor's text or a sitemap's news title. At the deadline the answer is what was found, status partial (failed when nothing), stoppedBy timeout. Compact by default ({ id, status, stoppedBy, links: [{ url, title?, description?, robots? }], warning?, agentHints?, counts }; robots only on a link robots.txt keeps out, under ignoreRobotsTxt); debug=true returns the full map with each link's evidence (via, sitemapFile, lastmod, robots), the sources read and the refusals. One page body is read at most: a site without a sitemap maps only its start page's links; crawl reads further pages.
scrape
scrape
Fetch one URL through the Octocrawl coverage ladder. Compact by default; set debug=true for the full audit. The result's warnings name what its content cannot vouch for: robots_overridden (robots.txt disallows the URL; a local server fetched it because you named it), or client_rendered_suspected when the HTTP page looks like a shell its scripts fill in and the browser rung found nothing better. Its agentHints, when present, say what to change next time (a login wall, a robots.txt rule, a gate, a wait). metadata.scrapeId names the call's record for get_scrape.

Tools removed

Tools this server used to expose. Anything built against them stopped working on the day they went.

scrape_product
removed 11 Oct 2026

Endpoints

URLTransportStateLatencyChecked
https://mcp.octocrawl.dev/mcp streamable-http answering 472 ms 13 min ago

Alternatives to Octocrawl

same job, measured the same way
Before Users Do
by bhuman-ai

Browser-backed QA with evidence and fix-ready reports for coding agents.

answering
Glint Render — web rendering for AI agents
by zzzjy765

Screenshots, PDFs, OG images, monitoring and evidence for AI agents. Glint Render by Moyu.

14 installs/wk local only
Reposcout
by rakesh1002

Discover, enrich, and rank GitHub repos for an objective with weighted, evidence-based scoring.

27 installs/wk local only
Paper Lantern
by paperlantern

Research intelligence for AI coding agents. 2M+ CS papers with evidence and tradeoffs.

answering
A
Agent Help Network
by agenthelpnetwork

Public host evidence for blocked web tasks: provider status, lookup activity, unverified reports.

2 tools answering
Sinter
by shellfu

Persistent, evidence-backed code graph for coding agents.

744 installs/wk local only
Agent Residue Evidence
by fantasyce

Report local task-scoped test and build residue without cleaning resources.

local only
Coador
by automaticqa

Evidence-backed Android repository knowledge for humans and AI agents

70 installs/wk local only

Octocrawl — questions

Answers built from our own checks of this server.

What can Octocrawl do?
It exposes 2 tools, read directly from the server on our last check. Among them: map, scrape. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README. How MCP servers expose tools in the first place →
Is Octocrawl working right now?
We send a real MCP handshake every 15 minutes. Over the last 24 hours 91 of 91 checks got a reply (100.0%), average response time 494 ms. The bar chart above shows every period we have measured.
Did Octocrawl ever remove tools?
Yes. scrape_product is no longer exposed — we recorded the date each one disappeared. A tool vanishing usually means a breaking change for anything that depended on it.
How do I connect Octocrawl?
Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.
Does Octocrawl need an API key?
No. Octocrawl completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 2 of them are readable on this page. This is what we observed, not what the docs claim.
How fast is Octocrawl?
It answers our handshake in 494 ms on average, which is faster than 26% of all working MCP servers we measure. The comparison comes from our own checks across the whole registry, every 15 minutes.
How many people use Octocrawl?
The npm package @octocrawl/mcp was installed 531 times in the last week. We show installs rather than GitHub stars on purpose: a star is a bookmark, an install is someone actually running it.
Is Octocrawl open source?
Yes — it is published under the AGPL-3.0 licence, written in TypeScript and 0 stars on GitHub. The source link is on this page, so you can read exactly what it does with your data before you connect it.