mcpbeat

TheCrawler MCP Server

io.github.manchittlab/thecrawler
local only

TheCrawler runs on your own machine — the client starts it, so there is no endpoint to ping. 26 installs a week from npm. Last commit 2 Jul 2026.

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Installs per day peak 42 · avg 10 · -50% w/w
a month agotoday
26
Installs / week
npm · thecrawler
1
Stars
1 open issues
2 Jul 2026
Last commit
0 releases in 90 days
AGPL-3.0
License
TypeScript

Connect this server

This server runs on your own machine — install it with the package manager and the client starts it for you. Package name taken from the official registry entry.

run in your terminal
claude mcp add thecrawler -- npx -y thecrawler
~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "thecrawler": {
      "args": [
        "-y",
        "thecrawler"
      ],
      "command": "npx"
    }
  }
}
~/.codex/config.toml
[mcp_servers.thecrawler]
command = "npx"
args = ["-y", "thecrawler"]
.cursor/mcp.json
{
  "mcpServers": {
    "thecrawler": {
      "args": [
        "-y",
        "thecrawler"
      ],
      "command": "npx"
    }
  }
}
.vscode/mcp.json
{
  "mcpServers": {
    "thecrawler": {
      "args": [
        "-y",
        "thecrawler"
      ],
      "command": "npx"
    }
  }
}

TheCrawler — questions

Answers built from our own checks of this server.

Why is there no uptime for TheCrawler?
TheCrawler runs on your own machine over stdio — there is no network address to reach, so uptime cannot be measured for it by anyone. What can be measured is adoption: the npm package thecrawler was installed 26 times last week.
How do I connect TheCrawler?
Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It runs locally, so the command pulls thecrawler straight from npm; nothing to host, nothing to sign up for.
How many people use TheCrawler?
The npm package thecrawler was installed 26 times in the last week. Week over week that is -50%. We show installs rather than GitHub stars on purpose: a star is a bookmark, an install is someone actually running it.
Is TheCrawler open source?
Yes — it is published under the AGPL-3.0 licence, written in TypeScript, 1 stars on GitHub and 1 open issue. The source link is on this page, so you can read exactly what it does with your data before you connect it.