Robots Txt Checker is answering right now. Last checked 10 min ago. It exposes 1 tools.
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
What changed 1
Every tool that appeared, vanished or quietly changed what it asks for. Recorded since 5 September 2026. No other catalogue keeps this.
Nothing serious here today
Today is the operative word: we check Robots Txt Checker every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.
Connect this server
Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 10 min ago.
claude mcp add robots-txt-checker --transport http https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcphttps://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp[mcp_servers.robots-txt-checker]
url = "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"{
"mcpServers": {
"robots-txt-checker": {
"url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
}
}
}{
"servers": {
"robots-txt-checker": {
"type": "http",
"url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
}
}
}This endpoint answered with an authorization challenge. The server is running, but it did not say what kind of credentials it expects.
Available tools 1
Read directly from the server with tools/list, grouped by what they act on.
If a tool disappears, we record the date.
robots_txt_checker
Endpoints
| URL | Transport | State | Latency | Checked |
|---|---|---|---|---|
| https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp | streamable-http | needs auth | 142 ms | 10 min ago |
Alternatives to Robots Txt Checker
same job, measured the same wayChecks current App Store version by bundle ID. x402 payment required (testnet USDC).
Extracts emails from texts. x402 payment required (testnet USDC).
Converts text titles into URL-safe slugs and flags duplicates. x402 payment required (testnet USDC).
Validates JSON against a JSON Schema, lists violations. x402 payment required (testnet USDC).
Validates email syntax and MX records. x402 payment required (testnet USDC).
Normalizes and deduplicates a list of URLs. x402 payment required (testnet USDC, Base Sepolia).
Test any x402 paid URL without paying: 402, PAYMENT-REQUIRED, USDC, amounts, payTo, EIP-712, CORS.
Pay-per-call x402 crypto research and Base address/approval risk checks. Testnet by default.
Robots Txt Checker — questions
Answers built from our own checks of this server.