Robots Txt Checker MCP Server

by ryuamarines Your server? Claim it

answering

Robots Txt Checker is answering right now. Last checked 13 min ago. It exposes 1 tools.

Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).

Uptime history 8 days of history · worst day 99%

8 days agonow

100.0%

Uptime 24h

92 of 92 checks

Tools

read from the server

619 ms

Response time

average over 24h

open, no key

Access

streamable-http

What changed 1

Every tool that appeared, vanished or quietly changed what it asks for. Recorded since 5 September 2026. No other catalogue keeps this.

5 Sep a tool description was rewritten robots_txt_checker

Robots Txt Checker does not always answer

Over the last week it answered 99.8% of our checks. We check every 15 minutes, so you hear about the next outage within the hour — not from your users.

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 13 min ago.

Claude Code Claude Desktop Codex Cursor VS Code

run in your terminal

claude mcp add robots-txt-checker --transport http https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}

~/.codex/config.toml

[mcp_servers.robots-txt-checker]
url = "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"

.cursor/mcp.json

{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}

.vscode/mcp.json

{
  "mcpServers": {
    "robots-txt-checker": {
      "url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
    }
  }
}

Available tools 1

Read directly from the server with tools/list, grouped by what they act on. If a tool disappears, we record the date.

robots

robots_txt_checker

Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).

Endpoints

URL	Transport	State	Latency	Checked
https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp	streamable-http	answering	696 ms	13 min ago

Alternatives to Robots Txt Checker

same job, measured the same way

App Store Version

by ryuamarines

Checks current App Store version by bundle ID. x402 payment required (testnet USDC).

1 tools answering

Email Extractor

by ryuamarines

Extracts emails from texts. x402 payment required (testnet USDC).

1 tools answering

Text Slugifier

by ryuamarines

Converts text titles into URL-safe slugs and flags duplicates. x402 payment required (testnet USDC).

1 tools answering

JSON Schema Validator

by ryuamarines

Validates JSON against a JSON Schema, lists violations. x402 payment required (testnet USDC).

1 tools answering

Email Validator

by ryuamarines

Validates email syntax and MX records. x402 payment required (testnet USDC).

1 tools answering

URL Normalizer

by ryuamarines

Normalizes and deduplicates a list of URLs. x402 payment required (testnet USDC, Base Sepolia).

1 tools answering

Micropay Test Suite

by netlify-micropay-test-suite

Verify your agent's x402 payment flow for free on Base Sepolia testnet before real money moves.

1 tools answering

Mapsource Geospatial API

by robit-man

OSM queries, raster tiles, elevation. Check status: queries currently use fixtures; x402 is testnet.

13 tools answering

Robots Txt Checker — questions

Answers built from our own checks of this server.

What can Robots Txt Checker do?

It exposes 1 tools, read directly from the server on our last check. Among them: robots_txt_checker. The full list with descriptions is on this page — we take it from the server itself via tools/list, not from a README. How MCP servers expose tools in the first place →

Is Robots Txt Checker working right now?

We send a real MCP handshake every 15 minutes. Over the last 24 hours 92 of 92 checks got a reply (100.0%), average response time 619 ms. The bar chart above shows every period we have measured.

How do I connect Robots Txt Checker?

Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.

Does Robots Txt Checker need an API key?

No. Robots Txt Checker completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. All 1 of them are readable on this page. This is what we observed, not what the docs claim.

How fast is Robots Txt Checker?

It answers our handshake in 619 ms on average, which is faster than 23% of all working MCP servers we measure. That is on the slow side — worth knowing if the tool sits inside an interactive loop. The comparison comes from our own checks across the whole registry, every 15 minutes.