AI Crawler Policy MCP Server

by lazymac2x Your server? Claim it

answering

AI Crawler Policy is answering right now. Last checked 9 min ago. Last commit 17 May 2026.

Cloudflare Workers MCP server: ai-crawler-policy

Uptime history 91 days of history · worst day 95%

91 days agonow

100.0%

Uptime 24h

92 of 92 checks

—

Tools

hidden behind auth

183 ms

Response time

average over 24h

Stars

last commit 17 May 2026

Nothing serious here today

Today is the operative word: we check AI Crawler Policy every 15 minutes and re-read its code on every release. Watch it and you find out the day that stops being true.

Connect this server

Endpoint below is the one we actually reach during checks — not the one copied from a README. Last verified 9 min ago.

Claude Code Claude Desktop Codex Cursor VS Code

run in your terminal

claude mcp add ai-crawler-policy --transport http https://api.lazy-mac.com/ai-crawler-policy/mcp

Settings → Connectors → Add custom connector, then paste this address

https://api.lazy-mac.com/ai-crawler-policy/mcp

~/.codex/config.toml

[mcp_servers.ai-crawler-policy]
url = "https://api.lazy-mac.com/ai-crawler-policy/mcp"

.cursor/mcp.json

{
  "mcpServers": {
    "ai-crawler-policy": {
      "url": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
    }
  }
}

.vscode/mcp.json

{
  "servers": {
    "ai-crawler-policy": {
      "type": "http",
      "url": "https://api.lazy-mac.com/ai-crawler-policy/mcp"
    }
  }
}

Endpoints

URL	Transport	State	Latency	Checked
https://api.lazy-mac.com/ai-crawler-policy/mcp	streamable-http	answering	190 ms	9 min ago

Alternatives to AI Crawler Policy

same job, measured the same way

Scrapling MCP Server

by d4vinci

Web scraping with stealth HTTP, real browsers, and Cloudflare bypass. CSS selectors supported.

334 654 installs/wk local only

Novada MCP

by novadalabs

One MCP server for all web data. Scrape, search, crawl, and AI research in one install.

816 installs/wk answering

Enrichment Crawlers MCP by InsightfulPipe

by insightfulpipe

Hosted Enrichment Crawlers MCP server, with each action switched on or off per account.

answering

Domduck

by domduck

Domain reputation over time: Domain Rating, Tranco rank, TLS, and AI crawler policy.

3 tools answering

Rattlesnakesbymail

by rattlesnakesbymail

Dated, sourced claims about AI crawlers and agents, offered as MCP tools.

6 tools answering

Cloudflare Solver

by jojapi

Cloudflare Solver: Scraping API designed to bypass Cloudflare protection.

1 tools answering

MaxAEO AI Visibility MCP

by maxaeo

Local-first MCP server for AI visibility, GEO/AEO, llms.txt, and AI crawler readiness audits.

81 installs/wk local only

ActableSite AI Crawler Monitor

by actablesite

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

5 tools answering

AI Crawler Policy — questions

Answers built from our own checks of this server.

Is AI Crawler Policy working right now?

We send a real MCP handshake every 15 minutes. Over the last 24 hours 92 of 92 checks got a reply (100.0%), average response time 183 ms. The bar chart above shows every period we have measured.

How do I connect AI Crawler Policy?

Copy the ready config from this page. We generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each in the form that client actually reads. It is a remote server, so there is nothing to install — the client connects to the address.

Does AI Crawler Policy need an API key?

No. AI Crawler Policy completed a full MCP handshake with us as an anonymous client and listed its tools without asking for anything. This is what we observed, not what the docs claim.

How fast is AI Crawler Policy?

It answers our handshake in 183 ms on average, which is faster than 74% of all working MCP servers we measure. The comparison comes from our own checks across the whole registry, every 15 minutes.

Is AI Crawler Policy open source?

We cannot say either way: 0 stars on GitHub, but we could not determine the licence, and without one the code is not open source by default.