Robotstxt AI MCP Server

by sharozdawa Your server? Claim it

local only

Robotstxt AI runs on your own machine — the client starts it, so there is no endpoint to ping. 29 installs a week from npm. Last commit 22 Mar 2026.

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

Installs per day peak 22 · avg 6 · -9% w/w

a month agotoday

Installs / week

npm · robotstxt-ai-mcp

Stars

0 open issues

22 Mar 2026

Last commit

0 releases in 90 days

MIT

License

TypeScript

What the code does

We read the source, 18 h ago · rules 3dff92dd89df

Capabilities

What this server is able to do. For an MCP server this is often the job itself — a terminal server runs commands because that is what it is for. Listed so you know what you are plugging in, not as an accusation.

File ships in the package but is absent from the source [пакет] src/index.ts:1

этот файл ставится пользователю, но в репозитории его нет

Is this your server and something here is wrong? Tell us — corrections are free and do not require a plan.

This code can reach further than it looks

We found places where it runs commands, builds paths or queries from values it is given. None of that is a flaw by itself — it becomes one when the code changes, and code changes quietly between releases. We re-read it on every one.

Connect this server

This server runs on your own machine — install it with the package manager and the client starts it for you. Package name taken from the official registry entry.

Claude Code Claude Desktop Codex Cursor VS Code

run in your terminal

claude mcp add robotstxt-ai -- npx -y robotstxt-ai-mcp

~/Library/Application Support/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "robotstxt-ai": {
      "args": [
        "-y",
        "robotstxt-ai-mcp"
      ],
      "command": "npx"
    }
  }
}

~/.codex/config.toml

[mcp_servers.robotstxt-ai]
command = "npx"
args = ["-y", "robotstxt-ai-mcp"]

.cursor/mcp.json

{
  "mcpServers": {
    "robotstxt-ai": {
      "args": [
        "-y",
        "robotstxt-ai-mcp"
      ],
      "command": "npx"
    }
  }
}

.vscode/mcp.json

{
  "mcpServers": {
    "robotstxt-ai": {
      "args": [
        "-y",
        "robotstxt-ai-mcp"
      ],
      "command": "npx"
    }
  }
}

Alternatives to Robotstxt AI

same job, measured the same way

PageCrawl

by pagecrawl

Monitor web pages for changes with AI-powered change detection, diffs, and alerts.

answering

llms.txt Checker - AI Readiness Audit

by apifymcpfactory-droid

Checks llms.txt, AI crawler access in robots.txt, and sitemap - with a 0-100 AI readiness score.

answering

Scrape-LE

by nolindnaidoo

Analyse robots.txt content and report whether a path may be crawled.

41 installs/wk local only

Momentic

by momenticmarketing

Free SEO, GEO, and AEO audits: analyze any page or domain, AI-crawler access, agent readiness.

11 tools answering

AI Visibility Checker

by krisdiallo

AI visibility checker: GPTBot vs OAI-SearchBot, robots.txt audit, raw-HTML schema. 165 crawlers.

local only

Robots Txt AI Crawler Audit

by jmshinhwa

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

310 installs/wk local only

ActableSite AI Crawler Monitor

by actablesite

Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

5 tools answering

MaxAEO AI Visibility MCP

by maxaeo

Local-first MCP server for AI visibility, GEO/AEO, llms.txt, and AI crawler readiness audits.

72 installs/wk local only

Robotstxt AI — questions

Answers built from our own checks of this server.

Why is there no uptime for Robotstxt AI?

Robotstxt AI runs on your own machine over stdio — there is no network address to reach, so uptime cannot be measured for it by anyone. What can be measured is adoption: the npm package robotstxt-ai-mcp was installed 29 times last week.

How do I connect Robotstxt AI?

Copy the ready config from this page — we generate it for Claude Code, Claude Desktop, Codex, Cursor and VS Code, each with the file path that client actually reads. It runs locally, so the command pulls robotstxt-ai-mcp straight from npm; nothing to host, nothing to sign up for.

How many people use Robotstxt AI?

The npm package robotstxt-ai-mcp was installed 29 times in the last week. Week over week that is -9%. We show installs rather than GitHub stars on purpose: a star is a bookmark, an install is someone actually running it.

Is Robotstxt AI open source?

Yes — it is published under the MIT licence, written in TypeScript and 1 stars on GitHub. The source link is on this page, so you can read exactly what it does with your data before you connect it.