Robots Txt AI Crawler Audit MCP Server

by jmshinhwaio.github.jmshinhwa/robots-txt-ai-crawler-auditv0.1.3

Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI

1Node.jsstdioofficial registry

context tax

queued

security

queued

cold start

queued

freshness

Active1d ago

Install Robots Txt AI Crawler Audit MCP server

claude mcp add robots-txt-ai-crawler-audit -- npx -y @readystack/robots-txt-ai-crawler-audit --mcp

{
  "mcpServers": {
    "robots-txt-ai-crawler-audit": {
      "command": "npx",
      "args": [
        "-y",
        "@readystack/robots-txt-ai-crawler-audit",
        "--mcp"
      ]
    }
  }
}

Add to ~/.cursor/mcp.json (global) or .cursor/mcp.json (project).

{
  "mcpServers": {
    "robots-txt-ai-crawler-audit": {
      "command": "npx",
      "args": [
        "-y",
        "@readystack/robots-txt-ai-crawler-audit",
        "--mcp"
      ]
    }
  }
}

Settings → Developer → Edit Config (claude_desktop_config.json), then restart.

{
  "servers": {
    "robots-txt-ai-crawler-audit": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@readystack/robots-txt-ai-crawler-audit",
        "--mcp"
      ]
    }
  }
}

Add to .vscode/mcp.json in your workspace.

{
  "mcpServers": {
    "robots-txt-ai-crawler-audit": {
      "command": "npx",
      "args": [
        "-y",
        "@readystack/robots-txt-ai-crawler-audit",
        "--mcp"
      ]
    }
  }
}

Add to ~/.codeium/windsurf/mcp_config.json.

Freshness

Active — last maintenance signal 1d ago. The newest of the signals below sets the band.

Last commit (default branch)
2026-10-08 · 1d ago · GitHub
Latest release
no data · GitHub
Package published
no data · npm/PyPI
Registry entry updated
2026-09-19 · 21d ago · official registry · v0.1.3

FAQ

›How do I install the Robots Txt AI Crawler Audit MCP server in Claude Code?

Run: claude mcp add robots-txt-ai-crawler-audit -- npx -y @readystack/robots-txt-ai-crawler-audit --mcp. For Cursor, VS Code, Claude Desktop and Windsurf, use the install tabs above.

›Does Robots Txt AI Crawler Audit require an API key?

No required environment variables are declared in its published metadata.

›Can I use Robots Txt AI Crawler Audit as a remote (hosted) MCP server?

No hosted endpoint is published; it runs locally over stdio.

›Is Robots Txt AI Crawler Audit in the official MCP registry?

Yes, as io.github.jmshinhwa/robots-txt-ai-crawler-audit.