Atalaya AI
ENES
Check my site

For developers and AI agents ยท free, no account, no key

AI crawler check as an MCP server

Ask your coding assistant "can ChatGPT search read my site?" and get the answer from the live robots.txt: which of 20 AI crawlers may read the site, the exact rule behind each verdict, whether it has llms.txt and a sitemap, and where the setting lives on its platform.

Endpoint

https://atalaya-ai.com/mcp

Remote server, Streamable HTTP transport, JSON responses, no session. Listed in the official MCP Registry as com.atalaya-ai/ai-crawler-check.

Add it to your client

Claude Code

claude mcp add --transport http atalaya https://atalaya-ai.com/mcp

Claude Desktop, Cursor, VS Code and other clients that read a JSON config:

{
  "mcpServers": {
    "atalaya": { "type": "http", "url": "https://atalaya-ai.com/mcp" }
  }
}

Some clients call the key servers or the type streamable-http; check your client's MCP settings page.

Tools

  • check_ai_crawler_access(domain, lang?): robots.txt verdict for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended and the rest of our list, each with the deciding rule; llms.txt and sitemap presence; platform and CDN hint. Text answer plus the same structured JSON the website uses. lang is en or es.
  • list_ai_crawlers(): the crawler tokens we check, by vendor and purpose (training, search index, fetch on a user's request), for writing a robots.txt that keeps AI search and opts out of training.

Both tools are read-only. They fetch only public files of the site you name: /robots.txt, /llms.txt and the home page.

Limits, honestly

robots.txt shows what a site asks crawlers to do. A CDN, firewall or bot-protection rule acts before that and is not visible from outside; our Cloudflare guide explains where to look. Results are cached for an hour per domain. Whether AI tools can read the text of your pages (server-rendered HTML versus content that appears only after JavaScript runs), structured data and metadata are measured in the full report, not by this server.

Privacy

We count how many checks run per day by channel, as totals. We do not store the domains you check or anything about your client. See the privacy notice.

Written with AI. If you spot an error, write to hola@atalaya-ai.com.