We read what crawlers read
One request for robots.txt and one for llms.txt, read with the same rules the crawlers follow.
Type your trade and city: we ask Gemini with Google Search who it recommends, which sources it read, and whether your business appears. Free, no sign-up.
Unlike asking Gemini yourself: we ask Gemini in 2 ways and an OpenAI model too (the answer changes every time), open the pages they relied on and tell you whether they name you or your competitors.
We open the top 3 to see who they name. That is where to be listed, with a complete profile and real reviews.
It is a mini audit: your 0-100 AI visibility score against the business AI recommends most, what Gemini and an OpenAI model answer, a check of your website (which AI crawlers it blocks, title, phone, business schema), the pages where your competitors appear and you do not, and ready-to-paste text and code. It reaches your inbox in about an hour.
Free technical check: whether ChatGPT, Claude, Gemini and 17 other crawlers can get into your site and how much text they can read.
Free, no sign-up. We fetch three public files once each (robots.txt, llms.txt and your home page). No domain handy? .
Built your app with Lovable, Bolt, Base44 or v0? The check also shows how much of your home page AI can read. Why AI-built apps can look empty to ChatGPT.
| Lamp | Crawler | Operator and product | Used for | robots.txt says |
|---|
We audit up to 10 pages for llms.txt, the sitemap, noindex and snippet limits, text that loads without JavaScript, structured data, titles, canonicals and language tags. You get a 0-100 score with every weight shown and fixes ordered by points at stake, on a page you can share or save as PDF.
One request for robots.txt and one for llms.txt, read with the same rules the crawlers follow.
Each of the 20 AI crawlers gets a lamp: lit if it can read your homepage, dark if a rule blocks it, with the exact line that decides it.
The full report audits up to 10 pages and orders every fix by the points at stake, so the first change you make is the one that matters most.
Example for a dental clinic in Monterrey (fictional business), built on real data from the Atalaya Index: where AI looks, how the business shows up today, and ready-to-use profile text.
Missing hours, 2 of 5 treatments and the neighbourhood. Complete it with the same name, address and phone as on Google Maps.
Create it and ask real patients for reviews with a direct link. Never paid reviews.
"Dental clinic in Del Valle, San Pedro Garza García. Cleanings, fillings, orthodontics and root canals. Monday to Saturday, 9 am to 7 pm. Appointments and same-day emergencies."
Every point comes from something you can check yourself. The free check covers the first three rows, 32 points. Training crawlers are left out on purpose: letting models train on your pages is a policy choice, and the score should not push you either way.
A score is a checklist. It does not predict rankings or promise that any AI product will cite you.
| What we check | Points |
|---|---|
| AI search crawlers allowed in robots.txt | 20 |
| Assistants allowed to fetch a page for a user | 8 |
| robots.txt reachable and well formed | 4 |
| Text present in the HTML the server sends | 14 |
| No noindex, nosnippet or tiny max-snippet | 12 |
| Structured data | 12 |
| Title, description, canonical, language, Open Graph, H1 | 10 |
| llms.txt present and in llmstxt.org format | 8 |
| Sitemap found and declared in robots.txt | 6 |
| HTTP status, HTTPS and response time | 6 |
We ran this check on the most-linked domains in each country. Many sites that opt out of AI training also shut out AI search crawlers, which takes them out of AI answers. Here is how to avoid it.
Samples: top 300 .mx and top 300 .ca domains, and for the US a proxy of the top 200 .com, 50 .gov and 50 .edu, from the Majestic Million (CC BY 3.0) of 2026-10-20. 751 of 900 gave a usable answer. Only robots.txt and llms.txt were read, so blocks made by a CDN or firewall are not counted.
See who AI recommends for your trade and city, which sources it read, and whether you appear.
Ask the AIWe check how your business shows up on those sources and send you ready-to-use profile text and 3 steps.
Request a reviewWe do it with you: profiles on the sources AI reads, real reviews, Google Business Profile and social videos, measured before and after.
Write to usIf the pilot worked, we continue every month and show you what changed. No lock-in; you see the price after the pilot.
See pricingAgency? Check up to 20 client sites free and hand over the result under your brand: Tools for agencies. Building tools? The technical check is also an API on Apify and an MCP server. Apify.
For local businesses, AI builds its recommendations from directories, booking platforms, guides and local media. What works: a complete profile with real reviews on those sources, a website that says in plain text what you do and where, and the same details everywhere. More
Ask the question a customer would ask, with your trade and city, in an assistant with web search. Our free tool does it with Gemini and Google Search and shows the sources it read. More
Yes, the check on this page: type your domain and see in seconds whether the crawlers of ChatGPT, Claude, Perplexity, Gemini and 16 more can read your site, with the robots.txt line that decides each one. Free, no sign-up. More
No. OpenAI, Anthropic, Google and Apple use separate crawlers for training and for search. You can block GPTBot, ClaudeBot or Google-Extended and still allow OAI-SearchBot, Claude-SearchBot and Googlebot. Here is a robots.txt you can copy that blocks training and keeps search.
Nobody has shown that it does. llms.txt is a community proposal, and no AI vendor has confirmed using it for ranking or citations. It is worth 8 of 100 points in our score and we list it as a low-severity fix.
Some vendors say their user-initiated fetchers (ChatGPT-User, Perplexity-User and others) may ignore robots.txt. We report what your file says and add the vendor's note next to that crawler.
Blocks made by a CDN, firewall or bot-protection service, which happen before robots.txt is read. If AI crawlers are allowed in robots.txt but still cannot reach you, look at those settings.
No, on purpose. Most AI crawlers read the HTML your server sends, so that is what we measure. A page that needs JavaScript to show its text loses points for it.
It identifies itself as AtalayaBot, reads robots.txt first, skips anything it disallows, and requests one page at a time from each site.
Atalaya AI is a small software brand operated from Mexico. We use AI to help write code and drafts, and a person owns the project and approves what goes public. Vigía, the guide in the corner, is a scripted helper: it does not send anything you type anywhere. We are not affiliated with OpenAI, Anthropic, Perplexity, Google, Apple, Meta, Amazon, ByteDance, DuckDuckGo or Common Crawl.
It takes one domain and about two seconds.