Guide, updated 2026-10-22

llms.txt: the format, a template to copy and the mistakes to avoid

llms.txt is a Markdown file at /llms.txt that gives AI tools a short, curated list of the pages on your site. It is a community proposal and it is optional. The crawler documentation we checked describes robots.txt as the control, and we have not found an AI search vendor that says it uses llms.txt to rank or cite pages.

What llms.txt is

The proposal at llmstxt.org was first written in 2024. It asks sites to publish a Markdown file at /llms.txt, or at a subpath such as /docs/llms.txt that covers the URLs under it, with a short description of the site and links to the pages that matter. A sitemap lists every page. llms.txt is a curated overview, and the proposal expects it to be read on demand, when an agent needs information while helping a user.

It is not an IETF or W3C standard, and nothing in the proposal obliges an AI company to read it. Our auditor only checks whether your file follows the proposed format.

The format

The proposal defines the parts of the file in this order. Only the first one is required.

An H1 with the name of the site or project, such as # Example Store. This is the only required part.
A blockquote with a short summary that holds the key facts needed to understand the rest of the file.
Optional paragraphs or lists with more detail. Any Markdown works here except headings.
Zero or more H2 sections. Each one is a list of links, one per line, written as - [name](url): notes. The link is required. The colon and the notes are optional.

One of the H2 sections can be named ## Optional. By convention it holds secondary links that an agent can skip when it needs a shorter context. A byte-order mark at the very start of the file is allowed.

A template to copy

Replace the placeholders and keep the order. Use full URLs so every link still works when a tool reads the file on its own.

# Example Store

> Example Store sells handmade ceramics online and ships to Mexico, the US and Canada. This file lists the pages that explain what we sell, shipping and returns.

Prices are in MXN. Every page exists in English and Spanish.

## Products

- [Mugs](https://example.com/en/mugs/): hand-thrown stoneware mugs, 350 ml
- [Plates](https://example.com/en/plates/): dinner and side plates

## Policies

- [Shipping](https://example.com/en/shipping/): carriers, delivery times and costs
- [Returns](https://example.com/en/returns/): 30-day return window

## Optional

- [About us](https://example.com/en/about/): who makes the ceramics
- [Blog](https://example.com/en/blog/): care guides and studio news

Save it as plain text at https://yoursite.com/llms.txt, then request it with curl -i https://yoursite.com/llms.txt. You want status 200 and a Content-Type that is not text/html.

A worked example: our own file

Our file at atalaya-ai.com/llms.txt follows the same order. This is an excerpt, with the middle cut out:

# Atalaya

> Atalaya checks whether AI search engines and assistants can read a website. A free check shows, for 20 AI crawlers (...), whether the site's robots.txt allows or blocks them and which rule decides it. ...

## Guides

- [Block AI training crawlers and stay in AI search](https://atalaya-ai.com/guides/block-ai-training/): copy-paste robots.txt that blocks 8 training crawlers and keeps 7 AI search crawlers
- [llms.txt guide: format, template and common mistakes](https://atalaya-ai.com/guides/llms-txt/): ...

## Optional

- [Terms of use](https://atalaya-ai.com/terms)
- [Privacy notice](https://atalaya-ai.com/privacy)

Under the summary, a short paragraph says what the score is and is not: a transparent checklist, not a ranking prediction or a promise of being cited by any AI product. The sections are Tools, Guides and Pricing. Terms and privacy sit under Optional because they are not needed to understand what the product does.

llms-full.txt

Some sites also publish /llms-full.txt, a single file that holds the text of their pages instead of links to them. It is not part of the proposal on llmstxt.org. Our auditor requests it once and reports whether it exists, whether it is served as HTML and how large it is. It does not validate the contents. Publish one only if you already keep the text of your key pages in Markdown.

The mistakes our auditor flags

The AI Search Readiness Auditor requests /llms.txt once and checks it against the proposed format. These are the findings, with the code it reports.

Finding	What triggers it	Fix
Served as HTML	The request returns 200, but the response is HTML: an HTML content type or a body that looks like a web page. Common on single-page apps and hosts that answer unknown paths with the home page. The file counts as absent.	Serve a plain text file at `/llms.txt` and check the Content-Type with curl.
Not found or empty	A 404 or another non-2xx status, or an empty body. The file counts as absent.	Publish the file at the site root.
`missing-h1`	No line that starts with `#` and a space. The file is invalid.	Start the file with `# Your site name`.
`h1-not-first`	Something comes before the H1: a comment, front matter or a line of text.	Move the H1 to the first line.
`multiple-h1`	More than one H1.	Keep one H1 and use H2 for sections.
`missing-summary`	No blockquote before the first H2.	Add a line that starts with `>` right after the H1.
`no-link-sections`	No H2 section holds a link.	Add at least one H2 section with a list of links.
`empty-section`	An H2 section has no links under it.	Add links or remove the heading.
`bad-list-items`	A list item inside a section has no Markdown link, such as a bare URL or plain text.	Write each item as `- [name](url): note`.
`heading-in-details`	An H3 to H6 heading sits between the summary and the first H2.	Use paragraphs or lists there, or make it an H2 section.

A file is reported as valid when it has exactly one H1 as its first line, a summary and at least one section with links. With an H1 but one of the others missing, it is partial. Without an H1 it is invalid. These checks test the format only. They say nothing about whether any AI product reads the file.

Do you need one?

It costs little, so publish one if you have a clear set of pages to point to. Fix robots.txt first. Vendor documentation describes robots.txt tokens for training, search and user requests, and a blocked search crawler is a problem you can measure. The AI crawler reference lists the tokens, and the robots.txt guide has a file to copy.

Adoption is still low. In our October 2026 check of popular sites, about 1 in 10 published a valid llms.txt: 9.1% in Mexico, 11.3% in the US sample and 8.3% in Canada. Details and limits are in the study write-up.

Check your site

The free check reads your robots.txt for all 20 crawlers and tells you whether your site publishes an llms.txt. The format checks above run in our full report and in the auditor on Apify.