Is llms.txt an Official Standard? What It Actually Is in 2026

No. llms.txt is not an official standard. As of 2026 it is a community proposal, originated by Jeremy Howard of Answer.AI, that has not been ratified by the IETF, the W3C, or any other standards body. It has no RFC number and no formal specification process behind it. It is a convention some sites have voluntarily adopted, which is a meaningfully different thing.

That distinction matters because a lot of writing about llms.txt implies that publishing one changes how AI assistants treat your site. There is currently no public evidence for that claim, and this guide explains exactly what is and is not known.

What llms.txt is proposed to do

The idea is straightforward and, on its own terms, sensible. A file at /llms.txt provides a curated, plain-text map of your site's most important content, written for a language model rather than a human or a traditional crawler. In the proposal it is a markdown file containing the site name, a short description, and an annotated list of key URLs.

The motivation is real. When a model retrieves from your site, it may fetch navigation-heavy, boilerplate-laden HTML and extract very little of value. A clean index of "here is what actually matters and here is what each page covers" would help. The reasoning is sound.

The problem is adoption on the consuming side.

Who actually reads it

This is the question that matters, and the honest answer is: no major AI crawler operator publicly documents support for llms.txt.

OpenAI, Anthropic, Google and Perplexity all publish documentation covering their crawlers, their user-agent strings, and their robots.txt behaviour. None of them documents fetching or honouring llms.txt. There is no announcement, no changelog entry, and no developer documentation from any of them describing it as a supported file.

Some site owners report that their llms.txt has been fetched. That is not the same as it being honoured. Any file at a predictable path will be fetched by something eventually: scrapers, research crawlers, monitoring services, and curious humans. A request in your access log is evidence of a fetch, not evidence that an assistant's answer changed because of it.

So when you see a claim that "AI crawlers read llms.txt in 2026," the correct response is to ask which crawler, and where that operator documented it. As of now, nobody can point at that documentation.

How this differs from robots.txt

The comparison people reach for is robots.txt, and the comparison breaks down in an instructive way.

robots.txt is also not a formal standard in the strictest sense for most of its life, but it has something llms.txt does not: near-universal documented adoption. Every major crawler operator publishes which user agents it uses and states that it honours robots.txt directives. It is enforced by convention so consistently that it functions as a standard in practice.

llms.txt has the opposite profile. It has a clear specification and essentially no documented consumers. A convention with no consumers does not control anything.

There is also a functional difference worth being precise about. robots.txt is an access control file, telling crawlers what they may fetch. llms.txt is a content curation file, suggesting what models should prioritise. Even in the most optimistic future where it is widely adopted, it would be advisory rather than binding.

If you want to actually control AI crawler access today, robots.txt is the mechanism that works, and our robots.txt checker will show you which AI crawlers your current file allows or blocks.

Should you publish one anyway?

Yes, with correct expectations. The case for it is not that assistants will read it.

The real benefit is the exercise. Writing a good llms.txt forces you to answer a question most sites answer badly: what are the fifteen pages on this site that actually matter, and what does each one cover in one plain sentence? That is a genuinely useful artifact. Teams routinely discover during this exercise that their most important pages are buried, that two pages cover the same topic, or that they cannot describe their own category clearly. Those findings are worth more than the file.

The cost is close to zero. It is a static text file. It will not slow your site or harm your search performance.

The risk is misallocation. If publishing llms.txt substitutes for work that demonstrably moves AI visibility, it has cost you something. It is housekeeping, not strategy.

If you want to generate one, the llms.txt generator will build a valid file from your site structure, and the llms.txt guide covers format details.

What actually influences AI citations

Since the file itself is not the lever, it is worth stating what is. Four things are verifiable and consistently matter:

Crawler reachability. The crawlers that are documented to exist need to be able to fetch your pages. If your robots.txt blocks them, or your content only renders after JavaScript execution that non-rendering crawlers do not perform, nothing else matters. Preparing your website for AI crawlers covers this properly.

Consistent description across sources. Models weight agreement between independent sources heavily. If your homepage, your review-site profiles, and third-party articles all describe you the same way, that description becomes what the model believes. If they conflict, the model hedges, and hedged brands do not get recommended.

Presence on pages that already contain shortlists. Roundups, comparisons and directories are the raw material answer engines assemble shortlists from. Being absent from those is the most common reason a brand is absent from answers, as covered in why competitors appear in AI search but you don't.

Extractable structure. A page an engine can lift a clean claim from will be cited more than a better-written page where the answer is buried. Question-shaped headings, the answer stated first, and valid FAQ markup all help. Schema markup for AI covers which structured data types earn their place.

The bottom line

llms.txt is a thoughtful proposal addressing a real problem, with no documented consumers as of 2026. Publish one if you like, because it costs nothing and the exercise of writing it is clarifying. Do not expect it to change your visibility, and be sceptical of anyone who tells you otherwise without pointing at crawler documentation.

If you want to know where you actually stand across the assistants that exist today, the free AI Visibility Scorecard measures it directly rather than inferring it from files at your domain root.