llms.txt, explained: what it is and whether anything reads it

What is an llms.txt file?

An llms.txt file is a plain markdown document at the root of a domain — yoursite.com/llms.txt — that summarizes the site for a machine reader. Jeremy Howard of Answer.AI proposed it on September 3, 2024. The reasoning was practical rather than promotional: a model handed a website has to guess which pages matter, and HTML is mostly navigation, scripts and markup that burn context before any content arrives.

So the file is a curated index. Rather than making a reader crawl your site and infer its shape, you hand it a short document stating who you are and which pages are worth opening.

It is a proposal, not a ratified standard. There is no registry, no validator anyone must pass, and no company obliged to read it. That last part matters more than the format, and the third section deals with it.

What goes inside an llms.txt file

The specification is short enough to hold in your head. In order:

• An H1 with the name. The only genuinely required element.
• A blockquote summary. One line, carrying the key information.
• Any markdown prose. Detail about the subject, with one rule: no headings inside it.
• H2 sections holding link lists. Each item is a markdown link, optionally followed by a colon and a note about what the link contains.
• A section named Optional. By convention, links a reader can skip when it needs to be shorter.

Written for a person rather than a software project, that comes out roughly like this:

# Jane Doe

> Founder and CEO of Northwind Climate, a Berlin company
> building grid-forecasting software for utilities.

Jane Doe has worked on energy systems since 2014, first at Acme
and since 2021 at Northwind Climate, which she co-founded.

## Background
- [About](https://janedoe.com/about): biography and current role
- [Talks](https://janedoe.com/talks): conference talks, with dates

## Coverage
- [Interview](https://example.com/interview): Handelsblatt, 2026

## Optional
- [Archive](https://janedoe.com/archive): older writing

Two properties do the work. It is markdown, so it survives being pasted into a prompt with its structure intact. And it is short, so it fits in a context window whole — which is the entire reason for preferring it to your homepage.

Does anything actually read llms.txt?

Mostly nothing does. Ahrefs studied 137,210 domains in May 2026 and found that 28% published a valid llms.txt, and that 97% of those files were fetched zero times that month. Around 1,100 domains saw so much as a single request. Of the requests that did land, 96% came from bots, and the largest single category was SEO audit tools at 21.7%, ahead of every AI category.

Named AI tools accounted for 19.5% of fetches in total: agents 10.5%, training crawlers 5.3%, assistants 2.5%, retrieval bots 1.1%.

The most informative finding is the quietest one. No AI bot ever requested an llms.txt from a domain that did not have one. Nothing probes for the file. It gets read only when something already knows the URL and goes looking — an agent you pointed at your own site, a developer pasting a documentation link — which is a different situation from a stranger asking a chatbot who you are.

That distinction is most of the answer. An llms.txt is plausibly useful once a machine has been sent to your domain. It has no route into the answer a model generates from memory, and that answer is where the reputational damage usually happens. Why ChatGPT gets your bio wrong covers what does shape it.

llms.txt is not robots.txt

The comparison is everywhere and it misleads. robots.txt works because crawler operators agreed to honor it — a directive with a committed audience. llms.txt is a courtesy document with no committed audience, and nothing obliges anyone to open it.

Google has said as much on the record. At its Search Central Deep Dive in July 2025, Gary Illyes said Google does not support llms.txt and is not planning to, and that it will not crawl the file; John Mueller had said earlier that no AI system was using it. Neither statement has been withdrawn, and no other major vendor has published a commitment to reading it.

Publishing one carries no risk. It costs a few hundred bytes and cannot degrade anything else on the site. Treating it as the lever that changes what a model says about you is the error, and a fair amount of advice sold in 2026 makes exactly that error.

Should you publish an llms.txt file?

Publish one if you already own a site and can write it in ten minutes. Skip it if writing it displaces work on the record itself. It is a cheap bet with a small and unproven payoff, and it should be ranked accordingly: below having a real page about you that a crawler can read without running JavaScript, and below a schema.org Person block that tells an entity resolver which person you are.

That is how it is ranked here. An IndexMe site check looks at six things on a page you own — whether it loads, whether your name appears in the delivered HTML, whether the text survives with JavaScript off, whether robots.txt blocks AI crawlers, whether there is Person markup, and whether an llms.txt exists. Those findings are advisory. None of them feed the AI Presence Index, because the index measures what models say, not what your site contains, and letting the two mix would allow anyone to raise a number by publishing a file nobody reads.

If you do write one, three things make it worth the bytes:

• Put the disambiguating facts in the blockquote. Name, role, organization, field, in one sentence. Where a namesake exists, that line is what separates you from them — two people with one name is the harder version of this problem.
• Link to pages that corroborate you, not only pages you own. A list of your own marketing is weaker than one that includes third-party coverage naming you and your work together.
• Date it and keep it current. A stale llms.txt is worse than none, because the one reader you get is handed your old job.

Then judge it honestly. Nobody can tell you an llms.txt will change an answer, and any tool that promises it is guessing. What you can do is publish it, keep reading the models on a schedule, and find out whether the record moved.