Published by Gist GEO · About our content
Where it came from
Jeremy Howard, an AI researcher, proposed llms.txt in September 2024. The idea is a plain file at the top of a website, written in Markdown, that lists the site’s most important pages with a line about each. An AI system could read that instead of working through the whole site.
Who uses it
The proposal’s own site says it is used most for software documentation, where coding assistants follow it to find the right page. The big AI companies publish one for their own developer documentation.
That is a different thing from an AI assistant reading your llms.txt before it recommends a brand of running shoes.
Does it help you show up in AI answers?
There is no good evidence that it does.
- Google says its Search does not use these files, and that having one neither helps nor hurts.
- OpenAI, Anthropic and Perplexity do not mention llms.txt in the documentation for their crawlers.
- One large study, by SE Ranking, looked at nearly 300,000 domains. About one in ten had the file, and having it showed no link to being cited by AI.
Our view
It is harmless and cheap, and it solves a problem most brands don’t have. If someone offers to improve your AI visibility by adding one, they are selling the wrong thing.
The files that decide whether AI can read you are older and duller: your robots.txt, and the pages themselves. Start with Can AI read my website?
Sources
- The llms.txt proposal. llmstxt.org
- Guide to optimizing for generative AI features (updated July 10, 2026). Google Search Central
- Study of llms.txt across nearly 300,000 domains. SE Ranking, Nov 7, 2025
Questions marketers ask
Should I add an llms.txt file?
If your website platform makes one for you, there is no harm in it. Don’t expect it to change what Google’s AI says about you, and don’t pay anyone for it.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may read, and the major crawlers say they follow it. llms.txt suggests what is worth reading, and nobody has agreed to follow it.