← Back to Journalai-news

The Honest Truth About llms.txt: What It Does, What It Doesn't, and Whether Your Business Needs One

🗓️ ·✍️ ElephantNY Editorial ·📍 NYC & Aegean Hospitality

If you have shopped for an “AI visibility” or GEO service lately, you have probably been told your site needs an llms.txt file. It is being sold as the robots.txt for AI — a quick fix that gets your business cited by ChatGPT, Perplexity and Gemini. As an agency that does this work, we think you deserve the unhyped version.

What llms.txt actually is

llms.txt is a proposed convention, not an official web standard. It was put forward by Jeremy Howard of Answer.AI in September 2024, and a second version of the spec was published on 10 August 2026 (llmstxt.org). The idea is simple: a plain-text/Markdown file at the root of your domain that points large language models to the pages worth reading. Where robots.txt tells crawlers what they may not touch, llms.txt suggests what is most useful.

What the adoption data really shows

Publishing is rising — but the numbers depend entirely on who is counting:

  • A Tranco Top-10,000 crawl in May 2026 found roughly 5.9% of sites had the file.
  • An SEOmator census of the top 100,000 domains in September 2026 reported about 8.6%.
  • An Ahrefs study of 137,000 sites reported 28% publishing a valid file — much higher — and, crucially, that 97% of those files were never actually read by an AI crawler.

Part of the jump is automatic: Shopify rolled the file out to every store in spring 2026 without asking merchants. So a high publish rate does not mean businesses chose it, and it does not mean anything is reading it.

The part the hype skips

No major AI company has confirmed it uses llms.txt for ranking or citation. Per Ahrefs, OpenAI’s GPTBot honors robots.txt but does not officially use llms.txt; Anthropic publishes its own file but does not say its crawlers consume the standard. And in May 2026, Google’s own AI-search guidance stated you do not need to create new machine-readable or Markdown files to appear in generative search (Search Engine Land).

Our recommendation

Adding a clean llms.txt is cheap, harmless, and reasonable future-proofing — so yes, publish one. But treat it as housekeeping, not a growth lever, and be skeptical of anyone selling it as the way to win AI visibility. What actually gets a business cited by AI engines is the same thing that earns a citation from a journalist: clear, well-structured, genuinely authoritative content with real data and sources that a model can quote with confidence. The file is the doormat; the content is the house.

Sources: llmstxt.org (official spec); Ahrefs — What Is llms.txt; Ahrefs — 137K-site study; Search Engine Land. Adoption figures vary by sample and methodology.

Editorial Transparency. A first draft of this story was produced with AI-assisted writing tools, then reviewed for accuracy and tone by the named editor before publication. More on our process: Editorial Policy.
Editorial Transparency. A first draft of this story was produced with AI-assisted writing tools, then reviewed for accuracy and tone by the named editor before publication. More on our process: Editorial Policy.

Subscribe to our newsletter

Weekly stories and what is opening this week.

Ready to Dominate AI Search Recommendations?

Run a free AI check on your restaurant or book a 20-minute call.

Launch AI Visibility Scanner →