What llms.txt Actually Is
llms.txt is a proposed convention, a plain-text file at the root of a domain that summarizes what a site is and links to its most important pages, written in a format meant to be easy for a language model to parse. It's the same idea as robots.txt, but aimed at giving AI systems a quick orientation instead of controlling crawler access.
It is not an official standard adopted by OpenAI, Anthropic, Perplexity, or Google. No major AI vendor has confirmed that their crawlers read it, weight it, or use it in retrieval at all. Some almost certainly do look at it. None have said it changes whether you get cited.
That distinction matters, because "add an llms.txt file" has become the go-to advice in a lot of AI-SEO content, often presented as if it's the fix. It's not. It's a nice-to-have on top of a site that already works. On a site that doesn't, it does nothing.
The Misconception
Here's the actual failure pattern I see in audits: a site adds llms.txt, points it at their pricing page, their docs, their comparison pages, and nothing changes. Citation rate in ChatGPT Search or Perplexity stays flat. The founder asks why the file "isn't working."
The file isn't broken. The premise is. llms.txt is a pointer. It tells a crawler where your important content lives. It says nothing about whether that content is actually retrievable once the crawler gets there, whether the crawler can tell your brand apart from a similarly-named competitor, or whether your pricing page renders anything at all without JavaScript.
Those three problems sit upstream of llms.txt, and they're the ones that actually decide whether an AI engine cites you.
What Actually Determines Citation
Can the crawler read the page at all
Most AI crawlers (GPTBot, ClaudeBot, PerplexityBot) fetch raw HTML. They don't execute your JavaScript bundle. If your pricing table, feature comparison, or product spec only exists after a script runs, the crawler sees an empty container. It doesn't matter what your llms.txt links to, if the destination page is blank on arrival, there's nothing to cite.
This is the single most common thing I find in audits: a client-side rendered SPA with genuinely good content that's invisible to anything that doesn't run JS.
Is the data structured enough to extract
A crawler that can read your page still has to figure out what the numbers and claims on it actually mean. "$150/user/month" sitting in a marketing paragraph is ambiguous. The same number in a SoftwareApplication JSON-LD offers block is unambiguous. AI engines lean heavily on structured data and fact-dense, declarative sentences because it's cheaper and more reliable to extract a fact from a labeled field than to infer it from prose.
Is your brand disambiguated
If your product name collides with something else, a common word, a book, another company, retrieval systems have to resolve which entity a query is actually about. sameAs links to Wikidata and Crunchbase are how you tell a knowledge graph "this domain is this specific entity." Without that link, you're competing against every other meaning of your own name.
Where llms.txt Actually Helps
None of this means skip it. Once the three problems above are fixed, on a site that renders cleanly, has structured pricing and product data, and has a disambiguated entity graph, llms.txt is a legitimate, close-to-free addition. It gives any crawler that does respect it a direct, low-noise summary instead of making it infer your site's structure from scratch. It costs nothing to add and there's no real downside.
The point isn't that llms.txt is useless. It's that it's the last five percent, not the first fix. Adding it to a site that's invisible to crawlers is like putting a well-organized table of contents on a book with blank pages.
Key Takeaways
- llms.txt is an unconfirmed, low-cost convention, not a ranking or citation mechanism. Treat it as a nice-to-have. - If your pricing, features, or comparisons only render via client-side JavaScript, fix that first. It's the most common reason sites are invisible to AI search. - Structured JSON-LD data (SoftwareApplication, Organization, sameAs links) does more for citation rate than any summary file. - Add llms.txt after the underlying site is retrievable, not instead of making it retrievable.