The llms.txt audience isn’t AI search. It’s coding agents.

Ahrefs just ran the numbers on 137,000 domains and the result is the sort of thing that should end an argument. Ninety-seven percent of llms.txt files got zero requests. No bots. No humans. Nothing. Of the roughly 38,000 domains with valid files, only about 1,100 received any traffic at all.

And of the traffic that did show up, the breakdown is the interesting part. SEO audit tools, 21%. Unidentified bots, 14%. Web crawlers like Googlebot, 13%. Tech profiling tools like BuiltWith, 11%. AI bots across all categories combined: 19%. Within that 19%, coding agents took 10%, training crawlers 5%, AI assistants 2%. The retrieval bots tied to ChatGPT and Perplexity — the ones that would actually generate citations in AI answers — were about 1% of total requests.

Slackbot fetched llms.txt files more often than PerplexityBot did. Let that sit for a second.

Google updated its own documentation the same week to confirm Search ignores the file entirely. Not penalises, not rewards, ignores. So we now have a file format with effectively no audience for the use case it was sold for, and a measurable audience for use cases nobody talks about. The story being told about llms.txt and the story the logs are telling are not the same story.

What the logs say the file is actually for

Strip the marketing layer off and read the data as infrastructure. The bots fetching these files are, in rough order: tools auditing whether the file exists, generalist crawlers indexing it because it's a URL, technology profilers cataloguing what sites publish, and coding agents that need terse machine-readable site descriptions to write code against.

Notice what that list isn't. It isn't ChatGPT browsing for a citation. It isn't Perplexity grounding an answer. It isn't Google's AI Overviews pulling structured context. The systems people are publishing llms.txt files *for* are not the systems requesting them.

The systems that are requesting them — Claude-Code, GPTBot in training mode, validators — have a different job. Coding agents want a compressed map of your site so they can build integrations against it. Training crawlers want bulk text. Validators want to grade the file's existence so somebody can sell a GEO readiness report off the back of it.

That's a perfectly reasonable ecosystem. It's just not the one the file was pitched as.

The validator industry is now larger than the audience

This is the bit I can't stop thinking about. Twelve percent of requests came from tools that audit, scan, or study llms.txt files. GEO and AEO readiness tools alone accounted for 5%. Dedicated scanners and validators another 3%. Research crawlers 2%, with the largest one identifying itself as a prompt injection survey.

Add it up and the tooling built to grade llms.txt files generates more traffic to those files than the AI retrieval bots they're supposedly written for.

Add it up and the tooling built to grade llms.txt files generates more traffic to those files than the AI retrieval bots they're supposedly written for.

That's the loop. And we built it.

This is what happens when an industry decides a thing matters before the underlying use case is real. You get a marketplace of validators, scanners, and dashboards measuring adoption of a file format whose primary audience is the validators themselves. The Chrome Lighthouse audit that reignited the whole debate in May generated about 22 requests across the entire dataset. Roughly one in a thousand. The audit drove more conversation than crawl activity.

I'm not throwing rocks at the file format itself. The original llms.txt proposal had a reasonable thesis: give models a compressed, machine-friendly site description so they don't have to render JavaScript and parse navigation chrome. For coding agents that have to build against your site, that's genuinely useful. The data confirms it — Claude-Code is one of the top individual requesters. Honest use case, honest result.

The dishonesty was further up the stack. Vendors and conference circuits took a developer-facing convenience file and resold it as an AI search ranking factor, with no evidence it was being used that way and now with direct evidence it isn't.

The Mueller test keeps passing

John Mueller has been saying for over a year that llms.txt is "not done for search" and described it as "a temporary crutch, perhaps to save some tokens" for AI coding tools. Lily Ray pressed him on the contradiction between Google Search's dismissal and Chrome Lighthouse promoting it as an audit item. He didn't move.

He was right the whole time. And SE Ranking's earlier analysis of 300,000 domains found no correlation between having llms.txt and AI citation frequency, which is exactly what you'd expect if the retrieval bots aren't requesting the file. You can't be cited for something nobody read.

The pattern here is one I've watched repeat through three or four hype cycles now. Google says a thing isn't a ranking signal. The SEO industry decides Google must be hiding something. Tools get built on the assumption Google is lying. Eighteen months later, the data confirms Google was telling the truth, and the tools quietly pivot to selling "AI readiness audits" of the next thing. Hreflang went through a version of this. So did AMP. So did several flavours of structured data that were sold as ranking factors before being demoted to "Google may use this for display."

The signal we keep ignoring is the one where the platform tells you straight that something doesn't matter.

What this means if you've been told to publish one

If you run a UK business and somebody pitched you llms.txt as part of a GEO package, here's the honest read.

Publishing one is fine. It costs little, it won't hurt your search rankings, and if Claude-Code or a similar coding agent ever needs to build an integration against your site, you've made their life slightly easier. That's a real, if modest, benefit. Treat it as developer documentation, not as a marketing asset.

Believing it will increase your ChatGPT or Perplexity citation rate is not supported by any data anyone has produced. The bots that would generate those citations are barely requesting these files. If your GEO consultant has llms.txt as a line item on a six-figure deliverable, ask them to show you the log data — yours or anyone's — where AI retrieval bots are actually pulling it.

The genuinely high-leverage activities for being cited by AI systems remain the same ones the SE Ranking and SEO Week studies keep confirming: traditional ranking signals, brand authority, earned media, and clean structured data on the pages themselves. That's the work. It's the same work it was three years ago. The branding around it changes more than the work does.

The bit nobody's flagging

There's one Ahrefs detail worth a closer look. The largest research crawler in their dataset identified as a prompt injection survey, studying llms.txt as an attack vector. Because agents trust the content they ingest, a file that a CMS auto-generates and that the site owner never audits is a clean injection surface.

If you're publishing one — particularly via a plugin that builds it from your post content or sitemap — somebody on your team should read what's actually in the file. The exposure is small but it's real, and the people thinking hardest about llms.txt right now appear to be security researchers rather than marketers.

That's usually a useful tell about where a technology actually is in its life cycle.

The interesting work in any maturing channel ends up being defensive before it's promotional. We're at the defensive stage on llms.txt. The promotional stage is still being sold.