AI Discovery File Adoption Research

Key Findings

The Q3 2026 crawl continues the upward trend in AI Discovery File adoption, which reached 9.4% of successfully crawled domains, up from 7.2% in Q2. The gain is driven almost entirely by llms.txt, now on 122 domains with 82 valid, complete implementations. The number of AI-Ready sites (Tier 4) grew from 33 to 44, and more organisations are explicitly permitting AI crawlers rather than staying silent. One caveat matters for reading these figures: this quarter 251 domains errored versus 90 in Q2, so the successfully crawled base fell to 1,744. Percentages are computed on that smaller sample, which softens some quarter-over-quarter comparisons that depend on the denominator.

1. llms.txt adoption keeps climbing and now carries the whole category llms.txt grew from 93 to 122 domains (+31%), reaching 7% of the crawled sample. Valid, complete files rose from 61 to 82, and the wider quality overview shows complete files across all domains up from 64 to 85. This one file now accounts for the overwhelming majority of all AI Discovery File adoption. Every other file type stayed at or near zero, so the growth story and the concentration risk are the same story.
2. AI-Ready sites grow by a third as Cloudflare and Adobe join the list Tier 4 (AI-Ready) domains rose from 33 to 44 (+33%). The top adopters now include Cloudflare (global rank 7) and Adobe (rank 68), alongside Bluehost, Dyson, Fox News, Klaviyo, and Groupon. These are high-traffic, infrastructure-scale properties, which matters because their choices tend to set defaults that smaller sites copy. Tier 5 (AI-Optimised) still stands at zero: no domain yet pairs three or more valid files with explicit crawler permission and Schema.org markup.
3. More sites explicitly allow AI crawlers instead of staying silent Domains that explicitly permit AI crawlers rose from 23 to 31, lifting the explicit-allow share from 1.2% to 1.8%. Sites with no AI policy at all eased from 85% to 84.2%. Outright blocking barely moved (blocks-all 22 to 23). The shift is small in absolute terms but it points one way: where organisations are making a deliberate choice, more of them are choosing to opt in to AI access rather than block it.
4. Crawl coverage dipped this quarter, which flatters some declines Only 1,744 of 1,995 domains were successfully crawled this quarter, down from 1,905 in Q2, because 251 domains errored versus 90. That matters when reading softer numbers: Schema.org presence fell from 29.7% to 26.3% and Partially Ready (Tier 3) dropped from 21.7% to 18.5%, but both are partly a function of the smaller, slightly different sample rather than a genuine retreat. The average readiness score held flat at 2.2 out of 5.0. We flag this openly rather than presenting the declines as behavioural change we cannot prove.
5. Adoption is still one file deep Of the ten AI Discovery File types we track, only llms.txt shows meaningful adoption. ai.json, identity.json, brand.txt, faq-ai.txt, and developer-ai.txt sit at zero to two domains each, and llms.html actually slipped slightly from 47 to 46 found. Publishers are adopting the single best-known file and stopping there, which leaves identity, brand, and FAQ signals largely unaddressed across the web.
6. Zero sites reference a formal specification All 164 domains with an AI Discovery File were published without any reference to a formal specification, exactly as in Q2. Adoption continues to run ahead of standardisation: publishers are writing these files by hand or from ad hoc templates, with no shared schema or version to validate against. That gap is the clearest opening for a canonical, machine-checkable standard to become the reference point.

— AI Visibility Research, July 2026