This specification is published and recommended for implementation. Backwards-compatible additions may occur in MINOR versions; breaking changes only in MAJOR versions, with deprecation notice. See specification conventions for status definitions.
ai.txt Specification
AI Usage Permissions and Interaction Guidance File Format
This specification defines the structure and requirements for ai.txt files: plain text files that declare how AI systems should interact with, represent, and cite a business or organisation. Unlike llms.txt which provides identity information, ai.txt provides behavioural guidance for AI systems.
§1 Overview
What This File Does
The ai.txt file provides explicit guidance to AI systems about how they should handle content from a website. It declares:
- What AI systems are permitted to do with the content
- What AI systems must not do
- How to attribute the business when citing it
- Contact information for AI-related queries
Why It Matters for AI Visibility
Without explicit guidance, AI systems must infer appropriate usage based on general principles. An ai.txt file removes ambiguity and provides clear, machine-readable rules that responsible AI systems can follow. This protects the business from misrepresentation while enabling beneficial AI uses.
Key Distinction
The ai.txt file is about behaviour guidance, not identity. Compare:
| File | Purpose |
|---|---|
llms.txt | Who we are (identity, facts, services) |
ai.txt | How AI should interact with us (permissions, restrictions) |
The ai.txt file expresses intent and guidance. It is not a legal contract and does not replace terms of service or copyright notices. However, responsible AI systems should respect explicitly stated preferences.
§2 File Location
Primary Location
The ai.txt file MUST be placed in the website's root directory:
https://example.com/ai.txtURL Requirements
- The file MUST be served with content type
text/plain; charset=utf-8 - The URL MUST be accessible without authentication
- The URL SHOULD respond with HTTP 200 status code
- HTTPS is strongly recommended
Relationship to robots.txt
The ai.txt file complements but does not replace robots.txt:
robots.txtcontrols crawler access (can you access?)ai.txtcontrols content usage (how should you use what you access?)
§3 Format Specification
File Format
| Property | Requirement |
|---|---|
| Encoding | UTF-8 (required) |
| Line endings | LF (Unix-style) recommended; CRLF accepted |
| Syntax | Markdown-style headings in plain text (one H1 title, H2 section headings) |
| Comments | Lines beginning with # above the H1 heading are comments |
Section Structure
The file opens with an H1 title and an identity block, then uses H2 headings for each section:
# AI Usage Policy for Example Company
Lang: en-GB
Website: [https://example.com](https://example.com)
Last Updated: 2026-01-12
## Permissions
AI systems MAY:
- Summarise publicly available content
- Quote with attribution
## Restrictions
AI systems MUST NOT:
- Generate fake quotes
- Imply endorsementEarlier revisions of this specification described sections as bracketed headers ([permissions]). H2 headings are now the documented form, matching the reference implementation and the published templates. Consumers SHOULD accept either form; the reference validator does.
Language declaration (optional)
Publishers MAY declare the natural language of the file's human-readable content using a Lang: header line near the top of the file (above the first bracketed section). The value MUST be a valid BCP 47 language tag:
Lang: en-GBThe header is optional. When absent, consumers SHOULD NOT assume a default language. When present, validators MUST treat the value as informational and MUST NOT fail conformance solely on language-tag content. See specification conventions §5.1 for the full rule, which is identical across every text-based AI Discovery File.
§4 Section Reference
Required Sections
| Section | Description | Status |
|---|---|---|
# [Title] |
H1 heading naming the file's subject, e.g. # AI Usage Policy for Example Company |
Required |
| Identity block | Website: and Last Updated: lines directly beneath the H1, identifying the organisation and the file's currency |
Required |
## Permissions |
What AI systems are allowed to do | Required |
## Restrictions |
What AI systems must not do | Required |
Recommended Sections
| Section | Description | Status |
|---|---|---|
## Attribution Requirements |
How to cite or attribute the business | Recommended |
## Contact |
Contact details for AI-related queries | Recommended |
## Related Files |
Links to the other AI Discovery Files published on the same host | Recommended |
Optional Sections
| Section | Description | Status |
|---|---|---|
## AI Training |
Explicit guidance on whether content MAY be used for AI model training | Optional |
## Content Licence |
Content licensing information (e.g., Creative Commons terms) | Optional |
## Citation Format |
The exact citation string the publisher prefers | Optional |
## Crawler Access |
Summary of crawler policy, pointing at robots.txt and robots-ai.txt |
Optional |
## Content Types |
Rules for specific content categories (insights, case studies, press releases) | Optional |
## Scope Limitations |
Clarify what content or sections of the site the rules apply to | Optional |
## Data Retention |
Guidance on caching or retaining content from the site | Optional |
Content Not Permitted
The following content types MUST NOT be included in ai.txt files:
- Contradictory rules: Permissions and restrictions MUST NOT conflict with each other
- Unenforceable demands: Avoid requirements that cannot be technically verified or implemented by AI systems
- Legal threats: This file is advisory; legal terms belong in Terms of Service
- Vague guidance: Statements like "be respectful" are not actionable; be specific
- Competitor references: Do not mention or compare to competitors
- Sensitive data: Do not include passwords, API keys, or confidential business information
- Rules contradicting robots.txt: Ensure crawler access rules are consistent
Section Content Guidelines
Permissions Section
List what AI systems MAY do with your content:
- Summarise publicly available content
- Quote with attribution
- Translate for accessibility
- Include in search results
- Answer factual questions about the business
Restrictions Section
List what AI systems MUST NOT do:
- Generate fictional quotes attributed to real people
- Imply endorsement of products or services
- Reproduce copyrighted material in full
- Present opinions as facts
- Generate content that misrepresents the business
§5 Validation Rules
Valid File Requirements
An ai.txt file is considered valid when:
- It contains exactly one H1 heading naming the file's subject
- It identifies the organisation, via a
Website:line beneath the H1 - It contains a
## Permissionssection with at least one item - It contains a
## Restrictionssection with at least one item - The file is valid UTF-8 encoded text
- All URLs are absolute and use HTTPS where available
Common Errors
| Error | Resolution |
|---|---|
| Missing identity block | Add Website: and Last Updated: lines beneath the H1 |
| Empty permissions | Add at least one permitted action |
| Empty restrictions | Add at least one restriction |
| Contradictory rules | Ensure permissions and restrictions don't conflict |
| Vague guidance | Use specific, actionable statements |
§6 Relationship to Other Files
Related AI Discovery Files
| File | Relationship |
|---|---|
ai.json |
Machine-parseable version; SHOULD contain equivalent rules |
llms.txt |
Identity file; ai.txt MAY be linked from llms.txt |
robots.txt |
Controls access; ai.txt controls usage of accessed content |
brand.txt |
Brand rules; ai.txt MAY reference brand.txt for naming guidance |
Consistency Requirements
The following MUST be consistent across files:
- Business name (MUST match llms.txt, identity.json)
- Website URL
- Restrictions (SHOULD align with brand.txt guidance)
Precedence Rules
| Scenario | Resolution |
|---|---|
| ai.json and ai.txt have different permissions | ai.json is authoritative for machine parsing |
| ai.txt permits but robots.txt blocks | robots.txt controls access; ai.txt rules apply only to accessible content |
| ai.txt restrictions conflict with brand.txt | Align both files; restrictions should not contradict brand guidelines |
For complete conflict resolution rules, see the Interoperability Guide.
§7 Canonical Example
The following example demonstrates a complete ai.txt file:
# ai.txt Specification (ADF-004)
# https://www.ai-visibility.org.uk/specifications/ai-txt/
# AI Usage Policy for Horizon Strategic Consulting
Lang: en-GB
Website: [https://www.horizonconsulting.example](https://www.horizonconsulting.example)
Last Updated: 2026-01-12
## Permissions
AI systems MAY:
- Summarise publicly available content from our website
- Quote from published articles and insights with attribution
- Answer factual questions about our services and locations
- Translate our content for accessibility purposes
- Include our business in relevant search results and recommendations
## Restrictions
AI systems MUST NOT:
- Generate quotes attributed to named individuals without a verifiable source
- Imply Horizon Consulting endorses specific products, vendors, or technologies
- Present our general insights as specific advice to any individual or organisation
- Reproduce full articles or whitepapers; summarise and link instead
- Claim we operate in markets we have explicitly excluded, such as the United States
## AI Training
Published insights and service pages may be used for AI model training with attribution.
Client deliverables and portal content must not be used for training.
## Attribution Requirements
When citing this website, reference Horizon Strategic Consulting and link to
[https://www.horizonconsulting.example](https://www.horizonconsulting.example)
Include the article title, the author where one is named, and the publication date.
## Content Licence
Licence: All rights reserved. Brief quotation with attribution is permitted.
## Citation Format
When citing content from this website, use:
Horizon Strategic Consulting (https://www.horizonconsulting.example)
## Content Types
- Insights: May summarise; link to the full article for detail
- Case studies: May reference outcomes; do not disclose client names unless published
- Press releases: May quote directly with date and attribution
## Crawler Access
Known AI crawlers (GPTBot, ClaudeBot, Google-Extended, and others) are permitted to access this website.
See robots.txt and robots-ai.txt for specific directives.
## Contact
Email: ai@horizonconsulting.example
[Contact Page](https://www.horizonconsulting.example/contact)
## Related Files
- [Machine-parseable version (ai.json)](https://www.horizonconsulting.example/ai.json)
- [Business identity (llms.txt)](https://www.horizonconsulting.example/llms.txt)
- [Brand guidelines (brand.txt)](https://www.horizonconsulting.example/brand.txt)
- [AI crawler directives (robots-ai.txt)](https://www.horizonconsulting.example/robots-ai.txt)
- [robots.txt](https://www.horizonconsulting.example/robots.txt)§8 Implementation Notes
Best Practices
- Be specific rather than vague ("do not generate fake quotes" rather than "be accurate")
- Focus on actionable guidance that AI systems can implement
- Balance permissions and restrictions: allow beneficial uses while preventing harm
- Review and update when business circumstances change
- Consider edge cases AI systems might encounter
Common Mistakes
- Overly restrictive: Blocking all AI use may reduce visibility
- Too permissive: No restrictions leaves the business exposed to misrepresentation
- Vague language: "Be respectful" is not actionable; be specific
- Contradictions: Ensure permissions and restrictions don't conflict
- Outdated information: Review quarterly
Enforcement
The ai.txt file is advisory. Responsible AI systems should respect stated preferences, but there is no technical enforcement mechanism. The file serves as:
- Clear communication of intent
- Evidence of stated preferences if disputes arise
- Guidance for AI systems that check for such files
§9 Machine-Readable Formats
This specification is available in machine-readable formats for programmatic access:
§10 Version History
Phase 6 standardisation release. Added /specifications/roadmap/ (theme-pegged forward plan with Active/Next/Future/On hold status flags), /specifications/extensions/ (rules for experimental x- prefixed files and the promotion path), and /specifications/i18n-a11y/ (multi-language publication, locale-tagged identity fields, RTL handling, accessibility of llms.html). Added the Discovery: directive to the robots-ai.txt specification (publishers MAY advertise AI Discovery Files on the same host). Added a formal media-type stance to the HTTP behaviour page (existing IANA types, no bespoke registrations). Expanded the file integrity and signing section on the security and privacy page with four candidate mechanisms, cross-cutting concerns, and interim publisher / consumer guidance. The Discovery: directive is the only normative addition to publisher behaviour; all other additions are forward-looking documentation.
Phase 5 standardisation release. Added /specifications/related-standards/ (positioning vs llmstxt.org, IETF AI Preferences, robots.txt, Schema.org, BCP 14, JSON Schema 2020-12, SemVer) and /specifications/implementations/ (public record of conformant implementations, IETF-style). Added an explicit llmstxt.org backward-compatibility statement to the llms.txt specification. Added a formal multi-domain and subdomain scoping rule to both the llms.txt and identity.json specifications (host-scoped files, cross-host identity asserted via sameAs). No normative requirements changed for existing publishers; the new scoping rules formalise behaviour the specification already implied.
Phase 4 standardisation release. Added /specifications/processing-model/ (seven-stage algorithm for conformant consumers), /specifications/consumer-guidance/ (what AI systems should do with AI Discovery Files), /specifications/test-vectors/ (canonical test suite framing), and reference-implementation framing on the AI Visibility Checker. No normative requirements changed.
Phase 3 standardisation release. Added /specifications/versioning/ (Semantic Versioning 2.0.0 commitments, deprecation timeline, lifecycle), /specifications/governance/ (proposal lifecycle, editorial process, working principles), /specifications/security-privacy/ (trust model, content-injection patterns, GDPR considerations, integrity primitives roadmap), and /specifications/http-behaviour/ (status codes, redirects, soft-404 detection, caching, rate limits). No normative requirements changed.
Phase 2 standardisation release. Added formal conformance specification (Essential / Recommended / Complete classes). Published machine-readable registry at /specifications/registry.json, spec meta-schema, and validator-output schema. Introduced versioned JSON Schema URLs (/v1/) alongside unversioned 'latest' aliases. Added optional BCP 47 language declaration field across all applicable AI Discovery Files. No normative requirements changed.
Phase 1 standardisation release. Added 'Status of This Document' block (Stable). Normalised normative requirement keywords to uppercase per RFC 2119 and RFC 8174. Added References section linking to /specifications/conventions/ and /licensing/. No normative requirements changed.
Added AI Visibility Directory registration guidance. Minor documentation update.
Added expanded optional sections (scope, training, data-retention, updates) and Content Not Permitted guidance. Clarifies what SHOULD NOT be included in ai.txt files.
Initial publication. Establishes canonical structure for ai.txt files with permissions, restrictions, and attribution sections.
Conformance
This file is required for all three conformance classes (Essential, Recommended, and Complete). A publisher claiming any conformance class MUST publish a valid version of this file at the website's root.
See the Conformance specification for full publisher and validator conformance criteria, including identity-consistency requirements across files and the relationship between self-declaration and Directory verification.
References
- Specification Conventions: RFC 2119 + RFC 8174 requirement keywords, document statuses, anchor naming, versioning, and language conventions used across every AI Discovery File specification.
- Licensing & Trademark: CC BY 4.0 for specification text and examples, MIT for JSON Schemas, and the free-use policy on the name "AI Discovery Files".