What Is llms.txt?
llms.txt is a relatively new idea designed to help AI systems understand a website.
It is a simple text file placed at the root of your website.
For example:
The idea is to provide AI systems with a simple, organised description of your website and links to the content that matters most.
Think of it as a quick guide to your website for AI.
Why would I need one?
Modern AI systems can find information from websites, but websites can be complicated.
A normal webpage may contain:
- Menus
- Images
- Advertisements
- Navigation
- JavaScript
- Footer information
- Repeated content
- Lots of design elements
An llms.txt file can provide a much simpler way of describing the important parts of your website.
It can point AI systems towards important information such as:
- Who you are
- What your business does
- Your main services
- Important pages
- Documentation
- Useful resources
- Contact information
The proposal describes llms.txt as a curated, AI-friendly map of important website content.
Is llms.txt the same as robots.txt?
No.
This is an important difference.
robots.txt is used to provide instructions to web crawlers about what they may or may not access.
llms.txt is intended to provide information and guidance about your website’s important content.
It does not replace robots.txt, and putting something in llms.txt does not block or allow access to that content.
Is llms.txt an official internet standard?
Not currently.
It is best described as a proposed convention rather than an established web standard that every AI company is required to support. Adoption is still developing.
That means having an llms.txt file is useful as part of preparing your website for AI discovery, but you should not expect it to automatically make your website appear in ChatGPT, Google AI or other AI systems.
What does our health check do?
We simply check whether:
yourwebsite.com/llms.txt
exists and contains information.
If it exists and contains content, the check passes.
If it is missing or empty, you receive a warning.
What should I do if this check fails?
You can create an llms.txt file for your website and place it in the root of your domain.
For example:
yourwebsite.com/llms.txt
It should contain useful information about your website and links to your most important content.
If you don’t know how to create one, ask your web developer to create an llms.txt file for you.
In simple terms
llms.txt is like a short “about this website” guide written for AI systems. It is optional and still an emerging convention, but it can help make your important website content easier for AI systems to understand.