In brief
llms.txt is a proposed Markdown file at the root of a website that gives AI tools a short summary and a list of your most useful pages. It is cheap to add and does no harm. But no major AI search engine has confirmed it uses the file to choose sources, and Google says it is not needed for Search.
Key points
- llms.txt is a proposal from 2024, published at llmstxt.org, not an agreed web standard.
- The file is plain Markdown with a title, a one paragraph summary and lists of links to your key pages.
- Google says llms.txt is not needed for Search and will not help or hurt rankings.
- No major AI search engine has said it reads llms.txt when choosing which pages to cite.
- It can still be useful for developer docs and tools that fetch pages on request, so it is a low cost extra, not a priority.
- Crawl access, clear pages and accurate facts about your business matter far more.
The steps at a glance
- 1
Choose your key pages
Pick 10 to 30 pages that best explain what you do, such as services, pricing, locations, about and your strongest guides.
- 2
Write the title and summary
Start the file with an H1 of the business name, then a one paragraph blockquote saying what you do, where, and for whom.
- 3
Group links under H2 headings
Add sections such as Services or Guides, each a Markdown list of links with a short plain description after each one.
- 4
Add an Optional section
Put secondary pages under an H2 called Optional, so tools with limited space can skip them.
- 5
Publish and check
Save it as plain text at /llms.txt, check it loads in a browser, and review it whenever key pages change.
What is llms.txt?
llms.txt is a plain text file, written in Markdown, that sits at the root of a website. It gives large language models a short summary of the site and a list of the pages most worth reading.
The idea was proposed by Jeremy Howard in 2024 and is published at llmstxt.org. The thinking behind it is simple. Web pages are full of menus, scripts and adverts. A model with limited space to read would get more from a short, clean map of the site.
It is often compared to robots.txt, but it does a different job. robots.txt tells crawlers what they may fetch. llms.txt suggests what is worth reading. It has no power to allow or block anything. For access rules, see the guide to AI crawlers and robots.txt.
Does llms.txt actually work?
For AI search, there is no evidence yet that it does. None of the major AI search engines has confirmed that it reads llms.txt when choosing which pages to cite.
Google has been the most direct. In June 2026 it added a note to its Search documentation updates saying these files are not needed for Google Search. It said they will not help or hurt your visibility or rankings, and that it is fine to keep them for other services. Its guide to generative AI features also says you do not need AI text files or Markdown versions of pages to appear.
OpenAI, Anthropic and Perplexity document their crawlers and how they follow robots.txt. They have not said their search systems use llms.txt. That may change, but it would be wrong to promise a result from it today.
Where the file has caught on is software documentation. The llmstxt.org site says thousands of sites now publish one, and that major AI labs use it for their own developer docs. Publishing a file is different from an AI search engine reading yours, though.
What goes in an llms.txt file?
The proposal sets out a simple order. Only the title is required.
- An H1 heading with the name of the site or business.
- A blockquote with a short summary of what you do.
- Optional paragraphs or lists with any extra context.
- One or more H2 sections, each holding a list of links with short descriptions.
- An H2 section called Optional for pages that can be skipped when space is tight.
Each link line follows the same pattern. A Markdown link, then a short plain description of what the page covers. The descriptions matter more than people expect, because they are the only context a tool gets before deciding what to open.
What does a good llms.txt example look like?
Here is an example for a made-up dental group with four practices.
# Example Dental Group
> Example Dental Group runs four NHS and private dental practices in West Yorkshire.
> We offer check-ups, hygiene, implants, Invisalign and emergency appointments.
> New patients can book online or by phone. Opening hours vary by practice.
Key facts
- Practices in Leeds, Wakefield, Harrogate and Bradford
- Registered with the General Dental Council
- Emergency slots held every weekday morning
## Services
- [Dental implants](https://www.example-dental.co.uk/implants/): What implants involve, who is suitable, recovery time and private fees
- [Invisalign](https://www.example-dental.co.uk/invisalign/): Clear aligner treatment, typical length of treatment and payment plans
- [Emergency dentist](https://www.example-dental.co.uk/emergency/): How to get a same day appointment and what counts as an emergency
## Practices
- [Leeds practice](https://www.example-dental.co.uk/leeds/): Address, opening hours, parking and team
- [Harrogate practice](https://www.example-dental.co.uk/harrogate/): Address, opening hours, parking and team
## Prices and booking
- [Fees](https://www.example-dental.co.uk/fees/): NHS bands and private price list, updated each April
- [Book an appointment](https://www.example-dental.co.uk/book/): Online booking for new and existing patients
## Optional
- [Our history](https://www.example-dental.co.uk/about/history/): How the group started and grew
- [Careers](https://www.example-dental.co.uk/careers/): Current vacancies
Notice what it does well. The summary says what, where and for whom in three lines. Each link has a plain description with the facts a person would want. Nothing is padded with sales language.
Should you use an llms.txt generator?
A generator is fine as a starting point. Most crawl your site and draft a file in seconds. The trouble is that they tend to list every page and reuse meta descriptions, which are often vague.
A better approach is to run the generator, then edit hard. Cut the list down to the pages that explain your business best. Rewrite each description so it states facts, not slogans. Check that every link works and points to the final URL, not a redirect.
For most small businesses, writing the file by hand takes under an hour. It is also a useful exercise. If you struggle to write a clear one paragraph summary of your business, AI engines will struggle too.
Does llms.txt replace robots.txt or a sitemap?
No. Each file has its own job, and llms.txt replaces neither of the others.
| File | What it does | Who reads it |
|---|---|---|
| robots.txt | Says which crawlers may fetch which paths | Search and AI crawlers that follow the standard |
| sitemap.xml | Lists every URL you want indexed, with dates | Search engine crawlers |
| llms.txt | Summarises the site and points to key pages | Tools that choose to look for it |
Keep your sitemap complete and your robots.txt accurate first. Those two have long, well documented support from the main search engines. An llms.txt file sits on top as an optional summary.
Who gets the most from llms.txt?
Some sites have more reason to add one than others. Software companies with developer documentation are the clearest case. Their users often paste docs into coding tools, and a clean index of pages makes that easier.
Sites with a lot of reference content come next. Think of a B2B software company with a large help centre, or a college with hundreds of course pages. A short map helps any tool find the pages that matter.
A small local business sits at the other end. A plumber with twelve pages gains little, because the whole site is already easy to read. It will not hurt to add one, but it belongs near the bottom of the list.
Can llms.txt hurt you?
On its own, no. Google has said it will not affect rankings either way. There are two small risks worth knowing about.
The first is stale facts. If the file says you open on Saturdays and you stopped last year, any tool that reads it now has a wrong detail. The second is exposure. Do not list private, staging or members only pages. The file is public, and anyone can read it.
How do you keep llms.txt up to date?
Treat it like any other page with facts on it. Give it an owner and a review date.
- Review it when you add or remove a service, a location or a price list.
- Review it after a site move, since old links will break.
- Check the summary still matches your home page and Google Business Profile.
- Keep it short, so updates take minutes rather than an afternoon.
A good habit is to review it on the same day each quarter as your other business details. That keeps it in line with the facts on your site.
What about llms-full.txt and Markdown copies of pages?
The proposal also suggests offering clean Markdown copies of pages, at the same address with .md added. Some sites go further and publish llms-full.txt, a single file holding the full text of their key pages.
These are most useful for software documentation, where developers paste docs into coding tools. For a service business, they add upkeep with little clear gain. Every copy is another place where prices or opening hours can go out of date. Out of date copies are one way AI gets your business wrong.
What matters more than llms.txt for AI visibility?
Several things have a much clearer link to being cited. A sensible order of work looks like this.
- Make sure AI search crawlers such as OAI-SearchBot and PerplexityBot are allowed in robots.txt.
- Make sure key pages load without JavaScript tricks and have clear headings.
- Publish accurate, consistent facts about your business across your site and profiles.
- Add clean schema markup with a stable identity for the business.
- Earn mentions on the sites AI engines already cite in your topic.
Once those are in place, an llms.txt file is a reasonable extra. It is cheap, harmless and quick to update. Just treat it as a tidy signpost, not a ranking lever.
How Axiom GEO helps
Axiom GEO does not generate llms.txt files. AI crawler analytics reads your server access logs and shows which AI bots crawl your site, how often and which pages. That lets you see whether any of them request the file at all. AI citation tracking shows which of your pages AI engines actually cite, which tells you what belongs at the top of your list. The entity audit checks the structured identity your site publishes, which has a clearer link to how engines understand you.
Sources
These are the external pages this guide relies on.