How to use the llms.txt Generator
The llms.txt generator runs entirely in your browser. You type in a few details, it assembles the file, and you copy or download the result. Nothing you enter is sent to a server or stored.
- Enter your site or brand name. This becomes the single top-level heading of the file, so use the name people actually search for, not a tagline.
- Write a one- or two-sentence summary of what your site is and who it is for. This becomes the blockquote near the top. Keep it factual and specific.
- Add optional background notes if there is something a machine reader should know before it starts following links, such as the products you sell, the regions you serve or how your docs are organized.
- Group your key pages into sections, such as Docs, Products, Guides or Policies. Give each section a short name.
- For every page, enter the title, the absolute URL and a one-line description that says what the page covers. Add as many as you need.
- Move less important pages into the Optional section. These are links an assistant can skip when it has limited room to read.
- Click Generate, review the preview, then copy the text or download it as
llms.txt. - Upload the file to the root of your domain so it loads at
https://yourdomain.com/llms.txtand open that URL in a browser to confirm it is plain text.
Tip: Start with ten to twenty pages. A short, curated file is more useful than a dump of every URL you own. You can always regenerate and replace it later.
What is llms.txt?
llms.txt is a proposed convention for giving large language models a clean, human-readable map of your site. It is a plain Markdown file that lives at the root of your domain, next to robots.txt and sitemap.xml. The idea was published in 2024 and is documented at llmstxt.org.
The reasoning is practical. A modern web page is wrapped in navigation, scripts, cookie banners, ads and layout code. When an AI assistant has to read your site, most of what it downloads is noise, and the context window it works in is limited. An llms.txt file lets you say, in a few hundred words, what the site is and which pages matter most, with direct links to each one.
The format is intentionally simple. It is not XML, JSON or a custom schema. It is Markdown, because Markdown is easy for people to write and easy for language models to read. A valid file follows a loose structure:
- A single H1 with the name of the site or project. This is the only required element.
- A blockquote with a short summary of the site.
- Optional paragraphs or lists with more detail, written in plain text.
- Zero or more H2 sections, each containing a list of links in the form
- [Title](https://example.com/page): short description. - An optional section named Optional, which signals links that can be skipped when a shorter context is needed.
It helps to think of llms.txt as a hand-written table of contents. A sitemap lists every URL for crawlers that want completeness. llms.txt lists the few URLs you would hand to a smart newcomer who has five minutes to understand your business.
llms.txt versus llms-full.txt
You will often see two related files mentioned together. llms.txt is the index: a short file of links with descriptions. llms-full.txt is an expanded companion that contains the actual content of those pages, concatenated into one large Markdown document.
The index is good when an assistant can follow links and fetch only what it needs. The full file is good when a tool wants everything in a single request, for example a coding assistant that loads your documentation into its context. Documentation sites are the most common users of the full variant, because developers paste those files directly into their tools.
For most marketing sites and small businesses, the index alone is enough. If you do publish a full version, keep it current, because a stale copy of your pricing or product details is worse than none. The HTML to Markdown Converter is handy for turning a page into clean Markdown when you build the full file by hand.
Why llms.txt matters for AI search
Answer engines and AI assistants such as ChatGPT, Claude, Perplexity and Gemini increasingly act as the first stop for questions that used to go to a search results page. When they answer a question about your business, they either rely on what they learned during training or they fetch live pages to ground the answer. In both cases, the quality of the material they can reach shapes what they say about you.
llms.txt is one way to improve that material. It gives a reader a short, accurate description of who you are, and it points to the pages that state your most important facts: what you sell, how much it costs, how it works, who it is for and how to contact you. If an assistant does read it, you have influenced which pages it looks at first and how your brand is summarized.
There is a second benefit that does not depend on any AI company. Writing an llms.txt file forces you to decide which pages are your cornerstone pages and to describe each one in a sentence. If you cannot do that, the pages are probably unclear for human visitors as well. Many teams find the exercise itself exposes gaps: a missing pricing explanation, an out-of-date product page, three pages that all answer the same question.
An honest look at adoption
It is important to be straightforward about the state of this convention. llms.txt is a proposal, not an official standard. No standards body has ratified it, and the major AI companies have not committed to reading it. In practice, AI engines rarely fetch llms.txt files today, and there is no public evidence that having one increases how often you are cited or mentioned in answers.
That does not make the file worthless, but it should shape your expectations. Treat it as a low-cost, low-risk measure with possible upside:
- It takes minutes to create and almost nothing to maintain.
- It cannot hurt your search rankings, because it is a separate file that search engines do not use for ranking.
- Some AI tools, developer assistants and agent frameworks do read it, especially for documentation.
- If adoption grows, you will already have a clean, current file in place.
What you should not do is expect it to fix a visibility problem on its own. The pages that AI engines cite are decided mainly by content quality, clarity, authority and whether the engine can reach and understand your pages in the first place. llms.txt supports those fundamentals; it does not replace them.
If you want to know whether any AI crawler is actually requesting your file, check your server logs for requests to /llms.txt. This is the only reliable way to learn whether the file is being used on your site, and it is one of the things the aeoora AI crawler analytics is built to show.
How AI engines and tools can use the file
Even without a formal standard, there are a few realistic ways the file gets used.
Live retrieval by assistants
When a user asks an assistant to read or summarize a site, some assistants and agents will look for an llms.txt file first, because it is a cheap way to learn the structure. If it exists, they may follow the links you chose instead of guessing from your navigation.
Developer and coding tools
Coding assistants and documentation tools are the clearest adopters. They load documentation indexes into their context so that a model can answer questions about a library or API accurately. If your product has docs, an llms.txt index and a full version can make those tools noticeably more useful for your customers.
Retrieval pipelines you do not control
Teams building their own retrieval systems sometimes start from a site’s llms.txt to decide what to ingest. A concise file with good descriptions makes their job easier and makes the result more faithful to how you describe yourself.
Your own internal use
You can also use the file for your own purposes. It is a handy reference to paste into an assistant when you want it to write about your company, because it holds your approved summary and your canonical URLs in one place.
Best practices
A good llms.txt file is short, accurate and opinionated. These practices apply whether you build it with the generator or by hand.
Lead with a precise summary
The blockquote is the most-read part of the file. State what you are, who you serve and what makes you distinct in plain language. Avoid slogans and superlatives. Compare “Project management software for construction teams, with scheduling, budgets and subcontractor tracking” with “The ultimate all-in-one platform for modern teams.” Only the first tells a model something it can repeat accurately.
Link only to pages you stand behind
Every link is an endorsement. Choose pages that are accurate, current and public. Prefer canonical URLs, avoid redirects and make sure each link returns a normal 200 response. Do not include pages blocked by robots.txt or hidden behind a login, because that sends a mixed message.
Write descriptions that carry information
The text after each link is where the file earns its value. Say what the page contains, not that it is a page. “Pricing for all three plans, including limits per plan and what the trial includes” is useful. “Our pricing page” is not. Aim for one sentence, with concrete nouns and numbers where they exist.
Group by what a reader is trying to do
Section names should map to tasks, not your org chart. Typical groups include Docs, Products, Pricing, Guides, Policies and Support. If you serve distinct audiences, you can separate them, but keep the number of sections small enough that the file stays scannable.
Use the Optional section deliberately
Anything under the Optional heading can be dropped by a tool that needs to save space. Put secondary material there: older posts, press pages, long policy documents, changelogs. Keep your essential pages above it, so a truncated read still captures the important content.
Keep it consistent with the rest of your site
The facts in llms.txt must match what your pages say. If the file claims a free trial that your pricing page no longer offers, an assistant that reads both will notice the conflict, and you lose trust with human readers too. Update the file whenever you change pricing, product names or policies.
Make the file easy to fetch
Serve it as plain text (text/plain or text/markdown) at the root of the domain over HTTPS. Do not put it behind a redirect chain, a cookie wall or bot protection that challenges unknown clients. If you want AI crawlers to reach it, the robots rules for your site must also allow them, which is where the AI Crawler robots.txt Generator helps.
Consider Markdown versions of your pages
The convention also suggests that pages linked from llms.txt can offer a clean Markdown version at the same URL with .md appended. This is optional, but it makes the linked material much cheaper to read. If you run WordPress, the free aeoora plugin can serve Markdown for every page automatically.
Examples
These examples show the shape of a good file. Replace the names, URLs and descriptions with your own.
A small business site
This file suits a local service business or small online shop. It is brief, factual and points to the pages customers ask about most.
# Harbor Light Coffee Roasters
> Small-batch coffee roaster in Portland, Oregon. We sell whole-bean coffee online, run a retail cafe and supply cafes across the Pacific Northwest.
We ship within the United States. Wholesale accounts are available for businesses ordering 20 lb or more per month.
## Shop
- [All coffees](https://harborlightcoffee.example/coffee/): Every whole-bean coffee we roast, with origin, roast level and tasting notes
- [Subscriptions](https://harborlightcoffee.example/subscribe/): Weekly, biweekly and monthly delivery, with pricing and how to pause or cancel
- [Shipping and returns](https://harborlightcoffee.example/shipping/): Delivery times, costs and our return policy
## Learn
- [Brewing guides](https://harborlightcoffee.example/brew/): Step-by-step instructions for pour-over, espresso, French press and cold brew
- [Our sourcing](https://harborlightcoffee.example/sourcing/): The farms and cooperatives we buy from and how we price green coffee
## Company
- [Wholesale](https://harborlightcoffee.example/wholesale/): Minimum orders, pricing tiers and how to open an account
- [Contact and cafe hours](https://harborlightcoffee.example/contact/): Address, opening hours, phone and email
## Optional
- [Press](https://harborlightcoffee.example/press/): Articles and mentions about the roastery
- [Journal archive](https://harborlightcoffee.example/journal/): Older posts about events and seasonal releases
A software product with documentation
Software companies usually have more pages worth listing. Notice how the sections follow the tasks a developer or buyer has, and how descriptions include specifics.
# Northwind Metrics
> Product analytics for B2B SaaS teams. Northwind Metrics tracks feature usage, retention and revenue by account, and exposes the data through a REST API and SQL access.
Pricing is per tracked account, not per seat. The service is hosted in the EU and the US, and data residency is chosen at signup.
## Product
- [Overview](https://northwind.example/product/): What the product does, who uses it and the main reports
- [Pricing](https://northwind.example/pricing/): Plans, tracked-account limits, overage rules and the 14-day trial
- [Security](https://northwind.example/security/): Encryption, access controls, hosting regions and compliance documents
## Documentation
- [Quickstart](https://northwind.example/docs/quickstart/): Install the SDK and send your first event in about ten minutes
- [Event API reference](https://northwind.example/docs/api/events/): Endpoints, payload fields, rate limits and error codes
- [Data model](https://northwind.example/docs/data-model/): Accounts, users, events and how they relate
- [SQL access](https://northwind.example/docs/sql/): Connecting a warehouse client and the available tables
## Guides
- [Measuring feature adoption](https://northwind.example/guides/feature-adoption/): A walkthrough of building an adoption report from raw events
- [Churn analysis](https://northwind.example/guides/churn/): How to define churn and find the usage patterns that precede it
## Optional
- [Changelog](https://northwind.example/changelog/): Release notes by date
- [Status page](https://status.northwind.example/): Current uptime and incident history
Serving the file correctly
After you upload the file, confirm it is reachable and served as text. You can check from a terminal with a single command.
curl -I https://yourdomain.com/llms.txt
HTTP/2 200
content-type: text/plain; charset=utf-8
A 200 response with a text content type is what you want. A 404 means the file is in the wrong folder, and an HTML content type usually means your site is rewriting the request to a page template.
Common mistakes
- Listing every URL. llms.txt is a curated guide, not a sitemap. If you include hundreds of links, you lose the signal that makes the file useful. Use your sitemap for completeness.
- Vague descriptions. Entries like “Learn more” or “Our blog” give a model nothing to work with. Say what the page actually answers.
- Marketing language in the summary. Superlatives and slogans do not help a model describe you correctly. Plain, specific statements do.
- Relative or broken links. Use full absolute URLs, and check that each one resolves. A file full of 404s signals neglect.
- Linking to blocked or private pages. If robots.txt disallows a page or it requires a login, do not list it.
- Letting it go stale. Old pricing, discontinued products or renamed features will be repeated back to your customers. Review the file whenever you ship a significant change.
- Putting the file in the wrong place. It must be at the domain root, not inside a folder or a CMS media library. A subdomain needs its own file.
- Expecting instant results. Because AI engines rarely fetch the file today, do not measure its success by citations next week. Measure whether the file is accurate and whether anything requests it.
- Treating it as access control. llms.txt only suggests what to read. It does not stop any crawler from fetching other pages, and it is not a way to block or allow bots. That is the job of robots.txt.
- Using it to hide content. Do not try to show models different information from what visitors see. Inconsistent content damages trust and can be treated as manipulation.
llms.txt, robots.txt and sitemaps compared
These three files are often confused, and each one does a different job. Understanding the split keeps you from putting the wrong information in the wrong file.
robots.txt is about permission. It tells crawlers which paths they may or may not request, and it is the file that controls whether AI crawlers such as GPTBot or ClaudeBot can fetch your pages at all. It is widely respected by well-behaved crawlers, although it is voluntary.
sitemap.xml is about discovery. It lists the URLs you want search engines to find, usually in full, along with optional modification dates. It is machine-oriented and exhaustive.
llms.txt is about orientation. It gives a language model a short, human-written description of the site and a prioritized list of pages. It is not a permission file and not a complete inventory.
A sensible setup uses all three. Allow the AI crawlers you want in robots.txt, keep the sitemap complete and current, and publish an llms.txt that points to the pages you most want understood. If you block a crawler in robots.txt, it should not matter what llms.txt says for that crawler, because it is not supposed to be fetching your pages at all.
Maintaining the file over time
An llms.txt file is only as good as its last update. Build a small habit around it.
Review it whenever you launch or retire a product, change pricing, restructure your docs or rename something. A quarterly check is a reasonable minimum for a stable site. During each review, click every link, reread each description and ask whether the summary still matches how you describe the business today.
If you manage a larger site or several sites, automate the parts that can be automated. Generating the file from your CMS means the links and titles stay in sync with the content, and you only hand-edit the summary and the descriptions. The free aeoora WordPress plugin does this for WordPress sites, and it can also produce an llms-full.txt and Markdown versions of pages. For other platforms, a short script that reads your sitemap and merges it with hand-written descriptions works well.
Finally, keep the file under version control if you can. A change history helps you see when a description drifted and makes it easy to roll back an error.
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal published in 2024, and no standards body has adopted it. That is why it is best treated as a lightweight, low-risk addition rather than something your visibility depends on. The format is stable and simple enough that creating a file now carries little risk.
Do ChatGPT, Claude, Perplexity and Gemini read llms.txt?
AI companies have not published commitments to read it as part of their normal crawling, and in practice AI engines rarely request the file today. Some tools and agents do, particularly developer-focused ones. The most reliable way to learn what happens on your own site is to check your server logs for requests to the file.
Will llms.txt improve my rankings or AI citations?
There is no evidence that it directly improves traditional search rankings, and no public evidence that it increases how often AI engines cite you. Citations depend far more on the quality, clarity and authority of your content and on whether engines can reach your pages. Think of llms.txt as clean supporting infrastructure, not a growth lever.
Where do I put the file and what should it be named?
Save it as llms.txt in the root of your domain so that it is available at https://yourdomain.com/llms.txt. Each subdomain that you want covered needs its own file. Serve it as plain text over HTTPS without redirects.
How long should my llms.txt be?
There is no fixed limit, but shorter is better. Many good files have between ten and fifty links. If you find yourself listing hundreds of pages, move the secondary ones to the Optional section or leave them out and rely on your sitemap.
Can llms.txt block AI crawlers?
No. The file does not control access. To allow or block specific AI crawlers, use robots.txt rules, and use the AI Crawler robots.txt Generator to build them. You can also check which bot is visiting with the AI Bot User-Agent Checker.
Should I use llms.txt, llms-full.txt or both?
Publish llms.txt first. Add llms-full.txt only if you have documentation or long reference content that tools benefit from loading in one go, and only if you can keep it current. Many sites do well with just the index.
Does the generator store my content?
No. The generator works in your browser, and the text you enter stays on your device. When you close the page, it is gone unless you copied or downloaded the file.
Go further with aeoora
An llms.txt file is one small piece of being understood by AI engines. If you run WordPress, the free aeoora WordPress plugin can generate and maintain llms.txt and llms-full.txt for you, serve a Markdown version of every page, and manage your robots rules for AI crawlers. See the plugin documentation for the details.
To see whether any of this is changing how AI engines talk about you, start a 2-day trial of aeoora Cloud. It tracks how your brand appears in answers across AI engines, and the GEO tools include an llms.txt optimizer for paid plans. You can compare plans on the pricing page, or browse the other free tools for schema, robots rules and more.