Pakistan
USA
Canada
UK
Dubai
Australia
Germany
- llms.txt is a community proposal, not an official web standard.
- Google says Google Search does not use llms.txt files.
- It can help some AI tools and coding agents find key pages.
- Crawlable pages, sitemaps and clear content matter far more.
What does an llms.txt file contain?
An llms.txt file is a Markdown document placed at yoursite.com/llms.txt. According to the proposal at llmstxt.org, it starts with a heading for the site or project name, then a short summary in a blockquote, then optional notes and lists of links to important pages with a brief description of each.
The idea is to give AI tools a short, clean map of your best content without making them parse navigation, adverts and scripts. Some sites also publish Markdown versions of key pages or a longer companion file, but these are optional extras within the same proposal. The file is meant to be short and human-readable, so anyone opening it in a browser should understand it at a glance.
Who proposed it and who reads it?
The format was proposed in 2024 by Jeremy Howard of Answer.AI and is maintained as an open proposal at llmstxt.org. It is not a standard from the W3C, the IETF or any search engine. Anyone can publish a file in the format, and any tool can choose whether to read it. Nothing obliges an AI company to fetch it, follow it or give it extra weight, and most have not said that they do.
Adoption has been strongest among software documentation sites, where developers point coding assistants at an llms.txt file to load documentation. There is little public evidence that mainstream AI chat or search products rely on it to choose which businesses to mention. Treat any claim that llms.txt boosts AI rankings with caution unless the platform itself documents it.
What has Google said about llms.txt?
Google's AI optimisation guide for site owners says you do not need to create AI text files, Markdown versions or special markup to appear in generative AI search. Google has also stated that Google Search does not use llms.txt files, and that having one will neither help nor harm your visibility in Google Search.
Google does say such files can be fine for other services that choose to use them. Separately, Chrome's Lighthouse tool has added an llms.txt check in its agentic browsing audits, which treats the file as optional. That check is aimed at AI agents in the browser, not at Search rankings.
How do I create one?
If you decide to add one, keep it short and accurate.
- Pick your key pagesChoose your main services, about page, contact page, pricing approach and best guides, usually ten to thirty links.
- Write the headerStart with
# Business Name, then a one or two sentence summary of what you do and where. - Group the linksUse headings such as
## Servicesand## Guides, with each link written as a Markdown list item and a short note. - Upload to the rootPublish it as plain text at /llms.txt so it returns a normal 200 response.
- Keep it currentUpdate it when pages are added, renamed or removed, the same as your sitemap.
How is llms.txt different from a sitemap and robots.txt?
Three files, three different jobs.
File | Purpose | Used by Google Search |
|---|---|---|
robots.txt | Tells crawlers which paths they may or may not fetch | Yes, for crawling rules |
sitemap.xml | Lists the URLs you want discovered and indexed | Yes, as a discovery hint |
llms.txt | Gives AI tools a curated Markdown summary of key pages | No, according to Google |
How we handle this for clients
We treat llms.txt as a small, optional task near the end of an AI visibility project, not the start. First we check crawler access, indexing, the XML sitemap, page structure and business details, because those affect every search and AI system. Any problems found there are fixed first, since an llms.txt file cannot make up for pages that crawlers cannot reach or understand.
If a client wants an llms.txt file, or has developer or software documentation where coding agents are likely readers, we write a short curated file and add it to the regular maintenance list. We are clear that it is not a ranking lever. This sits within our AI SEO, AEO and GEO services, where the bigger gains usually come from content, crawling and reputation work.
Does llms.txt matter for local businesses in Pakistan, the Gulf or the UK?
For most local service businesses, llms.txt is a low priority. A clinic in Lahore, a contractor in Dubai or a solicitor in Manchester will gain far more from a complete Google Business Profile, consistent directory listings, real reviews and clear service pages. Those are the sources AI tools and Google actually draw on when someone asks for a local recommendation, and they also help people who never use AI search at all.
If you run a multi-country site, an llms.txt file can list each market's main pages so the structure is clear. Keep the file in English unless your audience is mainly Urdu or Arabic, and make sure it never contradicts the details on your live pages. An out-of-date file listing old phone numbers or closed branches does more harm than having no file at all.
What matters more than llms.txt?
Do these first.
Crawler access
Allow the search crawlers you want, such as Googlebot, Bingbot and OAI-SearchBot. See our robots.txt guide.
Answer-first pages
Clear pages that answer specific questions are what AI systems actually quote.
Structured data
Organization and LocalBusiness schema help systems understand who you are.
Reputation
Genuine reviews and mentions on other sites shape how you are described.
Mistakes people make with llms.txt
The file is simple, but these errors make it pointless or misleading.
- Treating it as a ranking factorGoogle has said Google Search does not use llms.txt. Spend most of your effort on crawlable pages, clear content and internal links.
- Listing every page on the siteA long dump of URLs defeats the purpose of a short map. Link only to the pages that most clearly explain your business, products and services.
- Letting it go staleLinks to deleted or redirected pages give AI tools outdated information. Review the file whenever you change key pages.
- Blocking the bots you want to reachPublishing llms.txt while robots.txt or your CDN blocks AI crawlers sends mixed signals. Check that your bot rules match your intent.
- Saying different things from your pagesSummaries that promise more than your pages say can mislead. Keep descriptions short, factual and consistent with the linked content.
Junaid has more than 18 years in SEO, paid media, web development and custom software. He founded SEOnumber1, builds WordPress plugins, CRMs and web applications, has served 500+ clients in 48 countries, and holds Google Ads, Analytics and Google Blue and Green Belt certifications. He is a National Level Award Winner in Digital Marketing. Read his story or ask a question about this article.