llms.txt and AI Search

What llms.txt is, and what Google's AI features actually use.

llms.txt is a proposal, first published in September 2024, for a Markdown file that gives large language models a short summary of a site and links to its most useful pages, ideally with clean Markdown copies of those pages. Some developer documentation sites publish one so coding assistants can read their docs. It isn't a web standard, and no search engine is required to read it.

Google has said plainly what it does with these files in Search: nothing. Its guide to optimizing for generative AI features says Google Search ignores llms.txt and similar AI text files, so they neither help nor harm a site in Google Search, AI Overviews and AI Mode included. Those features ground their answers in pages from Google's search index, so the work that matters is ordinary SEO: pages that can be crawled, indexed and shown with a snippet, and content worth citing. This page sums up Google's documentation, the controls that actually change how your content is used, and what the llms.txt format looks like.

Last reviewed October 2, 2026 against the sources below.

llms.txt and Google's AI features, in short

Question	Answer
What is llms.txt?	A proposed Markdown file, first published by Jeremy Howard in September 2024, that gives language models a short summary of a site and links to its most useful pages. It's a community proposal, not a web standard.
Does Google Search use llms.txt?	No. Google's AI optimization guide says Google Search ignores llms.txt and similar AI text files, so they neither help nor harm visibility or rankings, including in AI Overviews and AI Mode.
What does a page need to appear in AI Overviews or AI Mode?	It must be indexed and eligible to show with a snippet in Google Search, and the site must be included in Search generative AI features in Search Console (the default). Even then, appearing isn't guaranteed.
How do Google's AI features pick sources?	They ground answers in pages from Google's search index (retrieval-augmented generation) and may run several related searches at once (query fan-out) to find more sources, then link to them.
Is optimizing for AI search different from SEO?	Not from Google's point of view. Its guide says optimizing for generative AI search (sometimes called AEO or GEO) is optimizing for the search experience, and so still SEO.
Where does AI feature traffic show up?	In Search Console's Performance report, under the Web search type, together with other Google Search traffic.

Controls that actually work

Each setting controls something different. None of them is llms.txt.

Control	What it does	What it doesn't do
Search Console: Search generative AI control	Include or exclude a site's links and content from AI Overviews, AI Mode and generative AI features in Discover. Include is the default; available worldwide since August 31, 2026	Not a ranking or inclusion signal for the rest of Search, and not a training control
Google-Extended (robots.txt)	Controls whether content Google crawls may be used to train future Gemini models and for grounding in Gemini Apps and the Vertex AI API for Gemini	Doesn't affect inclusion or ranking in Google Search
nosnippet, max-snippet, data-nosnippet	Limit or stop text from a page being shown as a snippet, which Google's AI features also respect	Don't remove the page from the index
noindex	Keeps a page out of Google Search entirely, AI features included	Doesn't stop crawling; the page must be crawlable for Google to see the rule
robots.txt Disallow for Googlebot	Stops Googlebot crawling the page, so its content can't be used	Doesn't guarantee the URL stays out of the index if other pages link to it

What to do and what to skip

From Google's guide to optimizing for generative AI features.

Practice	Verdict	Why
Publishing llms.txt to rank in Google's AI features	Not needed	Google Search ignores it. Publish one only if another tool you care about reads it
Splitting content into small chunks for AI	Not needed	Google says there's no need to break content into tiny pieces for AI to understand it
Rewriting pages in special phrasings for AI	Not needed	Google says you don't need to write in a particular way just for generative AI search
Chasing mentions on blogs, forums and videos	Not worth it	Google says seeking inauthentic mentions across the web isn't as helpful as it might seem
Adding extra structured data just for AI features	Not needed	Google says AI features need no special schema.org markup. Structured data still matters for rich results
Making pages indexable and snippet-eligible	Do	It's the basic requirement for appearing in AI Overviews and AI Mode
Writing original, first-hand content	Do	Google contrasts commodity content built on common knowledge with pages that offer unique expert or first-hand insight
Deciding on AI training separately from Search	Do	Training (Google-Extended) and AI features in Search (the Search Console control) are separate settings

The llms.txt format

From the proposal at llmstxt.org. Only the H1 is required.

Part	Required?	Example
H1 with the site or project name	Yes, the only required part	# Northwind Coffee
Blockquote summary	Optional	> Small-batch coffee roaster in Portland, Oregon
Paragraphs or lists with more detail	Optional	Roasting days, shipping regions, wholesale terms
H2 sections listing links	Optional	## Guides, then - [Brewing guide](https://www.example.com/brew.md): pour-over basics
Markdown copies of pages	Suggested by the proposal	https://www.example.com/brew.md next to /brew

Good to know

Other AI search tools

ChatGPT, Claude, Perplexity and others find pages with their own crawlers and fetchers, which you control in robots.txt by user-agent token. None of the operators in our AI crawler table says in its crawler documentation that it reads llms.txt. See the AI crawler user agents page for each token and whether it honors robots.txt.

Should you publish one anyway?

It's harmless, and it can help tools that do read it, such as some coding assistants loading documentation. Just don't expect it to change anything in Google Search. We publish a plain one at /llms.txt that lists our tools and reference pages.

Sources

Search engines and AI companies change their documentation often; check the source before relying on a detail.