Short answer
What matters most
Do not treat llms.txt as a general AI search ranking or citation shortcut. Google says it ignores the file for Search. OpenAI's publisher guidance focuses on allowing OAI-SearchBot so pages can be surfaced in ChatGPT Search; it gives no llms.txt step. Add a concise file only when a named agent workflow can read it, test the result, and keep its links current.
Start with the job, not the file
Put the intended outcome on the brief before asking for a file. A search result needs a public page that its engine can fetch and evaluate. A human-triggered agent may use a curated map while gathering context, if that product chooses to read it. These are separate paths; a file in the second path cannot repair a blocked or weak page in the first.
The diagram separates the two jobs. For search visibility, check access to the canonical page and the engine's own index. For a documentation workflow, name the agent that will read the file and test what it retrieves.
What platform documentation actually supports
The strongest evidence is unusually clear for Google Search: its guide says Google Search does not use llms.txt for its AI features and that the file neither helps nor harms visibility or rankings. Google's AI results still rely on pages that meet Search eligibility rules, and Google says crawling, indexing, and serving are not guaranteed.
OpenAI's publisher guidance says not to block OAI-SearchBot if you want site content to appear in ChatGPT Search summaries and snippets. Its crawler documentation distinguishes that search crawler from GPTBot, which may be used for model training. These pages focus on crawler access; they do not list llms.txt as a visibility step.
Google ignores llms.txt for Search
The guide, updated July 10, 2026, says new AI text files are not needed for Google Search and do not improve or harm visibility. It points site owners back to indexable pages and core Search requirements.
Google Search Central: generative AI search guidanceChatGPT Search has a named crawler
OpenAI identifies OAI-SearchBot as its search crawler, distinct from GPTBot. Its publisher FAQ says to avoid blocking OAI-SearchBot for site content to appear in ChatGPT summaries and snippets. Access is a prerequisite, not a guarantee of citation.
OpenAI: overview of crawlersThe file's proposal serves a different job
The proposal describes llms.txt as a curated, LLM-readable overview that an agent can consult on demand. That explains the intended use; it does not show that a particular search product reads the file or uses it for ranking.
AnswerDotAI: llms.txt proposalMeasure citations where reporting exists
Bing's February 2026 announcement describes AI Performance reporting for citations and sampled grounding queries. Microsoft says these counts do not show placement or ranking, so use them as visibility signals rather than proof that a file caused a citation.
Microsoft Bing: AI Performance announcement, February 10, 2026Match the action to the outcome
Use this matrix before assigning implementation work. It keeps a useful directory file from being mistaken for a crawler control or a search-ranking mechanism.
On a small screen, scroll the table to read every column.
| Goal | First check | When llms.txt fits |
|---|---|---|
| Google AI Search | Check that the canonical page is indexable and eligible for a Search snippet. | It does not: Google says it ignores the file. |
| ChatGPT Search | Check OAI-SearchBot access in robots.txt and at the site edge; then watch referrals. | Current OpenAI guidance calls out OAI-SearchBot access, not llms.txt. |
| Copilot or Bing answers | If available, inspect Bing Webmaster Tools AI Performance cited pages and query samples. | The reporting does not attribute citations to llms.txt. |
| A named agent needs site context | Test whether that agent reads a curated file and follows the intended current pages. | It may be a useful map when an owner can maintain it. |
A hypothetical priority call
Imagine a small consultancy whose public service pages return normally and appear in its sitemap. The team wants more discovery in Google AI Search and ChatGPT Search. A developer suggests adding llms.txt before checking anything else.
I would first inspect index eligibility in Search Console, then verify that OAI-SearchBot is not blocked by robots.txt or infrastructure rules. If a supported reporting view is available, compare the actual cited pages and referrals over time. If the service pages are missing clear answers to buyer questions, improve those pages with specific, verifiable expertise. Publishing a file alone would not address either gap.
The hypothetical team has no internal agent or documentation workflow that reads a curated site map. In that case, a custom generator would add maintenance without a named consumer. If the team later builds such a workflow, it can test a short file against that real task.
Use this decision brief before building
For a file intended to support an agent, write down the consumer and the job before implementation. The file should earn its place by helping that workflow find the right current source.
- Name the tool or agent expected to read the file.
- Describe one task it should complete using the listed pages.
- List the canonical pages it should use and who owns their accuracy.
- Repeat the task with the file available; check that the answer points to the right current source.
- Set an update owner and a trigger for changes to the underlying pages.
Skip the custom build when the file has no reader
For a public website with clearly linked pages, ordinary navigation, crawl settings, and the search engine's reporting tools are a simpler starting point. If a platform already creates a concise llms.txt file, keep it aligned with canonical pages and treat it as optional documentation. Do not install a generator or promise search gains without a documented consumer.
If your investigation turns into a broader question about which site or AI system work is worth funding, my AI consulting engagement can help scope the decision before you commit to a custom build.
Choose the next check
Before publishing another file, name the search engine or agent and the result you want from it. For search visibility, verify crawl access and page eligibility first, then measure what your available reporting actually shows. Add llms.txt only when a real context-loading workflow can test and maintain it.
If you are deciding whether a site visibility problem needs engineering at all, bring the affected page and the buyer question to a conversation with me. I can help separate a crawl or content gap from a system that actually needs to be built.