Loading
September 11, 2026
When you scroll through Threads and see someone ask ChatGPT, "Are there any recommended website builders for my brand?" and the AI provides a list, whether your brand makes the cut depends less on how good your site is and more on whether the AI can actually understand it. If an AI crawls your site and fails to grasp your content, misinterprets your services, or misses the point entirely, it won't recommend you. This is exactly what llms.txt and GEO (Generative Engine Optimization) aim to solve. This article explains what llms.txt is, whether it's necessary, what you need to do to set it up manually, and why you don't need to worry about it if you build your site with Kanorio.
More and more people are skipping Google and going straight to AI assistants like ChatGPT and Perplexity to find information. Even Google has integrated AI Overviews above many search results, providing AI-generated summaries that answer questions directly, meaning users may not even click through to a website.
This shift creates a new reality: your website needs to be indexed and ranked by Google, but it also needs to be "AI-readable" so that AI models understand what you do and why you're worth mentioning. If your site is slow to crawl, has a messy structure, or confuses the AI about your services, it won't appear in AI answers—even if your site is perfect for human visitors.
llms.txt is a plain-text Markdown file placed in your website's root directory (e.g., yourdomain.com/llms.txt). It is designed to help AI tools like ChatGPT and Perplexity quickly grasp who you are, what you do, and which pages are important, without the AI having to crawl and piece together your site page by page.
It is different from the robots.txt file you might be familiar with. While robots.txt sets "rules"—telling search engines or AI bots which pages they can or cannot crawl—llms.txt provides a "summary." It proactively organizes your brand overview, key page links, and blog posts into a format that AI can easily digest. Think of it as a site map specifically for AI, rather than a list of access permissions.
In practice, you will usually see two versions: llms.txt, which is a concise guide listing your brand summary and main page links; and llms-full.txt, which provides the full content of your pages in clean Markdown, allowing the AI to understand your entire site without needing to crawl every single page.
Let's be clear: llms.txt is an emerging convention that appeared in 2024, proposed by the developer community. It is not yet an official standard set by any governing body (like W3C or Google), and no AI company has publicly promised that they will "definitely read it" or "prioritize citing it" if you have one. In other words, setting up llms.txt does not guarantee that your brand will be recommended by AI. No one can promise that.
However, even without a mandate or guaranteed results, organizing your site content into a structured, AI-friendly format is genuinely helpful for AI crawlers. Plus, the setup cost is very low. Instead of waiting for this to become a mainstream standard, preparing early is a low-risk move that could help you get a head start on your competitors in the world of AI search.
Without this guide file, AI bots have to crawl every page, judge which content is important, and piece together an understanding of your brand on their own. This process is prone to errors, missed information, or incomplete data due to low crawling efficiency. Google's own developer documentation (in the Lighthouse tool notes) mentions that without such files, AI agents may take longer to retrieve site content, essentially leaving the question of whether the AI understands you correctly up to chance.
If you aren't using an automated tool like Kanorio, maintaining llms.txt manually involves several steps: condensing your brand introduction into an AI-digestible summary, organizing all public page links you want the AI to see, listing your blog posts, writing it all in the correct Markdown format, and uploading it to your root directory. The real headache is that it's not a one-time task—every time you update your content, add a page, or publish a new post, you have to manually sync the file, or the AI will be reading outdated information.
If you build your site with Kanorio, llms.txt and llms-full.txt are automatically generated after your site goes live. This is built into our free plan, so you don't need to upgrade. The auto-generated content includes your brand summary, a list of public pages, and your blog entries, allowing AI to understand your site in seconds. You don't need to study formatting standards, organize links, or upload files manually.
More importantly, this file updates automatically with your site content. When you edit copy, add a page, or publish a new post, llms.txt and llms-full.txt reflect the latest status instantly. Pages set to "noindex" are automatically excluded, following the same logic as your SEO settings, so you don't have to worry about private content leaking into your AI guide. If you have specific requirements for your brand positioning or tone, you can switch to custom mode in the dashboard to write your own version for AI—giving you the perfect balance of automation and control.
Preparing content for AI is just the first step. Another often-overlooked issue is that you have no idea which AIs are actually visiting your site. Kanorio's AI Visibility Management feature automatically identifies visits from over 30 AI bots, including GPTBot, ClaudeBot, PerplexityBot, and Googlebot. It categorizes them into "Training" (used to train models), "Retrieval" (used to find real-time answers), and "Search" types. You can view visit counts, block records, and IP verification status in your dashboard.
If you want to be cited by AI search engines like ChatGPT and Perplexity to gain new exposure, you can keep them open. If you're concerned about your original content being used to train models, you can selectively block them—for example, blocking "Training" bots while keeping "Retrieval" bots to maintain AI search exposure. Traditional search engine crawlers like Google, Bing, and Apple are protected by a whitelist, so adjusting your AI bot settings won't affect your Google ranking. Learn more in our AI Visibility Management guide.
It is not mandatory, and no AI company has promised to act according to this file. However, because the setup cost is low and there are no significant risks, preparing early is a smart choice rather than waiting for it to become an official standard.
No guarantees. llms.txt simply makes it easier for AI to understand your site correctly. Whether you are cited still depends on whether your content is valuable and solves real needs. No file format can replace quality content.
No, they are complementary. SEO deals with "can search engines find and rank me," while llms.txt deals with "can AI understand my content correctly." They serve different goals and do not interfere with each other.
You need to install a plugin or manually write the file, upload it to your root directory, and remember to sync it manually every time you update your site. It is a technical task that requires ongoing maintenance.
Yes. The automatic generation of llms.txt and llms-full.txt, along with AI bot tracking, is built into the Kanorio free plan. No upgrade is required.
If you want to understand the logic behind AI website building, check out What is an AI Website Builder? Principles, Real Workflows, and a 4-Step Test. If you are rethinking the risks of relying solely on social media traffic, read The Meta One Link Fee Controversy: What Should Brands Relying on Social Traffic Do Now?.
Try Kanorio for free and make your site friendly to both AI and Google