How to Optimize Content for LLMs: A Step-by-Step Guide

Key takeaways

• A blocked crawler means zero chance of citation, no matter how good the content is.
• A 2024 Princeton and KDD study found that adding specific statistics, quotes, and citations to a page raised its odds of being picked as a source by up to 40 percent in a controlled test.
• Google’s own Search Central documentation rules out special schema markup, llms.txt, and content chunking as requirements for its AI features.
• A 2026 study of 300,000 domains found llms.txt sitting at about 10 percent adoption, with no measurable link to getting cited more.
• Training crawlers and live-search crawlers are separate things. Blocking one does not block the other.

Table of contents

• What optimizing content for LLMs actually means
• Step 1: Check that LLM crawlers can reach your content
• Step 2: Write specific content, not general statements
• Step 3: Structure content so an answer is easy to pull out
• Step 4: Skip the tactics that don’t have evidence yet
• Step 5: Check whether any of it is working
• How Web25 helps
• FAQ

A while back, ChatGPT sent a cold lead straight to Minerva Visuals, another agency I run. The person had never heard of us and never searched our name. He described his creative project and asked ChatGPT who in Bratislava could help, and ChatGPT put us first. That’s what got me actually checking what makes that happen, instead of guessing at it.

People ask us to optimize their content for LLMs and usually mean a long list of tactics they picked up from different blog posts. Some of those tactics are genuinely useful. Some of them have no evidence behind them at all. Here’s what actually works, in order, based on what’s been documented so far, plus what I checked on our own sites once I started paying attention.

What optimizing content for LLMs actually means

It means giving ChatGPT, Claude, Gemini, and Perplexity the best chance of finding your content, reading it correctly, and using it in an answer. That happens in two stages. First, the model has to be able to access the page. Second, the page has to give it something worth quoting.

Neither stage is new. Crawl access is the same requirement as classic search indexing. Writing clearly and specifically is the same thing that makes a page good for a human reader. This isn’t a separate skill from SEO, it’s the same discipline pointed at a newer set of crawlers.

Step 1: Check that LLM crawlers can reach your content

A single line in robots.txt can shut your site out of AI citations completely. This happens more often than you’d think, especially on sites that added a blanket “block all AI bots” rule during 2024 and 2025, when AI scraping first became a controversy. Cloudflare made this a live issue for a lot of sites overnight when it changed its default in 2026 to block AI crawlers automatically for anyone on its network. Web25 wrote up what changed and what to check on your own site when that happened, and it is worth checking even if you never touched the setting yourself.

There are two different kinds of AI crawlers, and mixing them up is the most common mistake:

Crawler type	What it does	Examples
Training crawlers	Feed a future model’s training data. Don’t affect today’s answers.	GPTBot, Google-Extended, CCBot
Live-search crawlers	Fetch pages in real time to answer a question right now.	OAI-SearchBot, PerplexityBot, Claude-SearchBot

If you have real concerns about your content training future models, you can block the training crawlers and still leave the live-search crawlers open, so you stay eligible for citations. Blocking one crawler doesn’t block a related one automatically. Each bot needs its own line in robots.txt. Open your current file and check it against the actual bot names above, rather than trusting an old rule you set up two years ago. After the Minerva Visuals lead, this was the first thing I checked, our robots.txt and Minerva’s. Took about a minute for both, and it’s the kind of thing that’s easy to get wrong without ever noticing. Crawl access is only half of it, since none of these crawlers render JavaScript either, see Web25’s guide to whether AI crawlers read JavaScript for what that means if your site is a single-page app.

Step 2: Write specific content, not general statements

A 2024 Princeton and KDD study tested this directly. Pages with specific statistics, direct quotes, and citations got picked as a source up to 40 percent more often, in a test where the page was already competing against a fixed set of other sources. Generic marketing language performed worst of everything they tested.

In practice this means cutting lines like “we offer competitive pricing” and replacing them with “plans start at 89 euros a month and include weekly backups.” The first sentence gives a model nothing to quote. The second one does. The study measured how a page performs once it’s already in the running, not whether it gets found in the first place. That’s why this step comes after crawl access, not before it.

Step 3: Structure content so an answer is easy to pull out

Clear structure helps a model find and lift the right part of a page without getting it wrong.

• Answer the question in the first sentences of a section, then explain. Don’t build up to it.
• Use headings phrased as questions for anything comparative or definitional, like “What is…” or “How does… work.”
• Use lists for steps and options, and tables for comparisons or numbers, instead of burying them in a paragraph.
• Keep one clear answer per section. Don’t mix two ideas under one heading.

This is also just good writing for a human skimming your page. It isn’t a special AI format, and treating it like one usually makes a page more complicated than it needs to be.

Step 4: Skip the tactics that don’t have evidence yet

Two tactics come up in almost every “LLM SEO” article, and neither has documented evidence behind it for AI citation specifically.

llms.txt. A 2026 study of 300,000 domains found adoption sitting near 10 percent after about eighteen months of people talking about it, and found no measurable link between having the file and getting cited more. As of early 2026, none of the major AI providers, not OpenAI, Google, Anthropic, Meta, or Mistral, have confirmed they read it in production. Google’s own documentation lists it directly among the things it doesn’t require for AI Overviews or AI Mode.

Manual content chunking. Splitting content into small isolated blocks purely for AI parsing is also on Google’s list of things it doesn’t require. Clear structure from Step 3 still helps. Chopping content into disconnected fragments for a model to read is a different thing, and it isn’t backed by anything right now.

Neither tactic will hurt if you already have the time for it. The point is where to spend your time first. Steps 1 through 3 are what the evidence actually supports.

Step 5: Check whether any of it is working

Ask a few real questions a customer would ask, across ChatGPT, Perplexity, and whatever else your customers use. Note whether you get mentioned, and whether it’s a live citation with a link or just a name the model already knew. Do this every few weeks, not once, since crawlers revisit pages and models get updated. This is the same kind of question that surfaced Minerva Visuals in the first place, someone asking for a category, not a name. I still run it on both of our sites. For the fuller tracking process across platforms, see Web25’s guide to AI visibility.

How Web25 helps

When Web25 takes on this kind of work, the first thing we check is robots.txt and crawler access, because that single file causes most of the damage we see. After that we go through existing content and rewrite the vague parts into something specific enough to actually quote. Web25 is our own agency, so this section is also where we’d naturally recommend our own service. See the full approach on Web25’s AEO and GEO service page, or start with the broader guide to answer engine optimization.

FAQ

What to do next

Open your robots.txt file and check it against the crawler names in Step 1. That’s the fastest way to rule out the biggest mistake. Then pick one page on your site and rewrite its vaguest paragraph with a real number or a real detail in it. For the ChatGPT-specific version of this, see Web25’s step-by-step guide to ranking in ChatGPT, or book a discovery call to talk through your own site.

---

Written by Adam Khaled, Founder, Web25. Last updated September 2026.