Always-up-to-date AEO technical audit checklist

• An AEO technical audit checks four things: can a model find your site, fetch a page without a browser, resolve your brand as an entity, and pull a clean passage to quote.
• 10 checks decide your appearance in an answer at all — a missing Wikidata item, a blocked crawler, or a page that needs JavaScript to render can erase a brand from AI answers even when SEO looks fine.
• llms.txt helps when it reads like a short job index (the product description, getting started, docs, API, pricing) — equal-weight lists of 800-plus links fail this check even when every link works.
• Run the top 10 first if you're short on time, then work through the discovery, crawl, file, and on-page items that keep the top 10 working as those standards change.

You've probably already run a technical search engine optimization (SEO) audit. You've covered crawl budget, indexation, and Core Web Vitals. But ChatGPT, Perplexity, Google AI Overviews, and Gemini don't crawl, index, or rank the way Googlebot does, and a site that's technically sound for search can still be invisible, or misrepresented, in an answer.

Answer engine optimization (AEO) is a technical layer on top of the content work you've already done. It covers four things: finding your site, fetching a page without a browser, resolving your brand as an entity, and extracting a clean passage. This checklist covers the checks for those four jobs — the ones worth running now, and worth re-running as those standards change.

Note: AEO here means getting cited and represented accurately in answer engines. Model Context Protocol (MCP) servers, OAuth, and agent skills belong to a separate agent-experience audit.

The AEO technical audit, in order of impact

10 of these checks decide your appearance in an answer. The rest are the file, header, and identity checks that keep that working as those standards change. Start at the top if you're triaging.

1. Wikipedia or Wikidata entity

Answer engines resolve brand names through the public entity graph, and if your company has no Wikidata item and no Wikipedia article, assistants can attach your name to an unrelated film, animal, game, or open-source project that already has a Wikidata item. Wikipedia depends on notability, which a company can't decide to have. Wikidata is the check you can complete directly. Its inclusion threshold is lower and more mechanical than Wikipedia's, so marketing can create an item with the official website, founding date, and product names without waiting on a notability decision. That gives models something accurate to resolve to.

2. Category share of voice

Branded search rank and category rank are two different numbers. Pull the category queries you care about — the unbranded phrase you'd type for the job you do — from Search Console, and look at average position, impressions, and clicks. Sitting at the bottom of page one for the category term is a citation problem, because models often learn category membership from the sites that already occupy that search results page.

3. robots.txt AI crawler policy

GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and the rest need an explicit policy in robots.txt. An empty robots.txt effectively allows everyone, which is usually the outcome you want if you want to be cited. A blanket Disallow for AI bots is valid as a deliberate choice. A leftover security-plugin default is a defect.

4. robots.txt allows user-triggered agents

ChatGPT-User, Claude-User, and similar agents fetch a page because you asked a question, which is a different job than a training crawler scraping the web at scale. Training-crawler blocks and user-triggered-agent blocks are separate settings; check both. Keep those agent user agents allowed unless you have a specific legal reason not to.

5. Not blocked by bot detection

A robots.txt policy doesn't matter if a web application firewall (WAF), bot score, or "prove you're human" challenge trips on any client that isn't a browser. Fetch a handful of key URLs with a scripted client using an AI crawler's user agent string. You should get the same HTML a browser would get, with no interstitial in the way.

6. Content readable without JavaScript

Answer engines are generally worse at executing JavaScript than Googlebot is. Main copy, headings, and the links in your template need to exist in the raw HTML. A listing or hub page that injects every card client-side can be invisible as a hub to a model, even when each individual article page underneath it renders fine.

7. Structured data and schema markup

Structured data — Organization markup sitewide, plus Article or BlogPosting, Product or SoftwareApplication, frequently asked questions (FAQ), and Event where those templates exist — gives machines a way to read type, author, and date without guessing. (FAQ schema here is read for question-and-answer extraction quality by answer engines. It is not treated here as a Search rich-result check.) Valid JSON-LD (the structured-data format search and AI crawlers parse) that matches the visible page content is the target; duplicated Organization blocks and recording pages with no markup at all are the usual defects. Schema removes one layer of ambiguity for the system reading the page, even without an AI Overview placement.

8. llms.txt exists

A /llms.txt file at the site root is the current convention for a machine-readable index of the site's worth-reading pages. Ship one even while the standard is still settling — an empty file or a 404 at that path is a real finding, even when the rest of the site has no other defects.

9. llms.txt is a usable job index

A file that lists every sitemap URL is a dump, and agents tend to stop reading before they get through it. A usable llms.txt points at the pages that answer a job: the product description, getting started, docs home, API, pricing, and a handful of canonical explainers. Equal-weight lists of 800-plus links fail this check even when every link returns a 200.

10. Product story matches the live company

Your homepage, product pages, llms.txt, and schema should all describe the product you sell today. Retired taglines, discontinued SKUs, and old positioning left sitting in agent-readable files are the lines that get quoted back to a prospect. This is the AEO version of a content-freshness pass — a model repeats the story sitting in the HTML, current or not.

Discovery

These checks cover a model's path to the parts of your site that matter beyond the homepage.

• Developer resources findable by name. If you have docs, an API, or an MCP server, Search Console should show queries like "{brand} docs" and "{brand} api" ranking. Developer pages buried three clicks deep don't get cited.
• An Agent Resource Directory (ARD) or AI catalog at /.well-known is an optional, emerging convention. Note its presence or absence; don't fail an audit on absence alone until a standard settles.
• Listed in MCP registries. Registry listing is mostly an agent-experience concern. For an AEO audit, it matters mainly when your public story already talks about MCP and a crawler can't find a page that names the server's location.
• Open Graph (OG) tags on pages. OG title, description, and image are the card many crawlers use when they don't parse the full page. Every shareable template needs them set.

Crawl

Being allowed in robots.txt can still leave you unreachable.

• Agent crawler reachability. The same test as bot detection, applied broadly: scripted GET requests to the homepage, a blog post, and a docs page should each return 200 with real body text.
• One canonical docs host. A docs subdomain that returns an empty shell and then JavaScript-redirects to a new host strands every old link and every non-JS agent. The fix is a path-preserving 301 instead of a client-side redirect.
• HTTP Link headers to sitemap or docs. Link headers pointing at a sitemap or llms.txt are a nice extra that few agents depend on yet. Record their presence; don't prioritize them over the links already in your HTML.

Files

The file-level checks that keep llms.txt useful.

• Keep llms.txt under roughly 30,000 characters. Most agents truncate long files. A 100,000-character dump rarely gets finished; rewriting the file as a job index (see check 9) usually fixes size at the same time.
• llms.txt links resolve. Every URL in the file should return a 200, or a single obvious 301 to the live page. Stale sitemap mirrors fail this immediately.
• Markdown copies of important pages. Shipping a .md version alongside the HTML is optional. If you skip it, put the clean text into an llms-full.txt file instead.
• Agent discovery file or front desk. A dedicated "start here if you're an agent" file is still an emerging idea. llms.txt is the current stand-in for that role.
• Agent instruction, or "when to use." A short, quotable line at the top of llms.txt — "use this product when…" — is the line models tend to copy into routing decisions. A 900-link list with no opening instruction fails this check even if every link works.

On-page overlap with SEO

These items show up on a technical SEO checklist too. AEO tests a model's extraction of a clean answer; SEO tests Googlebot's indexing.

• Semantic heading tags. Headings are the chunk boundaries a model uses to extract a passage. Skip-level or unlabeled sections make extraction harder.
• No duplicate title tags. Titles are often the one string an answer engine shows as the citation label, so every page needs a unique one.
• H1 on every page. The H1 is the claim the rest of the passage is about.
• No thin content pages. A page with two lines of text — a bare recording page, for example — gives a model nothing worth quoting. Add a real summary.
• Noindex low-value content. Call-to-action (CTA) shells and form-gated pages should carry noindex so they can't end up as the page a model cites.
• Sitemap present and matching indexed pages. Answer engines still discover content partly through sitemaps and through Google itself, so submit the pages you want quoted.
• Crawlability, generally. This one's a carryover from the SEO checklist rather than a section to rebuild here — if a page isn't crawlable at all, nothing downstream matters.

Keep it current

These standards change often, and so does your own site. Treat this checklist as a scheduled rerun, since redesigns, new docs platforms, and positioning changes are when llms.txt, schema, and your product story drift out of sync.

This checklist sits next to two companion lists: a technical SEO checklist for the crawl-and-index side of things, and an agent-experience checklist for the MCP, OAuth, and agent-skills work that's a separate project from citation. Treat AEO as its own workstream. Citations behave differently as a performance signal than the clicks the rest of your SEO program chases. If you want a structured way to run both halves together — the crawl-and-index work and the AEO layer on top of it — the related reading below walks through the process.

Frequently asked questions

What makes this an AEO audit instead of just another technical SEO audit?

A technical SEO audit and an AEO audit test different things. A technical SEO audit tests Googlebot's crawl and index of a page; this one tests a model's ability to find the page, fetch it without a browser, resolve your brand as an entity, and pull a clean passage to cite. Several checks here — semantic headings, unique titles, H1 tags, thin content, sitemap accuracy — show up on a standard SEO checklist too, but here we're reading them for extraction, a different test than indexing. If a page isn't crawlable at all, nothing downstream on either checklist matters.

Does this checklist cover MCP servers, OAuth, or other agent-integration work?

No. AEO here is specifically about citation and accurate representation in answer engines. MCP, OAuth, and agent skills live on a separate agent-experience checklist. The one place they overlap is MCP registry listings: that's primarily an agent-experience item, but it becomes an AEO concern if your public story already mentions MCP and a crawler can't find a page that names the server's location.

I already have a Wikipedia page — do I still need a Wikidata item?

Yes, and it's worth doing regardless of the Wikipedia article's status. Answer engines resolve brand names through the public entity graph, and Wikidata is the piece you can control directly. Its inclusion threshold is lower and more mechanical than Wikipedia's, so marketing can create an item with the official website, founding date, and product names without waiting on Wikipedia's notability requirement. Without one, assistants can end up attaching your name to an unrelated film, animal, game, or open-source project that already has a Wikidata item.

Will blocking AI crawlers in robots.txt keep my content out of training data without hurting my citations?

Blocking AI crawlers in robots.txt doesn't automatically protect citations, because training crawlers and user-triggered agents are different bots with different jobs. ChatGPT-User, Claude-User, and similar fetch because you asked a question. Training crawlers and user-triggered agents need separate robots.txt rules; set both on purpose instead of trusting a plugin default. A permissive robots.txt still leaves a second check. A WAF or bot-detection challenge can block a crawler that robots.txt already allows, so it's worth fetching key URLs with a scripted client and an AI crawler user agent to confirm a match with the browser-rendered HTML, with no interstitial.

What should go in llms.txt, and how big can it get?

Keep it under roughly 30,000 characters, since most agents truncate longer files — a 100,000-character file rarely finishes loading. llms.txt should read like a job index: the product description, getting started, docs home, API, pricing, and a handful of canonical explainers. A list of 800-plus links fails this check even when every link resolves, and a 900-link list with no short "use this product when…" instruction at the top fails it too, because that line is the one models copy into routing decisions.

Do we need new tooling to run this audit, or can we use what we already have?

Most of it runs on tools a marketing or SEO team already has. Category share of voice and developer-resource findability come out of Search Console (average position, impressions, clicks for the right queries). Crawler reachability and bot-detection checks need a scripted client that sends GET requests with an AI crawler's user agent string and compares the response to the browser-rendered page. The newer pieces — a Wikidata item, an llms.txt file, JSON-LD that matches the visible page — take some one-time setup, but none of them require new infrastructure.

If I only have time for a handful of checks right now, where should I start?

Start with the top 10. They're the checks that decide your appearance in an answer. Run the discovery, crawl, file, and on-page items once the top 10 hold, and again whenever you ship a redesign, a new docs platform, or a positioning change.

Related reading

• Getting started with AEO — the case for treating AEO as its own workstream, measured by citations instead of clicks.
• Technical SEO audit: how-to and template — the spreadsheet-based process for running the crawl-and-index half of this work alongside AEO.
• SEO and GEO: visibility in search and LLMs — a look at running technical SEO and LLM optimization as one coordinated effort instead of two separate backlogs.