Making your website machine-readable
AI models are becoming one way people find things. Machine-readable structure can make a public site easier for agents to inspect, but no checklist guarantees visibility in an AI answer. Here is what we know so far, based on Google recommendations and the new Lighthouse Agentic Browsing audits.
What the evidence covers
notiduck uses no LLMs in its verification pipeline. It runs rule-based checks over observable public signals: HTTP, DNS, TLS, timing, Lighthouse, page structure, metadata, accessibility, and machine-readable interfaces. Results reflect network, browser, and page state at check time. It checks whether automated tools can inspect the public site's structure and interfaces; it does not measure training data, rankings, mentions, or citations.
Search is changing. AI models are the new interface.
For twenty years, website optimization meant one thing: rank higher on Google. You optimized keywords, built backlinks, and chased Core Web Vitals. That model is being remodeled.
People increasingly use ChatGPT, Gemini, and Claude to find products, compare services, and complete tasks. These systems can use different representations of a public site, including screenshots, raw HTML, and the accessibility tree. Clear structure and interfaces make inspection easier; they do not determine whether a model recommends or cites a site.
Google recognized this shift in May 2026 with Lighthouse 13.3, introducing the Agentic Browsing category. It is a new set of automated checks that evaluate whether your website is constructed for machine interaction. Not as a 0 to 100 score, but as a pass ratio: how many of the applicable AI-readiness checks does your site actually pass.
At Google I/O 2026, Google announced a fundamental restructuring of Search toward conversational, AI-driven experiences. This is not a side experiment or a beta feature. It is the direction of the product. When the company that built the traditional search model starts rebuilding it around AI conversations, the writing is clear.
The same practices that make your site easier for automated tools to inspect also make it more accessible to humans. Semantic HTML, stable layouts, clean structure. The incentives finally align.
How AI models read your website
AI agents do not look at your site on a monitor. They may operate on machine-readable representations. The quality of these representations affects how readily a tool can inspect your content, but does not determine whether a model recommends or cites it. This is based on Google web.dev guidance on how agents interpret web content.
Screenshots
The agent captures a visual snapshot and uses a vision model to identify elements. Color, size, and proximity signal importance. A large Delete button gets more caution than a small Help link. But screenshot analysis is slow and token-expensive, making it a fallback when structure is unclear.
HTML DOM
The agent parses the DOM directly, reading element nesting, IDs, classes, and data attributes. A Buy Now button inside a product container is understood to belong to that product. Clean semantic HTML gives the agent a clear structural map of your content hierarchy.
Accessibility Tree
The browser distills the DOM into roles, names, and states of interactive elements. This gives automated tools a semantic map for inspecting and interacting with the page, while the exact representation and behavior vary by tool.
Page speed matters more for AI than it has in years
For the last few years, page speed was primarily a ranking signal and a human UX concern. With AI search, it becomes something else entirely: a resource constraint.
Some AI agents operate on a limited token budget per session. Lean HTML and prompt responses can make automated inspection more efficient, while slow responses add waiting time. The exact effect depends on the agent and its implementation; notiduck measures the response you can observe, not the agent's private budget.
Slow response times and bloated payloads can frustrate human visitors and make automated inspection less efficient. Fast TTFB, lean HTML, and minimal blocking resources remain sound performance practices; they are not proof that a model will discover or cite your site.
notiduck tracks TTFB from three geographic locations every 5 minutes. If your server response time regresses, you get an observable signal about a condition that may affect both people and automated tools.
What is Google Lighthouse Agentic Browsing?
Lighthouse has always been a website quality testing tool, covering performance, accessibility, SEO, and best practices. Version 13.3 added a fifth category: Agentic Browsing.
Unlike other Lighthouse categories, Agentic Browsing does not produce a weighted 0 to 100 score. Because the standards for the agentic web are still emerging, the current focus is on gathering data and providing actionable signals. The report displays a fractional pass ratio showing how many AI-readiness checks your site passes, along with specific pass or fail statuses for individual audits.
Source: Chrome for Developers, Lighthouse agentic browsing scoring
What it checks
- -Presence and quality of llms.txt file
- -WebMCP integration for forms and actions
- -Accessibility tree integrity and labeling
- -Cumulative Layout Shift for agent stability
What it does not penalize
- -Sites without AI-specific features still pass
- -Only applicable audits count toward the total
- -Informational audits do not lower your score
- -Category is marked experimental, not punitive
How to make your website AI-friendly
Actionable steps based on Google web.dev recommendations and the Lighthouse Agentic Browsing audit spec. None of this is guaranteed to improve your AI discoverability, but it is the best guidance available today.
Add an llms.txt file
Place a file named llms.txt at your domain root. It should contain an H1 heading, a concise description of what your site offers, and links to your most important pages. The file is intended as a compact reference for automated tools, although adoption and impact are not established. Keep it focused and updated. Lighthouse checks that the file exists, has an H1, is not too short, and contains links.
Use semantic HTML
Prefer <button> and <a> tags over styled <div> elements. Agents recognize semantic elements as interactive. If you must use divs, add appropriate role and tabindex attributes. Set cursor: pointer in CSS as an actionability signal.
Ensure a clean accessibility tree
Every interactive element needs a programmatic name. Use aria-label where text labels are insufficient. Link <label> tags to inputs with the for attribute. Avoid ghost elements or transparent overlays that hide interactive content from the accessibility tree. Lighthouse validates tree integrity as part of the agentic audit.
Stabilize your layout
Agents that take screenshots get confused when content shifts between the time they identify an element and the time they attempt to interact with it. Set explicit dimensions on images and embeds. Avoid injecting content above existing elements. Reduce cumulative layout shift not just for Core Web Vitals, but for agent reliability. This audit reuses the existing CLS score.
Consider WebMCP integration
WebMCP is a proposed standard that lets websites expose their functionality to AI agents. You can annotate HTML forms declaratively so agents understand what each field does, or register tools programmatically via navigator.modelContext.registerTool. This is still experimental and in early preview. Lighthouse validates WebMCP schema correctness when present; implementation is not a guarantee of agent use.
Track your AI readiness score over time
notiduck runs the Google Lighthouse Agentic Browsing test automatically as part of every audit cycle. You get a pass ratio displayed on your dashboard, trend history showing how your score changes, and alerts when your AI readiness regresses. Because the number of applicable checks varies per site, we show it as a fraction like 3 out of 3, not a percentage.
The GEO and AEO community: what to take seriously
A growing community has formed around Generative Engine Optimization (GEO) and Answer Engine Optimization (AEO). GEO focuses on making your content more likely to appear in AI-generated responses, while AEO targets the growing number of AI-powered answer engines and chatbots that pull information directly from websites to answer user questions. The idea is that you can optimize your content specifically for these systems. Some of the advice is solid. Some of it is not. As Ahrefs puts it, "GEO, LLMO, AEO... it's all just SEO."
One claimed difference from traditional SEO is speed. Search engine optimization takes months: you publish, wait for crawlers, build backlinks, and hope the algorithm notices. Some AI systems with real-time web search may re-crawl independently of traditional ranking cycles, but there is no general timetable for structural changes to influence an AI-generated recommendation. That uncertainty is also why the snake oil spreads so quickly.
Another difference: there are currently no sponsored results in AI chat interfaces. The major AI assistants do not sell placement in their organic responses. You are competing on content quality and machine-readability alone, not against companies with larger ad budgets. This will likely change as AI companies figure out monetization, but for now the playing field is genuinely open.
Claims like "your TTFB must be under 500ms from every location on Earth" or "every page must follow the BLUF format (Bottom Line Up Front) or AI models will ignore it" are circulating widely. There is no hard evidence to back these specific thresholds. No AI company has published research showing that a 501ms TTFB causes your site to be excluded from model training data, or that a paragraph without a BLUF sentence gets silently deprioritized.
Even llms.txt falls into this gray area. Lighthouse 13.3 now audits for it, which gives it a veneer of importance. But server logs from large hosts show AI bots are not downloading these files yet. Google's John Mueller has compared it to the keywords meta tag: something you claim your site is about, but not something the crawlers necessarily trust or use. We track it because it is part of the Agentic Browsing score, and being ready costs nothing. But do not expect it to move the needle on its own.
That said, a fast TTFB will never hurt you. It helps human users, it helps agents, and it helps your Lighthouse scores. The 500ms bar is a reasonable target even if the "must be under 500ms globally or else" framing is overstated.
BLUF is similar. Writing your conclusion first is a well-established communication technique from military and business writing long before AI existed. If you write your BLUF naturally, it will not look strange to human readers. It might actually help them. The idea that it is a secret signal to AI models is probably overthinking it, but the practice itself is harmless and often beneficial.
Our take: focus on what Google actually measures. The Lighthouse Agentic Browsing audits use rule-based checks over observable public signals. Results reflect network, browser, and page state at check time. Community trends are worth watching, but do not restructure your site around unverified claims.
Sources: Google web.dev, Build agent-friendly websites · Chrome for Developers, Lighthouse agentic browsing scoring · Google Blog, Search at I/O 2026 · Ahrefs, GEO, LLMO, AEO: It's All Just SEO · Ahrefs, What Is llms.txt, and Should You Care? · Search Engine Journal, Google Says llms.txt Comparable To Keywords Meta Tag
Frequently asked questions
What is an AI-friendly website?
An AI-friendly website has a public structure and interfaces that automated tools can inspect and interact with. This means providing machine-readable files like llms.txt, using semantic HTML, maintaining a clean accessibility tree, and implementing stable layouts that do not shift during loading.
What does notiduck measure and what does it not claim?
notiduck uses no LLMs in its verification pipeline. It measures observable properties of the public web surface: HTTP, DNS, TLS, timing, Lighthouse, page structure, metadata, accessibility, and machine-readable interfaces. It does not measure brand or keyword presence in model training data, predict LLM mentions, or guarantee AI citations.
How do I optimize my website for AI search?
Add an llms.txt file at your domain root with an H1 heading, a summary of your site, and key links. Use semantic HTML elements like button and a tags instead of divs. Ensure every interactive element has a programmatic name in the accessibility tree. Reduce cumulative layout shift. Consider implementing WebMCP to expose your site logic to AI agents.
What is Google Lighthouse Agentic Browsing?
It is a new experimental category in Lighthouse 13.3 that evaluates how well a website is constructed for machine interaction. It checks for llms.txt presence, WebMCP integration, accessibility tree integrity, and layout stability. The score is displayed as a pass ratio like 3 out of 3 rather than a 0 to 100 score.
What is llms.txt and why does it matter?
llms.txt is a proposed standard file placed at the root of a domain that provides AI models and crawlers with a concise, machine-readable summary of what a website offers. It should include an H1 heading, a brief description of the site, and links to key pages. It is intended to help automated tools understand a site without parsing the full HTML, but adoption and impact are not established.
What is WebMCP?
WebMCP is a proposed web standard that lets websites expose their functionality to AI agents through a machine-readable API. It allows agents to discover and execute actions on your site, such as filling forms, searching, or completing transactions. It can be implemented declaratively through HTML form annotations or programmatically via the navigator.modelContext.registerTool API.
Will making my site AI-friendly also improve it for human users?
Often. Semantic HTML can improve screen reader accessibility, stable layouts can reduce visual confusion, and clean accessibility trees can help assistive technologies. The practices overlap in useful ways, but they serve different tools and people.
Your competitors are already optimizing for AI.
Machine-readable, structurally clean, agent-accessible sites are easier for automated tools to inspect, but no service can guarantee an AI model recommendation or citation. notiduck monitors these public signals alongside performance, so you can see what changed.
Get started for free