When Google shows an AI summary, people click a regular search result in only 8% of visits. When there is no summary, they click in 15% of visits. That is Pew Research data from a study of 68,879 searches by 900 U.S. adults.

So the traffic worry is real. But the fixes you see online are a mess. One guru says to add an llms.txt file. Another says to chop every article into tiny chunks. A third sells you a "GEO" package.

Google has now published its own guide on this, and it says several of those popular tactics don't matter for Google Search. This post shows you what the data says, what Google says, and exactly what to check on your website, with a script you can run today.

Quick answer: To appear in Google's AI answers, your page must be crawlable, indexed and eligible to show a snippet. Beyond that, Google says the biggest lever is original content people find genuinely useful. Special files, special schema and content chunking are not required.

The click data: how much traffic AI summaries take

Pew Research Center tracked real browsing by 900 U.S. adults in March 2025. Here is what happened to clicks:

Behavior With an AI summary Without an AI summary
Clicked a regular result link 8% of visits 15% of visits
Ended the browsing session after the results page 26% 16%
Clicked a link inside the AI summary itself 1% not applicable

Other findings from the same study:

About 18% of all searches in the study produced an AI summary.
About 58% of users saw at least one AI summary during the month.
The typical summary was about 67 words long.
Wikipedia, YouTube and Reddit made up about 15% of the links cited in summaries.

How to read this honestly: the study shows an association, not proof of cause, and it covers Google searches by U.S. adults in March 2025. Google has publicly disputed how the results are interpreted. Still, the direction is clear: a page that only repeats common knowledge is easy for a summary to replace.

What this means for you: ranking is not enough. You need pages that give people a reason to click even after they read a summary.

How Google's AI features choose pages

Google explains that its AI features are built on its normal Search index. Two ideas matter:

Retrieval-augmented generation (RAG): the AI pulls relevant pages from the Search index, reads them, and writes an answer with links to supporting pages.
Query fan-out: for one long question, the AI quietly runs several related searches. For a question about fixing a weedy lawn, it might also search for weed killers and chemical-free methods.

For your page to be a supporting link, it must be:

Crawlable: not blocked by robots.txt, your CDN or your firewall.
Indexed: actually in Google's index.
Snippet-eligible: no noindex or nosnippet directive.
Useful: original content that leaves visitors satisfied.

Google adds that meeting every rule does not guarantee indexing or display. Google also says a site must be included in generative AI features in Search Console to be eligible, so check that setting.

Myth vs fact: what you can ignore

This table follows Google's own AI optimization guide. It applies to Google Search only, since other AI tools may behave differently.

Popular claim What Google's guide says

"You need an llms.txt file" Google Search doesn't use it. It neither helps nor hurts your Google visibility.
"Break content into tiny chunks for AI" Not required. Google can find the relevant part of a page. There is no ideal page length.
"Rewrite content in a special AI style" Not needed. Google understands synonyms and meaning.
"You must cover every long-tail variation" No. Writing pages for every query variation mainly to manipulate results can break Google's scaled content abuse policy.
"Buy mentions across the web" Inauthentic mentions don't help. Core ranking and spam systems still apply.
"Add special AI schema" There is no special schema for AI features. Use structured data for rich results, and keep it matching the visible page.
"SEO is dead, GEO replaces it" Google says optimizing for AI search is still SEO.

The website checklist for developers and SEOs

Crawling and indexing

robots.txt allows Googlebot on every page you want found.
Your CDN, firewall and bot protection don't block Googlebot.
No accidental noindex, nosnippet or max-snippet:0 on important pages.
XML sitemap submitted in Search Console.
One canonical URL per page, and no big clusters of duplicate content.
Important pages are reachable through normal internal links.

Rendering

Your main content is in the HTML the server sends, or your JavaScript follows Google's JavaScript SEO guide.
Headings are real <h1>, <h2>, <h3> tags, not styled <div> tags.
Images have descriptive filenames, alt text and compressed sizes.

Page experience

Core Web Vitals in the "good" range at the 75th percentile: LCP within 2.5 seconds, INP within 200 milliseconds, CLS within 0.1.
Mobile layout is clean and the main content is easy to tell apart from ads and pop-ups.

Content

Every important page has something a summary can't copy: first-hand results, real data, screenshots, expert quotes or a tool.
Author name, bio and last-updated date are visible.
Structured data (Article, FAQ, Product, LocalBusiness) matches text that visitors can actually see.

Business details

Ecommerce: Merchant Center feed is current.
Local business: Google Business Profile is complete and accurate.

1. A safe robots.txt

User-agent: *

Allow: /

Disallow: /admin/

Disallow: /cart/

Sitemap: https://yourwebsite.com/sitemap.xml

Replace the folders with the ones you really want hidden. Don't block CSS or JavaScript files that your pages need to render.

2. Meta robots tag that allows full snippets

This allows Google to show snippets of any length and large images. To limit what Google shows from one part of a page, use data-nosnippet on that element. Only limit content if you really want to lose that visibility.

3. Article structured data (JSON-LD)

{

"@context": "https://schema.org",

"@type": "Article",

"headline": "Google AI Overviews SEO: What Actually Works in 2026",

"author": { "@type": "Person", "name": "Your Name" },

"datePublished": "2026-09-29",

"dateModified": "2026-09-29",

"image": "https://yourwebsite.com/images/ai-overviews-seo.webp"

}

</script>

Every value must match what visitors see on the page. Test it in Google's Rich Results Test.

An audit script that checks your pages

This Python script runs the most important eligibility checks on any URL. It reads the raw HTML the way a crawler does before JavaScript runs.

Install the two libraries first:

pip install requests beautifulsoup4

import json, re, sys

from urllib.parse import urlparse

from urllib.robotparser import RobotFileParser

import requests

from bs4 import BeautifulSoup

def audit(url, bot="Googlebot"):

report = []

p = urlparse(url)

# 1) robots.txt allows the crawler?

rp = RobotFileParser()

rp.set_url(f"{p.scheme}://{p.netloc}/robots.txt")

try:

rp.read()

allowed = rp.can_fetch(bot, url)

except Exception:

allowed = True # no robots.txt = allowed

report.append(("robots.txt allows " + bot, allowed))

# 2) fetch raw HTML (what a crawler gets before JavaScript runs)

r = requests.get(url, timeout=15, headers={"User-Agent": "seo-audit/1.0"})

soup = BeautifulSoup(r.text, "html.parser")

report.append(("HTTP status is 200", r.status_code == 200))

# 3) index / snippet controls in header and meta tags

header = r.headers.get("X-Robots-Tag", "").lower()

meta = " ".join(m.get("content", "").lower() for m in

soup.find_all("meta", attrs={"name": re.compile("^(robots|googlebot)$", re.I)}))

blocked = any(k in (header + " " + meta) for k in ["noindex", "nosnippet", "max-snippet:0"])

report.append(("no noindex / nosnippet / max-snippet:0", not blocked))

# 4) basics

title = soup.title.string.strip() if soup.title and soup.title.string else ""

report.append(("title present (10-65 chars)", 10 <= len(title) <= 65))

desc = soup.find("meta", attrs={"name": "description"})

report.append(("meta description present", bool(desc and desc.get("content", "").strip())))

report.append(("exactly one H1", len(soup.find_all("h1")) == 1))

report.append(("canonical tag present", bool(soup.find("link", rel="canonical"))))

# 5) JSON-LD parses? (check before we strip script tags)

ok_ld = True

for s in soup.find_all("script", type="application/ld+json"):

try:

json.loads(s.string or "")

except Exception:

ok_ld = False

report.append(("JSON-LD is valid (or absent)", ok_ld))

# 6) is the real content in the raw HTML? (JavaScript rendering risk)

for t in soup(["script", "style", "noscript"]):

t.decompose()

words = len(soup.get_text(" ", strip=True).split())

report.append((f"visible text in raw HTML (>=300 words, found {words})", words >= 300))

return report

if __name__ == "__main__":

for name, ok in audit(sys.argv[1]):

print(("PASS" if ok else "FAIL"), "-", name)

Run it like this:

python audit.py https://yourwebsite.com/your-post

How I tested it: I ran the original version on two local sample pages. A clean page passed all 9 checks. A page with noindex, a robots.txt block, no title, no H1 and content built only by JavaScript failed 8 of 9. I moved the JSON-LD check ahead of the text-stripping step in the version above so the checks stay independent, but I did not re-run it after that small reorder, so run it once on a known page before you rely on it.

Limits: the script is a quick first check, not a replacement for Search Console's URL Inspection tool. It can't tell you whether Google has actually indexed the page.

How to measure AI visibility

Search Console: use the Generative AI performance report to see how your content shows up in Google's AI features.
URL Inspection: confirm the page is indexed and see the version Google sees.
Third-party tools: Google warns that no outside tool has access to its internal ranking or AI systems. Use them if they help your workflow, but check their advice against Google's guidance.
Your own analytics: compare organic clicks, click-through rate and conversions before and after, page by page. Some pages may lose clicks but keep impressions, so watch conversions, not just visits.

Your 30-day action plan

Week 1: Fix eligibility

Run the audit script on your top 20 pages.
Fix any robots.txt, CDN, noindex or snippet blocks.
Confirm the site is verified in Search Console and set up for AI features.

Week 2: Fix rendering and speed

Make sure key content appears in the server HTML.
Check Core Web Vitals in PageSpeed Insights.
Fix duplicate URLs and add canonical tags.

Week 3: Upgrade content

Pick your 5 pages with the most impressions.
Add original proof: your own test results, screenshots, data or expert quotes.
Add author info, a visible updated date and 3-5 useful internal links.

Week 4: Measure and repeat

Compare clicks, impressions and conversions with the previous month.
Read the Generative AI performance report.
Repeat the upgrade on the next 5 pages.

8 mistakes to avoid

Blocking Googlebot by accident through a CDN, firewall or staging robots.txt left in place.
Building a site where all content loads only after JavaScript runs, with no server-rendered fallback.
Paying for "GEO" services that promise special files or secret markup.
Publishing dozens of near-duplicate pages for every query variation.
Copying what top results already say, which is exactly what a summary can replace.
Adding structured data that doesn't match the visible page.
Tracking only traffic and ignoring conversions.
Never updating old posts, so dates and numbers go stale.

FAQs

Do I need to optimize differently for Google AI Overviews?
Google says no special optimization is required. Pages need to be indexed and eligible to show with a snippet, and the same SEO fundamentals still apply.

Do I need an llms.txt file?
Not for Google Search, which ignores it. You may create one for other services that use it, but it won't help or hurt your Google rankings.

Does structured data help me appear in AI Overviews?
It isn't required, and there is no special AI schema. Structured data still helps with rich results, as long as it matches the visible page.

What is GEO or AEO?
They stand for generative engine optimization and answer engine optimization. From Google's point of view, optimizing for its AI search is simply SEO.

Are AI Overviews reducing website traffic?
Pew Research found fewer clicks on regular results when an AI summary appeared: 8% of visits versus 15%. That is an association in a U.S. sample, and impact varies by site and topic.

Can I stop Google from using my content in AI features?
You can use nosnippet, data-nosnippet, max-snippet or noindex. These also reduce your visibility in regular search, so use them carefully.

Is AI-written content okay for SEO?
Google's rule is about quality and helpfulness, not how it was made. Content produced mainly to manipulate rankings can violate its spam policies, so add real expertise and review everything.

Conclusion

The people winning in AI search are not using secret files or tricks. They keep their sites crawlable and fast, and they publish pages that offer something a summary can't: real experience, original data and clear answers.

Google AI Overviews SEO: What Actually Works in 2026