Google-Extended and AI Overviews: What Gary Illyes Said and What Google’s Pages Say

What Google-Extended controls, why blocking it does not keep pages out of AI Overviews, what Gary Illyes said in August 2025, and Google’s plan to support the AIPREF training signal in 2027.
Google-Extended explained: what the robots.txt token controls for Gemini and what it does not change in Search Google-Extended explained: what the robots.txt token controls for Gemini and what it does not change in Search

Does blocking Google-Extended keep your site out of AI Overviews? Not according to Google. Its documentation says Google-Extended covers Gemini Apps and Vertex AI, and does not affect Google Search [3]. In an August 2025 interview, Google’s Gary Illyes said AI Overviews and AI Mode ground their answers in Google Search [1].

So the control that matters for Google’s search AI is still Googlebot, plus the snippet controls Google already offers [5]. This report sets out what Illyes said, what Google’s pages say, and what has changed since.

Updated 19 November 2026: rebuilt from Google’s documentation and named reports of the interview. The earlier headline said Google had “revealed” that search AI uses different training data. Illyes said the model “might” have been trained differently and that he did not know the details. We also removed quotes we could not trace to a source we could open, and added Google’s plan to support a new AI training signal in 2027.

In short: Google-Extended lets you opt out of Gemini model training and Gemini Apps grounding, without affecting Search [3]. AI Overviews and AI Mode use a custom Gemini model and draw on Google Search for grounding, Illyes said [1]. To limit what Search shows, Google points to nosnippet, data-nosnippet, max-snippet and noindex [5]. From early 2027, Google intends to support a shared AI training signal from the IETF [3].

What happened. Kenichi Suzuki and Rio Ichikawa of Faber Company in Japan interviewed Illyes in English. Search Engine Roundtable shared the video on 11 August 2025 [2]. Search Engine Journal published his answers on AI on 12 August 2025 [1].

Illyes said the model behind AI Overviews and AI Mode is a custom Gemini model. He added that this “might mean that it was trained differently”, but said he did not know the exact details [1].

Expert quote: “if you disallow Google Extended then Gemini is not going to ground for your site” — Gary Illyes, Google, interviewed by Kenichi Suzuki, as reported by SEJ on 12 August 2025 [1]

He also explained how grounding works. Gemini, AI Overviews and AI Mode all use Google Search for grounding, he said. They send several queries to Google Search, which returns results for each one [1]. Gemini does not fetch pages live from the web, according to SER’s summary [2].

Suzuki asked whether the training data for AI Overviews and AI Mode is collected by regular Googlebot rather than Google-Extended. Illyes did not answer that directly. He said that when grounding happens, “there’s no AI involved”, and that it is the generation that Google-Extended affects [1].

Why it matters. Many site owners added a Google-Extended block hoping to keep their content out of Google’s AI answers. The interview and the documentation both suggest that block does not reach AI Overviews or AI Mode.

What does Google’s documentation say about Google-Extended?

Google added Google-Extended on 28 September 2023, as a control over whether sites help improve Bard and Vertex AI [4]. In February 2024, after Bard became Gemini, it said Gemini Apps is affected and that Google-Extended does not affect Google Search. It updated the description again on 25 April 2025, based on publisher feedback [4].

The current page calls Google-Extended “a standalone product token”. It controls whether content Google crawls may be used for two things [3]:

  • training future generations of Gemini models that power Gemini Apps and the Vertex AI API for Gemini;
  • grounding answers in Gemini Apps and in Grounding with Google Search on Vertex AI.

It has no separate user agent. Google crawls with its existing user agents and reads the token only from robots.txt [3]. Google says it does not affect a site’s inclusion in Google Search and is not a ranking signal [3].

Google’s page on AI features in Search says the same from the other side. The control for how sites are crawled for Search is robots.txt for Googlebot. To limit what Search shows from your pages, it lists nosnippet, data-nosnippet, max-snippet and noindex [5]. Our guide to snippet controls shows how each one works.

Which control does what?aiseojournal.net

Pick a robots.txt or page control to see what Google says it changes, and what it does not.

Pick a control

What it changes: Google may not use your pages to train future Gemini models for Gemini Apps and the Vertex AI API, or to ground answers in Gemini Apps and Grounding with Google Search on Vertex AI.

What it does not change: Your inclusion or ranking in Google Search. Google does not list AI Overviews or AI Mode among the products it affects.

Source: Google crawler documentation, read 19 November 2026 [3]

What it changes: Google stops crawling those pages for Search. AI Overviews and AI Mode ground their answers in Google Search, Gary Illyes said.

What it does not change: Nothing is gained for Search: you lose normal results as well as AI features.

Source: Google AI features page [5]; Illyes interview via SEJ [1]

What it changes: Limits what Google Search shows from the page, including in AI features.

What it does not change: It also limits your normal search snippets, so test on a few pages first.

Source: Google AI features page [5]

What it changes: Removes the page from Google Search results.

What it does not change: There is no partial version: the page leaves Search completely.

Source: Google AI features page [5]

What it changes: Google says it intends to support the shared signal user-agent: __usage_ai-training in early 2027, for AI training preferences.

What it does not change: Google has not said it will change how AI Overviews or AI Mode ground their answers.

Source: Google crawler documentation [3]; IETF AIPREF drafts [6]

Based on Google's documentation as read on 19 November 2026. It does not check your site or give legal advice. aiseojournal.net

What did Illyes say about AI-generated content?

The interview also covered content written with AI. Illyes said he is not worried about the search index. But model training “definitely needs to figure out how to exclude content that was generated by AI”, or it ends up in a training loop [1].

On quality, he said the origin of content matters less than its accuracy. As long as quality is high, which today typically needs a human to review it, AI-generated content is fine for model training [1]. He then corrected Google’s own wording: “human created” is wrong, he said, and “it should be human curated” [1].

SEJ noted that this means real editorial oversight before publishing. Adding a “human reviewed” label to a page is not what Illyes suggested [1].

What else came up in the interview?

SER’s summary lists several other topics [2]. They include whether to block AI crawlers or wait and see, given possible revenue opportunities. They also include 404 pages and crawl budget, images and server resources, and ads in AI Mode. According to that summary, Illyes also said Google Search does not use social media shares and metrics [2].

The earlier version of this article quoted Illyes at length on these points. We could not find a written transcript we could open, so we have not repeated those quotes.

What has changed since: the AIPREF training signal

In the interview, Illyes mentioned ideas being discussed in an Internet Engineering Task Force working group about more crawling controls [2]. That group is AI Preferences, known as AIPREF. Its charter covers a vocabulary for AI-related preferences and ways to attach them to content [6].

The work is not finished. The vocabulary draft reached revision 08 on 14 September 2026, and the draft on attaching preferences reached revision 05 on 19 August 2026. Both are still working group documents, not published standards [6].

Google has now said it will act on it. Its crawler documentation says Google intends to support the AIPREF shared training preference signal, written as “user-agent: __usage_ai-training”, in early 2027 [3].

Worth knowing: the AIPREF signal is about training. Google’s note does not say it will change how AI Overviews or AI Mode ground their answers [3]. Until Google says more, assume the Search controls above still apply to those features.

What does Google-Extended mean for publishers? Our analysis

This is our view. Google-Extended is a narrow control. It answers one question: may Google use my pages to train Gemini and to ground Gemini Apps? It does not answer the question most publishers care about, which is whether Google’s search AI can summarise their pages [3] [5].

For that, the trade-off is stark. Blocking Googlebot removes you from Search. Snippet controls can limit what is shown, but they also limit your normal search snippets [5]. There is no Google switch that keeps you in classic results while keeping you out of AI Overviews.

The planned AIPREF support is a step towards a shared, cross-company signal [3]. But it covers training, not answers. Publishers who want payment or control over AI answers are looking at other routes, such as the Cloudflare pay per crawl model and publisher partnerships with AI companies.

Quick check: Open yoursite.com/robots.txt and search for “Google-Extended”. If it is there, note why it was added. If the goal was to stay out of AI Overviews, it is not doing that job.

What should you do?

  1. Decide what you want to control. Gemini training and Gemini Apps grounding, or what Search shows. They use different controls [3] [5].
  2. Keep Googlebot allowed unless you want to leave Google Search. AI Overviews and AI Mode draw on the Search index [1].
  3. Use snippet controls page by page, not site-wide, if some content should not be shown in Search [5].
  4. Watch for the AIPREF signal in early 2027, and add it if you want to opt out of training across companies that support it [3].

Our guide to managing AI crawlers covers the other companies’ tokens, and AI training opt-outs explained compares the options.

Frequently asked questions

A robots.txt product token. It controls whether Google may use your content to train future Gemini models for Gemini Apps and Vertex AI, and to ground answers in Gemini Apps and Grounding with Google Search on Vertex AI.

Google’s documentation does not list AI Overviews or AI Mode among the products it affects, and says it does not affect Google Search. Gary Illyes said AI Overviews and AI Mode ground their answers in Google Search.

No. Google says it does not affect a site’s inclusion in Google Search and is not used as a ranking signal.

Google points to nosnippet, data-nosnippet, max-snippet and noindex. These also affect your normal search results.

A shared way to state AI training preferences, being developed by an IETF working group. Google says it intends to support it in early 2027.

Where to go next

For Google’s wider position on AI search, read Google on AEO and GEO and our Search Central Live APAC 2025 report, where Illyes spoke a few weeks earlier. For how the two search features differ, see AI Overviews vs AI Mode.

Sources

4 comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use