Google’s leadership has said that people love AI Mode and that it sends more visitors to the open web than ever before. A new randomised experiment points the other way. If enquiries for your business start with a Google search, the details are worth knowing.
What the researchers did
Researchers at Northeastern University and the University of Pennsylvania published a draft paper called AI in Search Reduces Publisher Referrals Without Improving User Experience. They enrolled 1,444 volunteers and tracked the searches of 1,100 of them for seven days using a browser extension. A further 956 completed a survey afterwards.
Each person was randomly placed in one of three groups:
- Normal Google, with AI Overviews appearing whenever Google chose to show them
- Google with AI Overviews hidden
- Google with every search redirected into AI Mode
Random assignment matters. It means differences between the groups can be put down to the search experience rather than to the type of person in each group.
What they found
People forced into AI Mode, compared with standard Google Search:
- Clicked through to outside websites far less often. External click-through fell by 18.8 percentage points.
- Ran fewer search sessions each day, although each session lasted longer.
- Clicked less on every type of site the researchers checked, including news sites, Reddit and Wikipedia.
- Were more likely to use Bing, DuckDuckGo or Yahoo.
- Rated Google lower for usefulness, trust, satisfaction and personal relevance.
The group with AI Overviews hidden behaved differently. Those people clicked through to websites more often, and their opinion of Google Search did not measurably change. In plain terms, taking the AI summary away did not make searching worse.
How far to trust the results
The paper is a draft, and there are reasons to read it with care:
- The sample was American. Around 75% of participants were under 45, and most had some college education. UK searchers may behave differently.
- AI Mode was forced. Most people meet it as an option, so the study shows what happens when people are pushed into it, not what they choose.
- Hiding AI Overviews only partly worked. Google changed the code of its results page during the test, and overall about half of the overviews were hidden.
- News searches showed no clear difference between AI Mode and standard Google.
The direction still matches earlier evidence. A separate randomised experiment earlier this year found that AI Overviews cut outbound clicks by around 38% on the searches where they appeared. The exact size of the drop varies from study to study. The pattern of fewer clicks does not.
What to do about it
You cannot control how Google presents its results, but you can control what you measure and which pages earn their place. The steps below start with what you can check today and move towards longer-term changes.
Measure enquiries, not just visits
Open Google Search Console and look for pages where impressions hold steady but clicks fall. That gap is where AI summaries are most likely answering the question before anyone reaches your site. Then check whether calls, form submissions and quote requests have moved with the traffic. If they have not, the lost clicks may have been low-value ones. Good reporting and analytics makes this comparison straightforward.
Give informational pages a job a summary cannot do
A short definition is easy for AI to lift and answer on the spot. Pricing guidance, worked examples from real jobs, comparison tables, local detail and case studies give a reader a reason to click. They are also what make you worth citing when AI does mention a source.
Protect the searches that end in a phone call
People looking for a local supplier still need a name, an address, opening hours and proof that others have used you. A complete Google Business Profile with recent reviews does that work. Our Google Business optimisation and local SEO services cover it.
Look again at paid search for your highest-value terms
If organic clicks slip on searches that lead directly to sales, paid placement can fill part of the gap. The earlier randomised experiment found clicks on sponsored results stayed flat when an AI Overview appeared. Treat that as a reason to test, not as a promise. Google Ads management works best when it is tied to a clear cost per enquiry.
Build routes to customers that do not depend on one search box
An email list, repeat customers and people searching for your brand by name are all harder for an AI summary to intercept.
Do not rewrite your whole site in a panic
This is one draft study with a small US sample. The sensible response is to measure, adjust the pages that matter most, and keep search engine optimisation grounded in what brings in customers.
The short version
Forced into AI Mode, people clicked out to websites less and rated Google lower. Hiding AI Overviews sent more clicks to websites without harming the experience. Expect fewer clicks on informational searches, and judge your results by enquiries and sales instead.
Graig Upton
Graig Upton is a UK SEO and Google Ads consultant with 23 years of hands-on digital marketing experience. Google certified in Ads, Analytics and Conversions, he has helped businesses ranging from local providers to national brands - including Nando's and Financial Times - dominate search and scale their leads. Based in Preston, working with clients across the UK.