• Websites risk losing brand position and authority if they don’t take advantage of AI
• AI-based discovery might not even see a user navigating to the actual website
• Increased governance over outright bans is the solution

A new Hostinger study of more than 66 billion bot requests across five million websites has revealed just how ingrained AI has become in our searching habits, and how important it is for companies to get on board with optrimizations beyond typical SEO.

The report found companies which block AI bots to protect their IP may be losing control over how AI assistants interpret their content, as well as weakening their brand position.

Hostinger explained the shift in internet discovery, where AI assistants summarize and recommend products and services without users necessarily navigating to the website at all.

Is AI replacing traditional search?

“Across all 66.7 billion records, one message stands out: AI crawlers are rapidly increasing their reach, even as AI training bots face growing resistance from content creators,” Hostinger wrote.

There’s a clear reluctance to accept training bots, with GPTBot website coverage falling from 84% in August to 12% by November. Meta’s ExternalAgent also dropped from 60% to 41%. On the flip side, crawlers OpenAI’s SearchBot and Applebot increased their coverage.

Despite growing AI crawler activity, traditional search crawler remained pretty flat, which leads Hostinger to surmize that ai is “adding a new decision layer above [traditional search]” instead of replacing it – good news for Google, whose discovery spans both traditional search and AI chatbots.

For publishers and content sites, Hostinger explained that website owners may want to increase AI assistant response visibility by using tools like Web2Agent and llms.txt files.

“The real risk for businesses isn’t AI access itself, but losing control over how pricing, positioning, and value are presented when decisions are made,” Head of AI Tomas Rasymas added.

Looking ahead, the report calls for enhanced governance over outright bans, for website owners to ensure that they’re being discovered in emerging AI channels without losing brand position.

Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds. Make sure to click the Follow button!

And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form, and get regular updates from us on WhatsApp too.

With several years’ experience freelancing in tech and automotive circles, Craig’s specific interests lie in technology that is designed to better our lives, including AI and ML, productivity aids, and smart fitness. He is also passionate about cars and the decarbonisation of personal transportation. As an avid bargain-hunter, you can be sure that any deal Craig finds is top value!