Scrape expectations 🤖

AI scraping, GEO and the challenge of being found

There’s a battle going on. 
 
Probably you’re aware of this already, but it’s worth us raising it with you this summer. AI is changing the way people search the web. Or rather, people are being made to search the web differently, with AI overviews being layered on top of traditional web search. 

Chatbots give answers rather than simply provide links. And this is wreaking havoc with organisations’ marketing strategies and tactics – or at least it means that they need to adapt fast. 

There are other skirmishes on the fringes. Some say you need to make your content as scrapable as possible. On the other hand, Reddit has tightened rules to discourage or stop indiscriminate scraping of its content. Cloudflare has introduced what looks like the beginnings of a new permission-based model. Elsewhere, products such as Anubis seek to deter or prevent web crawlers scraping without permission.  

There is renewed interest in how companies and their messages are found on the web, through SEO, or through its fresh-faced cousin GEO (generative engine optimisation). You may already be looking at ways to manage all of this. You definitely should be. But don’t lose the storytelling and creative muscle in your business. Without it, you could be left high and dry. 

The investment in AI search is still predominantly supply driven. The answers that AI search gives are sometimes unreliable, always unoriginal and often unremarkable. These are not at all the ingredients of, for example, distinctive thought leadership. However the AI/content battle shakes out, you will always need people to tell your stories to other people, alongside, perhaps, constructing them so that they can be scraped by machines. 

Until next time, 

The Collective Content Team 

From our blog

On mimetic desire and work

Senior editor Andrew Smith unpacks what Girard’s theory of mimetic desire, beloved of JD Vance and Peter Thiel, means for the workplace.

Podcast episode: Mis World: Misinformation, mistrust and missing the point of AI

CC MD Tony Hallett is joined by an expert panel including Nessa Keddo, Phil Szomszor and Harendra Kapur, live from a rooftop bar in London.

We like this

Word of the Moment: Gaslight-driven development

A term used in this blog post by Niki Prokopov, a ‘gaslight-driven development’ is a change made to an API because AI said so. “AI is not just using tools we gave it. It now has opinions about how these tools should’ve been made. And instead of asking nicely, it gaslights everybody into thinking that’s how it’s always been,” he writes.