This Media Briefing covers the latest in media trends for Digiday+ members and is distributed over email every Thursday at 10 a.m. ET. More from the series →

In this week’s Media Briefing, Digiday senior media editor Jessica Davies looks at how the AI compensation model is evolving beyond flat-fee licensing, with options like pay-per-crawl and pay-per-query starting to emerge as ways for publishers to monetize AI bot traffic.

• Pay-per-query vs. pay-per-crawl
• The media’s “traffic apocalypse,” Bloomberg cuts newsroom staff, and more.

New ways emerge to make money from AI bot traffic

AI compensation is evolving — and new models, not just publisher demands, are driving the shift beyond flat-fee licensing.

There’s already a slew of funding options hitting the market — or about to — that would compensate publishers based on a variety of functions. But part of the problem is the inconsistency in which AI is scraping data.

Cloudflare has moved to make AI crawler opt-outs a one-click default. IAB Tech Lab is considering pay-per-crawl and pay-per-query options as it begins work on developing its LLM Content Ingest framework. In the U.K. a Digital Pathfinder Project as a non-profit has been set up centered around using blockchain to track AI crawling, and automate royalty payments to content creators based on usage. 

The issue of AI crawlers is nuanced. Not all bot scraping is the same: some bots crawl for search and web indexing, some for RAG, some for training LLMs. “Not every scrape, not every crawl is equal,” said Toshit Panigrahi, co-founder and CEO of TollBit, a data marketplace for publishers and AI companies. 

Pay-per-query vs. pay-per-crawl

Anthony Katsur, IAB Tech Lab CEO is betting on pay-per-query with an eye toward scale. “One crawl could feed, hypothetically, 10,000 queries, 50,000 queries, and the publisher has just been paid once for the crawl, whereas with cost-per-query, every time your content informs a query result, the publisher gets paid,” said Katsur. The IAB Tech Lab aims to release its first iteration of the framework in the fall. 

He added that it’s a misconception that once a crawler mines a publisher’s page, it doesn’t recrawl. But the revisits aren’t enough to generate meaningful revenue for a publisher, he added.

Others are also skeptical of the pay-per-crawl model. “I think it will just incentivize the AI companies to crawl far less, which is sort of a good outcome, but doesn’t really solve the actual problem,” said Paul Bannister, chief strategy officer at Raptive. Pay-per-query or pay-per-inference is more attractive — but needs attribution, he noted.

Priced per content

Part of Katsur’s work is to create tiers of content, each of which has different licensing models, either for tiers of content set by the publisher or for all their content.

Tollbit’s bot paywall lets publishers charge crawlers different rates for accessing their content. “There’s tons more complexity here than just treating all bots as the same and just charging per crawl,” Panigrahi said.

Tollbit also uses pay-per-crawl and gives AI scrapers the option to pay a “toll” to access a publisher’s content. A web scraper or AI agent tries to go to a publisher’s webpage, gets redirected to Tollbit’s platform, and then is met with a transaction fee to access that page, set by the publisher. Tollbit has deals with publishers like Penske Media Corporation, Time and Trusted Media Brands. Two weeks ago, it also started giving publishers the option to charge AI bot crawlers per query – instead of per crawl – using Microsoft’s NLWeb protocol that debuted in May.

Prorata.ai offers another form of compensation model: it pays out 50% of all its revenue to publisher partners monthly, depending on how often their content powers AI responses.

Blockchain technology project experiment

Other experimental projects are underway, like one in the U.K. called Digital Pathfinder, which uses blockchain technology to make AI content licensing more secure and transparent for publishers. The concept is that blockchain can track how and where licensed content is used by AI systems. Then, smart contracts on the blockchain can automate royalty payments based on usage. 

The project was the brainchild of Paul Hood, a former News UK exec, who this week started a new role as head of publishing for the U.K .government. He set it up as a non-profit, and it will be continued by global media trade body FIPP, whose members include Axel Springer, The Economist and Creative UK, the national body for the creative industries.

Choosing the right AI compensation model

To create a real market for AI content licensing, three key pieces need to fall into place, according to Bannister:

1. Friction: moves like Cloudflare’s AI bot blocking tool make it more difficult for AI companies to quietly hoover up publisher content, forcing them to strike deals. 
2. A pricing model: whether it’s pay-per-crawl, pay-per-query, or something new, there needs to be a clear framework for compensation.
3. Attribution: publishers and platforms alike need attribution tools that can measure exactly how much content is being accessed and used in AI outputs — the foundation for any scalable payment system.

Until now, publishers have faced a harsh reality: LLMs have scraped their content for months, and those without the clout to strike big-money deals have seen no return.

No one’s claiming these payment models will radically shift publisher revenue overnight. But with infrastructure players like Cloudflare offering stronger defenses, there’s a growing sense that publisher compensation could evolve beyond flat-fee licensing into something more dynamic — and potentially more hopeful. 

“I think we’re going to see an explosion of good ideas in this space in the next few months,” said Bannister.

— Sara Guaglione, Digiday senior media reporter, contributed to this story.

What we’ve heard

“It’s a cost-cutting measure. They don’t need to pay heads of editorial content the same as EICs because they’ll say it’s a new role with different duties. But that person will have the same (if not more) responsibilities.”

– A former Condé Nast editor on the publisher hiring a “head of editorial content” to oversee Vogue’s daily operations.

Numbers to know

2: The number of years Linda Yaccarino served as CEO of X before resigning.

65%: The increase in AI training bots’ crawl activity over the past 6 months.

1,700:1: OpenAI’s crawl-to-referral ratio in June 2025.

$16 million: The amount CBS News parent Paramount Global is paying to resolve a lawsuit filed by President Donald Trump over a “60 Minutes” interview.

What we’ve covered

Publishers pull back their dependence on digital revenue

• Digiday’s survey found a big year-over-year difference in how much publishers depend on traditional channels vs. digital channels for revenue. 
• Last year, nearly two-thirds of publisher pros (64%) said their companies generated revenue mostly or entirely from digital channels. This year, that percentage fell to 51%.

Read the Digiday+ Research study here.

WTF is request duplication?

• Request duplication, or “bid duplication” can be seen as the costliest quirk in programmatic media trading, but sell-side advocates can argue it’s a legitimate monetization tactic. 
• It refers to the practice of sending the same ad opportunity — a single ad slot on a publisher’s site or app — to demand-side platforms multiple times.

Find out what this means for publishers here.

Generative AI is the new antitrust battleground for Google

• As generative AI reshapes how content is surfaced and monetized, regulators could scrutinize the next front: AI overviews, LLM training, and the impact on publisher traffic and revenue. 
• Google faces a third potential major antitrust battle, with an EU complaint about AI Overviews’ impact on independent publishers.

Read more here.

Yahoo takes cues from creator-focused platforms

• As Yahoo’s creator program matures, the company is increasingly evolving from a publisher into a creator platform in its own right.
• Yahoo now has 135 lifestyle-focused creators in its program. 
• In June 2025, Yahoo Creators generated its highest revenue and engagement since launch.

Read more about the creator-focused model Yahoo is building here.

Podcast companies turn to live events to capture growing advertiser spend

• As brands step up their podcast marketing spend, podcast companies are putting on more live events to meet the growing demand.
• Companies like SiriusXM and Vox Media have each hosted around 20 live podcast recordings so far this year.

Read more about how live podcast events are bringing in ad dollars here.

What we’re reading

Inside the media’s traffic apocalypse

Publishers from Bustle to The Washington Post are reeling from dramatic declines in traffic from platforms like Google and Facebook as AI-generated summaries explode, forcing them to focus even more on building direct relationships with audiences through subscriptions, newsletters and original content, Intelligencer reported.

Former Deadspin EIC on private equity chiseling away the media industry

Megan Greenwell, former editor in chief of Deadspin, talks to the Columbia Journalism Review about her new book chronicling private equity’s role in carving away parts of the media industry.

Google tries to win over publishers

Google is taking steps to help smooth relations with publishers, ranging from hiring new staff on its ad tech unit helping publishers sell ad space, and rolling out a new product to help them offer readers different ways of paying for content, The Information reported.

Bloomberg is cutting newsroom staff

Bloomberg is going through a round of layoffs as it reorganizes its newsroom, though it plans to end the year with a bigger head count, Business Insider reported.

G/O Media shuts down

In a statement posted to G/O Media’s website, CEO Jim Spanfeller announced the company is shutting down, after selling off most of its portfolio (which once included sites like The Onion, Jezebel and Deadspin).