I don’t want to admit it, but I did spend a lot of money online this holiday shopping season. And unsurprisingly, some of those purchases didn’t meet my expectations. A photobook I bought was damaged in transit, so I snapped a few pictures, emailed them to the merchant, and got a refund. Online shopping platforms have long depended on photos submitted by customers to confirm that refund requests are legitimate. But generative AI is now starting to break that system.

A Pinch Too Suspicious

On the Chinese social media app RedNote, WIRED found at least a dozen posts from ecommerce sellers and customer service representatives complaining about allegedly AI-generated refund claims they’ve received. In one case, a customer complained that the bed sheet they purchased was torn to pieces, but the Chinese characters on the shipping label looked like gibberish. In another, the buyer sent a picture of a coffee mug with cracks that looked like paper tears. “This is a ceramic cup, not a cardboard cup. Who could tear apart a ceramic cup into layers like this?” the seller wrote.

The merchants reported that there are a few product categories where AI-generated damage photos are being abused the most: fresh groceries, low-cost beauty products, and fragile items like ceramic cups. Sellers often don’t ask customers to return these goods before issuing a refund, making them more prone to return scams.

In November, a merchant who sells live crabs on Douyin, the Chinese version of TikTok, received a photo from a customer that made it look like most of the crabs she bought arrived already dead, while two others had escaped. The buyer even sent videos showing the dead crabs being poked by a human finger. But something was off.

“My family has farmed crabs for over 30 years. We’ve never seen a dead crab whose legs are pointing up,” Gao Jing, the seller, said in a video she later posted on Douyin. But what ultimately gave away the con was the sexes of the crabs. There were two males and four females in the first video, while the second clip had three males and three females. One of them also had nine instead of eight legs.

Gao later reported the fraud to the police, who determined the videos were indeed fabricated and detained the buyer for eight days, according to a police notice Gao shared online. The case drew widespread attention on Chinese social media, in part because it was the first known AI refund scam of its kind to trigger a regulatory response.

Lowering Barriers

This problem isn’t unique to China. Forter, a New York-based fraud detection company, estimates that AI-doctored images used in refund claims have increased by more than 15 percent since the start of the year, and are continuing to rise globally.

“This trend started in mid-2024, but has accelerated over the past year as image-generation tools have become widely accessible and incredibly easy to use.” says Michael Reitblat, CEO and cofounder of Forter. He adds that the AI doesn’t have to get everything right, as frontline retail workers and refund review teams may not have the time to closely scrutinize each picture.

Reitblat says organized crime groups are using the same tactics as individuals to orchestrate refund fraud at scale. In one case, he says, scammers submitted over a million dollars worth of refund claims using AI-altered images that showed cracks or dents in various home goods. The requests were submitted in a tight time window, seemingly to overwhelm the system, and the fraudsters also used rotating IP addresses to conceal their identity.

Some sellers are using AI to fight back against AI. A Chinese toy seller demonstrated to WIRED how they feed refund requests to an AI chatbot to analyze if the photos are doctored. But these tools are far from perfect right now. Plus, even with supposed confirmation from a chatbot, ecommerce platforms won’t necessarily always side with the seller. Reitblat warns that retailers might eventually respond by tightening their return policies, but that would hurt the shopping experience of customers acting in good faith.

This story echoes an earlier backlash that happened on Chinese digital marketplaces, when sellers were the ones being criticized for using AI-generated product photos. Shoppers complained that buying online had become like gambling, and you never knew if the product that arrived would actually look like the pictures.

But really, these trends are two sides of the same problem: Ecommerce relies heavily on trust, and widespread availability of AI is making it increasingly difficult to operate under the assumption that the majority of people are honest actors. Existing guardrails, like AI watermarks, are often too easy to remove. If shopping platforms want systems built for humans to keep working, they’ll need to figure out how to respond, whether with new verification rules, revised refund policies, or better accountability mechanisms for AI-enabled scams.

This is an edition of Zeyi Yang and Louise Matsakis’ Made in China newsletter. Read previous newsletters here.