David Chiu, the city attorney for San Francisco, isn’t asking nicely.
He’s demanding that Apple and Google delete 13 specific apps from their stores. The apps? They swap faces. More importantly, they generate non-consensual nude images using AI. The tech world calls it “nudification.” Chiu calls it sexual abuse. He wants the giants to stop profiting. He wants them to stop aiding. And he wants them to sever ties with the developers entirely.
This isn’t a request. It’s a cease-and-desist order.
The legal basis
California law is clear on this point.
Creating or supporting services that make deepfake pornography is prohibited. The letters seen by WIRED argue that by allowing in-app payments, Apple and Google are taking a cut of the harm. Millions of dollars, potentially, in fees. That’s not passive hosting. That’s participation.
“The fact that some of the most established technology companies are facilitating this is completely unacceptable,” Chiu said.
He’s not just talking about policy gaps. He’s talking about liability. If you profit from a crime, you’re complicit. The city’s position is that moderation isn’t working, and it has to get better immediately.
A transparent mess
It shouldn’t come as a shock to anyone at Cupertino or Mountain View.
For years, researchers have flagged these tools. The problem isn’t new. It’s persistent. Apps that promise face-swapping often hide their true purpose until after you download them. One app, with over a million downloads, lists styles like “bikini queen” and “calm busty” right on its landing page. Another boasts about “uncensored” content.
They dress themselves up as benign. Dual-use tools, the academics call it.
Benign on the surface. Dangerous underneath.
A study from Cornell and Georgetown in May found that 420 face-swap apps on both major stores could generate nudes. In 70 percent of tests, the apps had zero safeguards. None. They didn’t market themselves as nudification tools, so they slipped through the cracks of content moderation algorithms looking for explicit keywords in descriptions.
Is that accidental? Or just convenient for the bottom line?
The scale is terrifying
We’ve seen the numbers before. The Tech Transparency Project found about 100 such apps earlier this year. They’ve been downloaded collectively nearly half a billion times. They’ve likely generated $120 million.
Half a billion downloads.
The damage is real. Victims suffer humiliation, bullying, and severe mental health impacts. Some have contemplated suicide. The images appear in schools—90 incidents documented by previous reporting—targeting minors. It is a industrialized scale of violation.
“This industry has a horrific impact,” Chiu said. “There have been victims who’ve been suicidal. It’s absolutely horrifying.”
Do the platforms care?
Google’s Dan Jackson claims they’ve deleted hundreds of violating apps. He says they restrict search terms like “nudify” and act swiftly when violations are reported. They claim proactivity. They claim safety.
Apple didn’t comment.
Katie Paul at the Tech Transparency Project is less optimistic. She says the problem kept coming back. Worse each time.
“We didn’t think after the first报告 that we’d see this again,” she said. “Apple and Google promise trust and safety in their marketing. But that is just not playing out in reality.”
Chiu isn’t waiting for the next quarterly review. He says his office will consider all legal options if the apps don’t disappear.
He hopes the tech giants will do the right thing. He really does. But given the money on the line, and the track record so far, that hope feels thin. The apps are still there. The revenue is still flowing. And the laws are finally catching up to the technology, leaving the platforms with a simple choice: cut the profits or face the courts.
Who wants to bet which path they pick?
