This case could hold AI companies accountable for deepfake nude photos

Three Jane Does in Tennessee are suing Elon Musk’s xAI after someone used Grok to generate sexually explicit images of them. And to do it, they used real, clothed photos of the girls.

It’s a case that highlights a growing problem with the growth of AI: Anyone can take a regular photo of a teenage girl at her homecoming dance or posing with her friends and turn it into child pornography. And it’s happening a lot.

New York Times opinion writer Jessica Grose recently wrote about it and The Show spoke with her more about the problem, potential solutions and this class action lawsuit out of Tennessee.

Full conversation

JESSICA GROSE: I was interested in the lawsuit because it took a bit of a novel approach to get around something called Section 230. Section 230 is a law that was written in the ‘90s. It's a federal law and it basically says internet companies are not responsible for anything that individual users post on their platform. So this lawsuit is arguing that AI is co-creating images with the posters. Those images would not exist without AI, which is owned in this case by Grok.

And so Section 230 cannot apply to these circumstances. And it's working its way through the system. I don't know if, you know, judges or juries are going to agree with this approach, but I thought it was really smart because, you know, I'm just a layperson, but in my mind, AI is co-creating these images. They are not real. These young women did not take nude pictures of themselves. They were simply existing in the world. They were going to the beach, they were appearing in a yearbook.

LAUREN GILGER: Right. So a girl today can post something on social media or even just be in a yearbook picture doing something normal, clothed, by the way, and these chatbots, these AI images are turning those into nude photos. It sounds like in this case there were 18 underage girls that the perpetrator had allegedly posted nude photos of to — to Discord.

JESSICA GROSE: That is correct. And in the days since I published that piece, I've heard from, you know, a handful of parents who said, I'm going through this right now with my daughter at her high school. So this is not something that is, you know, a rare regrettable incident that is being prosecuted. This is commonplace, unfortunately, now.

And it's I actually don't know anyone who is sort of tracking how commonplace, but you know, I imagine many of the victims of this behavior, and to be clear it's not just girls, this happens to boys as well, many may be too humiliated to bring it forward or don't even know that it's happening. Because you have to be alerted to the fact that it's happening in the first place.

LAUREN GILGER: Right. These girls in this case were identified, their schools were named. How did in this lawsuit, at least the victims here kind of describe the impact on their lives? You describe it as their lives were narrowed.

JESSICA GROSE: So they were completely stress and anxious, humiliated. They would go to school worrying had anyone seen the images, do they think differently of them, do people think they're real? I mean they sort of assumed that many people did think that they were real even if they were not. But it really made them paranoid, I would say.

And is there a term for appropriately paranoid, right? Because this did actually happen. Yeah. So — so paranoid sort of implies unreasonable, but they had a sort of reasonable paranoia after the fact that this had happened and — and just a fear that all of their peers had not only seen the images but were judging them for these images.

LAUREN GILGER: There was a case in New Mexico as well where a jury found Meta liable for kind of misleading users about their safety practices when it comes to protecting kids from, like, child predators essentially.

Does this all go hand in hand in terms of what you're watching on this media company, social media company front and how it's impacting kids?

JESSICA GROSE: I do believe that more people, because that was a jury trial, so more everyday people are waking up to the fact that these social media companies are incredibly powerful and that they have more of a responsibility to keep young people safe than they have been accepting for years. And, you know, I think it's unreasonable to think nothing bad will ever happen on these platforms.

But, you know, in the discovery for the trials that have been going on recently, not just with Meta but also YouTube, there's been ample evidence that these companies knew what they were doing was damaging and they did not really take much action to fix the problem because it would hurt engagement.

I mean this is so skin-crawling to say out loud, but a — a Facebook whistleblower, someone who used to work for Meta, told me that, you know, if what you are going to these sites for is to look at underage girls, it will continue to serve you underage girls in suggestive pictures because it will understand that that's what you like. It's very effective at doing that.

You know, I like cooking. It is very effective at serving me many, many, many recipes. Right? I'm not inside these companies, but from the people who were and have, you know, admirably come forward to talk about what it is like in there, it's pretty clear that they could be doing a lot more to prevent this kind of behavior.

LAUREN GILGER: So what would be a potential solution here from your point of view, Jessica? Like Congress has not passed any legislation to reform that Section 230 of the Communications Decency Act, right, that you mentioned at the beginning. And so we're seeing these cases sort of take various novel approaches. But what would fix this?

JESSICA GROSE: Well, you know, like I said, I don't think that anything is going to fix it 100%. I would like to see them basically ban all deepfake nude images of real people. I can't see a compelling reason why those should be created and posted. You can create them alone in your home.

But then the second you post them, putting it of underage people, that's already illegal. You're just seeing people be prosecuted for it. Which the scale of it is so big, you cannot possibly begin to prosecute all the people who have made child sex abuse materials out of pictures using AI, right?

So it's already illegal, it's just you want to stop it before it happens. Because once it's happened, the damage is already been done. Those images are out there. And so that should also apply to adults. These companies should, you know, they're always talking about their incredible capabilities and I think about it with AI all the time as it's like we can do anything and change the world and destroy the job market.

But if you ask us to stop deepfake nudes, our hands are tied. That I just feel like we hear that constantly and so if their AI is so magnificently capable, could they use it to prevent this?

So I would like to see the companies step up themselves. I would like to see laws preventing the dissemination of any of this, not just for young people. I would like to see a rewriting of Section 230. I am not someone who writes policy, but I would like to see that legislation hashed out in a more modern way.

I mean the way the internet was, I believe, in 1996 when this was passed, is just a completely different universe. I don't think it will be easy to come up with legislation that I think sort of balances those needs to protect children but also protect free speech. But they've got to do something.

LAUREN GILGER: Jessica Grose, opinion writer for the New York Times joining us to talk about this. Jessica, thank you very much for coming on. I appreciate it.

JESSICA GROSE: My pleasure. Thanks for having me.