SACRAMENTO — California announced an investigation into Elon Musk’s artificial intelligence company xAI on Wednesday, with Gov. Gavin Newsom saying that the social media site owned by the billionaire is a “breeding ground for predators to spread nonconsenual sexually explicit AI deepfakes.”
Grok, the xAI chatbot, includes image-generation features that allow users to morph existing photos into new images. The newly created images are then posted publicly on X.
In some cases, users have created sexually explicit or nonconsensual images based on real people, including altered depictions that appear to show individuals partially or fully undressed. Others have generated images that appear to show minors, prompting criticism that there are not sufficient guardrails to prohibit the creation of child pornography.
The social media site has previously said “we take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary. Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”
Newsom called the sexualized images being created on the platform “vile.” Atty. Gen. Rob Bonta said his office will use “all tools at our disposal to keep Californians safe.”
“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” Bonta said in a statement Wednesday. “This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet. I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material.”
Newsom signed a pair of bills in 2024 that made it illegal to create, possess or distribute sexually charged images of minors even when they’re created with computers, not cameras. The measures took effect last year.
Assembly Bill 1831, authored by Assemblymember Marc Berman (D-Menlo Park), expanded the state’s child-porn prohibition to material that “contains a digitally altered or artificial-intelligence-generated depiction [of] what appears to be a person under 18 years of age” engaging in or simulating sexual conduct. Senate Bill 1381, authored by Sen. Aisha Wahab (D-Hayward), amended state law to more clearly prohibit using AI to create images of real children engaged in sexual conduct, or using children as models for digitally altered or AI-generated child pornography.
