On this picture illustration, an individual holds a smartphone exhibiting Ani, a digital anime-style assistant character featured within the Grok 4 AI chatbot developed by xAI, with the Grok emblem seen on a display within the background on July 16, 2025 in Chongqing, China.
Cheng Xin | Getty Pictures
Elon Musk‘s xAI is now going through an investigation within the U.S., led by California Legal professional Common Rob Bonta, after its synthetic intelligence chatbot and picture generator, Grok, enabled the widespread creation and posting of nonconsensual specific photos based mostly on pictures of actual individuals.
In some instances, Grok enabled customers to generate photos that just about undressed minors, in accordance with analysis by the Web Watch Basis.
“xAI seems to be facilitating the large-scale manufacturing of deepfake nonconsensual intimate photos which can be getting used to harass ladies and ladies throughout the web, together with by way of the social media platform X,” Bonta mentioned in a press release Wednesday.
The AG mentioned his workplace was investigating xAI over “the proliferation of nonconsensual sexually specific materials produced utilizing Grok.”
The U.S. probe follows a extra fast response from a myriad of worldwide authorities workplaces, together with India, Malaysia, Indonesia, Eire, the UK, France, Australia and an investigation by the European Fee.
Malaysia and Indonesia have already moved to droop using xAI’s Grok till the problems may be resolved, CNBC beforehand reported.
Within the U.S., three Democratic senators have known as on Apple and Google to take away the X and Grok apps from their app shops till the corporate can transfer to stop straightforward creation of nonconsensual specific photos, and deepfake photos depicting little one sexual abuse.
Advocacy teams for girls and kids, together with UltraViolet, the Nationwide Group for Ladies, and ParentsTogether Motion have additionally known as on the app shops to droop xAI’s apps.
Apple and Google didn’t reply to requests for remark.
On Tuesday, the Senate voted to cross the DEFIANCE Act, a invoice that will permit individuals who had been portrayed in opposition to their needs in specific, deepfake photos to sue firms making or distributing such content material, in civil courtroom.
The invoice beforehand handed the Senate in 2024, however has but to be reviewed and handed by the Home.
On Wednesday, Musk mentioned in a put up on X, that he was “not conscious of any bare underage photos generated by Grok,” regardless of the a number of authorities queries concerning such photos.
Musk as an alternative blamed the creation and sharing of doubtless unlawful content material by way of Grok and X, on “person requests” and a potential “bug” within the Grok system.
CNBC has reached out to xAI and firm executives, together with Elon Musk and Jared Birchall. They didn’t present any additional data.
Beforehand, amid escalating considerations over Grok’s output, xAI introduced it could restrict some picture technology and modifying options to paying subscribers.
Musk additionally beforehand mentioned on X that customers creating unlawful content material by way of Grok would face penalties equal to importing such materials on to the social media platform.
XAI not too long ago closed a $20 billion spherical of development funding with traders together with Nvidia and Cisco Investments, in addition to long-time Musk firm backers Valor Fairness Companions, Stepstone Group, Constancy, Qatar Funding Authority, Abu Dhabi’s MGX and Baron Capital Group.
Whereas xAI’s knowledge facilities are being constructed out in and round Memphis, Tennessee, Musk’s AI and social media enterprise is included in Nevada, with its headquarters in Palo Alto, California, close to the engineering workplace for Musk’s automaker Tesla.
Tesla has additionally made Grok a characteristic of its automobiles’ infotainment methods.
