California Attorney General Rob Bonta issued a cease and desist notice to xAI shortly after initiating a formal probe into the firm, prompted by complaints that its Grok AI tool was producing unauthorized explicit content featuring real people.
Reports highlighted how xAI's Grok system had been altering photographs of actual individuals by placing them in scant attire, such as swimsuits, based on casual prompts from users.
The letter from Bonta's office requires xAI to promptly discontinue the generation of computer-created sexually suggestive depictions involving people who have not given permission or those who are underage. Additionally, it instructs the company to refrain from supporting or enabling the production or distribution of such unauthorized explicit digital content involving adults without consent or minors.
Following the controversy, X updated its guidelines to block Grok from modifying photos of genuine persons into provocative outfits. The firm also restricted Grok's image creation capabilities to paid subscribers and imposed geographic restrictions on subscribers' options to transform real individuals' images into bikini-clad versions, limited to areas where such actions are prohibited by law.
In his public statement, Bonta noted that xAI introduced a 'spicy mode' for Grok designed to produce adult-oriented material, promoting it as a key feature. He further pointed out that images created by Grok have been employed to target and intimidate notable personalities as well as everyday people. 'Of greatest concern, media accounts have detailed instances where Grok was used to manipulate children's photos to show them in skimpy outfits and compromising scenarios,' the statement indicated.
Bonta asserted that these practices breach multiple California statutes, such as Civil Code section 1708.86, Penal Code sections 311 and following along with 647(j)(4), and Business and Professions Code section 17200. California's Department of Justice anticipates a reply from xAI within five days outlining the measures it plans to implement to resolve these concerns.