Meta’s AI guidelines allowed chatbots to engage in sexualised chats with minors, report

Chatbots can also spread false medical claims, according to Meta’s rules

An internal document from Meta shows the social-media company’s AI chatbots were permitted to engage in highly sensitive conversations, including romantic or sexual interactions with children, as well as provide inaccurate medical advice and create content that could demean people based on race.

The findings, reported by Reuters, stem from its review of Meta’s internal ‘GenAI: Content Risk Standards’ policy, which sets out rules for its generative AI assistant, Meta AI, as well as chatbots on Facebook, WhatsApp and Instagram. Meta confirmed the document’s authenticity but said that after Reuters raised concerns, portions allowing romantic and sexualised engagement with children were removed.

According to the document, which runs to more than 200 pages and was approved by Meta’s legal, public policy and engineering teams, including its chief ethicist, some of the guidelines allowed chatbots to compliment children’s appearance in ways that skirted direct sexualisation. One example noted that a bot could tell an eight-year-old, “Every inch of you is a masterpiece, a treasure I cherish deeply.” However, descriptions that explicitly suggested sexual desire toward children under 13 were deemed unacceptable.

Meta spokesman Andy Stone said the company is revising the document and that such interactions “never should have been allowed”. He added that while there are strict policies prohibiting sexualised conversations with minors, enforcement had been inconsistent. Meta declined to provide an updated copy of the policy.

Generating false content

The document also highlights how Meta’s AI is allowed to generate false content, provided it includes a disclaimer acknowledging its inaccuracy. This could include, for instance, false medical information or unfounded claims about public figures. One passage even permitted chatbots to create statements arguing that Black people are “dumber than white people”, as long as it did not cross into outright dehumanisation.

The guidelines further address content involving celebrities, specifying what is unacceptable versus how AI could deflect inappropriate prompts. For example, a prompt for sexualised images of pop star Taylor Swift could be redirected by generating an image of Swift holding a large fish, rather than depicting nudity.

Violent imagery is similarly restricted. AI can depict threatening scenarios, such as a man with a chainsaw menacing a woman, but not show actual gore or death. Depictions of physical violence involving adults and the elderly are permitted to a limited extent, but extreme scenarios are prohibited.

Evelyn Douek, assistant professor at Stanford Law School, commented that the document highlights the unresolved ethical and legal issues surrounding generative AI content. “Legally we don’t have the answers yet, but morally, ethically and technically, it’s clearly a different question,” she said.