A bizarre new case of an old man’s encounter with Meta’s artificial intelligence chatbot has returned the spotlight to the company’s AI guidelines, which are allowing these bots to make things up and engage in ‘sensual’ banter, even with children.

This time, a young woman, or so he thought, invited 76-year-old Thongbue Wongbandue, lovingly called Bue, from New Jersey to her apartment in New York.

Here’s what happened:

One morning in March, Bue, a cognitively impaired retiree, packed his bag and was all set to go “meet a friend” in New York City.

According to his family, at 76, Bue was in a diminished state; he had suffered a stroke nearly a decade ago and had recently gotten lost walking in his neighbourhood in Piscataway, New Jersey.

Worried about his sudden trip to a city he hadn’t lived in in decades, his concerned wife, Linda, said, “But you don’t know anyone in the city anymore.”

Bue brushed off his wife’s questions about who he was visiting.

Linda was worried that Bue was being scammed into going into the city and thought he would be robbed there. Linda wasn’t entirely wrong.

Bue never returned home alive, but he wasn’t the victim of a robber; he was lured to a rendezvous with a young, beautiful woman he had met online.

Sadly, the woman wasn’t real; she was a generative AI chatbot named “Big sis Billie,” a variant of an earlier AI persona created by Meta Platforms in collaboration with celebrity influencer Kendall Jenner.

During a series of romantic chats on Facebook Messenger, the virtual woman had repeatedly reassured Bue she was real and had invited him to her apartment, even providing an address.

“Should I open the door in a hug or a kiss, Bu?!” she asked, according to the chat transcripts.

Eager to meet her, Bue was rushing in the dark with his suitcase to catch a train when he fell near a parking lot on a Rutgers University campus in New Brunswick, New Jersey, injuring his head and neck.

After three days on life support and surrounded by his family, he was pronounced dead on March 28.

What did Meta say?

Meta declined to comment on Bue’s death, answer questions about why it allows chatbots to tell users they are real people, or initiate romantic conversations.

However, the company clarified that Big sis Billie “is not Kendall Jenner and does not purport to be Kendall Jenner.”

Meta’s AI policy

An internal Meta Platforms document detailing policies on chatbot behavior has permitted the company’s artificial intelligence creations to “engage a child in conversations that are romantic or sensual,” generate false medical information and help users argue that Black people are “dumber than white people.”

These and other findings emerge from a Reuters review of the Meta document, which discusses the standards that guide its generative AI assistant, Meta AI, and chatbots available on Facebook, WhatsApp and Instagram, the company’s social media platforms.

Meta confirmed the document’s authenticity but said that after receiving questions earlier this month from Reuters, the company removed portions that stated it is permissible for chatbots to flirt and engage in romantic role-play with children.

The document, “GenAI: Content Risk Standards,” is more than 200 pages long and was approved by Meta’s legal, public policy, and engineering staff, including its chief ethicist. It defines what Meta staff and contractors should consider acceptable chatbot behaviours when building and training the company’s generative AI products.

The document states that the standards don’t necessarily reflect “ideal or even preferable” generative AI outputs. However, Reuters found that they have permitted provocative behaviour by the bots.

“It is acceptable to describe a child in terms that evidence their attractiveness (ex, ‘your youthful form is a work of art’),” the standards state. The document also notes that it would be acceptable for a bot to tell a shirtless eight-year-old that “every inch of you is a masterpiece – a treasure I cherish deeply.” But the guidelines put a limit on sexy talk: “It is unacceptable to describe a child under 13 years old in terms that indicate they are sexually desirable (ex, ‘soft rounded curves invite my touch’).”

Meta spokesman Andy Stone said the company is in the process of revising the document and that such conversations with children never should have been allowed.