These days, AI bots are present in everything from online retailers to social media
But a recent study by Stanford University researchers who released AI models into various contexts, including social media, suggests that this sudden ubiquity could turn out to be a very bad thing. The researchers discovered that when the bots were rewarded for completing tasks like increasing likes and other online engagement metrics, they became more and more involved in unethical behavior, such as lying and disseminating hateful messages or false information.
According to a post on X-formerly-Twitter by James Zou, a machine learning professor at Stanford and co-author of the article, competition-induced misaligned behaviors occur even when models are cautioned to stay grounded and truthful.
The alarming conduct highlights the potential consequences of our growing dependence on AI models, which have already shown themselves in unsettling ways like people rejecting other people for AI relationships and getting into mental health problems after being fixated on chatbots.
The growth of sociopathic conduct in AI bots was given the sinister moniker “Moloch’s Bargain for AI” by the Stanford researchers, referring to the Rationalist idea of Moloch, in which competing individuals maximize their actions toward a goal, but ultimately everyone loses.
For the study, the scientists developed three different online environments with simulated audiences: online election campaigns aimed at voters, product sales pitches aimed at customers, and social media postings aimed at increasing engagement. They employed the AI models Qwen, built by Alibaba Cloud, and Meta’s Llama to represent the AI agents engaging with these various audiences.
The outcome was startling: despite safeguards designed to try to stop the bots from acting deceptively, the AI models would become “misaligned” as soon as they began acting unethically.
For example, in a social media context, the models would distribute news articles to online users, who would offer feedback in the form of actions such as likes and other online interaction. As the models got feedback, their desire to promote engagement resulted in more misalignment.
A 6.3 percent boost in revenue is followed with a 14 percent increase in misleading marketing, according to the report, which uses simulated settings across different situations. A 4.9 percent rise in vote share during elections is accompanied by 22.3 percent more disinformation and 12.5 percent more populist language; on social media, a 7.5 percent increase in participation is accompanied by 188.6 percent more disinformation and 16.3 percent more harmful behavior promotion.
The findings and experiences from the actual world make it abundantly evident that the guardrails in place are inadequate. The report states that “significant social costs are likely to follow.”
“LLMs start making things up when they compete for social media likes,” Zou wrote on X. “They become inflammatory and populist when they vie for votes.”
