Have you ever attempted to dispel conspiracy theories by showing someone that the moon landing wasn’t faked? According to research by psychology professor Thomas Costello of American University and David Rand of MIT Sloan School of Management, who conducted the study while holding a postdoctoral fellowship at MIT Sloan, you probably didn’t succeed, but ChatGPT might have better luck.
The researchers demonstrate in a recent paper “Durably reducing conspiracy beliefs through dialogues with AI” published in Science that people’s beliefs in conspiracy theories can be successfully reduced by large language models, and that these reductions last for at least two months. This finding provides new insights into the psychological mechanisms underlying the phenomenon as well as possible countermeasures to stop conspiracies from spreading.
Diving into the rabbit hole
For a long time, people have been fascinated and concerned by conspiracy theories, which are the ideas that certain events are the product of covert plots by influential individuals. Their steadfastness in the face of contradicting data has led to the conclusion that they satisfy ingrained psychological needs that make them immune to reason and reality. It’s common knowledge that once someone “falls down the rabbit hole,” it’s nearly impossible to get them back out.
However, that conclusion didn’t sit well with Rand, Costello, and their co-author professor Gordon Pennycook from Cornell University, who have done a great deal of research on the dissemination and absorption of misinformation. Rather, they thought a more straightforward explanation was in order.
“We wondered if it was possible that people simply hadn’t been exposed to compelling evidence disproving their theories,” Rand said. There is a wide range of conspiracy theories; the details and reasoning behind each theory vary amongst believers. Thus, you won’t be ready to refute them if you are attempting to refute the conspiracy but haven’t heard these specific arguments.”
To put it another way, two things would be necessary in order to effectively refute conspiracy theories: individualized arguments and extensive information, both of which are currently easily obtainable through generative AI.
Conspiracy conversations with GPT4
Using GPT-4 Turbo, OpenAI’s most sophisticated large language model, Costello, Pennycook, and Rand tested their theory by holding individualized, fact-based conversations with over 2,000 conspiracy believers.
A distinctive methodology used in the study enabled a close examination of each participant’s personal beliefs. First, participants were asked to name, in their own words, a conspiracy theory they believed in, along with the proof for that theory.
After that, GPT-4 Turbo used this data to create a customized synopsis of the participant’s beliefs and start a conversation. The AI’s task was to convince users that their beliefs were false, adjusting its approach in response to each participant’s distinct arguments and supporting data.
An approach that was impossible to test at scale before the technology’s development, these conversations lasted an average of 8.4 minutes and allowed the AI to directly address and refute the specific evidence supporting each person’s conspiratorial beliefs.
A noteworthy and long-lasting impact
The intervention’s outcomes were remarkable. Approximately one in four of the participants, who had all previously believed in the conspiracy, disavowed it after the AI conversation, with an average reduction in conspiracy theory belief of about 20%. Even two months after the conversation, the effect remained unchanged, demonstrating the durability of this impact.
The usefulness of the AI discussion was not restricted to any particular kind of conspiracy theory. It was successful in dispelling a variety of myths, including those that might have significant political and social significance, such as COVID-19 and election-related fraud in the United States in 2020.
Little variation was seen across demographic groups in the effectiveness of the intervention, even though it was less successful among participants who stated that the conspiracy was essential to their worldview.
Remarkably, the influence of the AI conversations went beyond simple shifts in perception. In relation to conspiracy theories, participants also showed changes in their behavioral intentions. They claimed to be more willing to participate in discussions dispelling conspiracy theories and more inclined to “unfollow” individuals who promoted them on the internet.
AI’s advantages and disadvantages
Although the technology may be used to persuade people to believe in conspiracies as well as to reject them, Costello, Pennycook, and Rand are cautious to emphasize the necessity for ongoing responsible AI deployment.
However, there is a good chance that AI will be used in ways that discourage conspiracy theories. Artificial intelligence (AI) tools, for instance, could be incorporated into search engines to provide precise results to users who enter terms related to conspiracies.
Evidence matters far more than we previously believed, according to this research, Pennycook said, as long as it is genuinely connected to people’s beliefs. Conspiracy theories alone are not the only implications of this: This strategy could theoretically disprove a wide range of beliefs that are supported by poor evidence.
Beyond the study’s specific conclusions, Costello noted that the methodology—which included using GPT-4 Turbo to screen respondents and analyze data in addition to conducting conversations—also demonstrates how large language models could transform social science research.
According to Costello, psychology research was previously restricted by its reliance on graduate students interviewing or treating other students. Then, we switched to platforms for online interviews and surveys, which gave us scale but eliminated the nuance. Artificial intelligence gives us the ability to have both.
These results pose a serious challenge to the theory that conspiracy theorists are irrational. Rather, they imply that many people are amenable to altering their opinions in the face of convincing and unique counter-evidence.
Costello stated that theories regarding conspiracies satisfying psychological needs resulted from the majority of conspiracy research being observational and correlational before we had access to AI. Our explanation is more commonsensical: most of the time, people were simply misinformed.
In addition, the public can visit a website to test out the intervention if they are interested in this ongoing project.
