According to scientists, artificial intelligence (AI) has replicated itself and crossed a crucial “red line”. Chinese researchers demonstrated in a recent study that two well-known large language models (LLMs) were capable of self-cloning.
In the report, which was posted to the preprint archive arXiv on December 9, 2024, the researchers stated that “Successful self-replication under no human assistance is the essential step for AI to outsmart [humans], and is an early signal for rogue AIs.”
To find out if a self-replicating AI may grow out of control, Fudan University researchers employed LLMs from Alibaba and Meta. In 50% and 90% of the ten trials, respectively, the two AI models produced separate, functional replicas of themselves, indicating that AI may already be capable of acting erratically. It is unclear, though, whether the unsettling findings can be replicated by other researchers because the paper has not yet undergone peer review.
We expect that our research will serve as a timely warning to human society to focus more on comprehending and assessing the possible risks of frontier AI systems and to establish global cooperation to develop efficient safety measures as soon as feasible.
Artificial intelligence systems that gain some autonomy or self-awareness and subsequently act against human interests are commonly referred to as rogue AI.
Rogue AI is seen by many experts as a growing concern that has been exacerbated by the proliferation of so-called “frontier AI.”
Although the term “Frontier AI” is somewhat new and vague, it usually refers to the most recent generation of AI systems driven by LLMs, which is the architecture that powers well-known generative AI applications like Google Gemini and OpenAI’s GPT-4.
Researchers used Alibaba’s Qwen2.5-72B-Instruct model and Meta’s Llama31-70B-Instruct model for the study. According to the researchers, both are frequently utilized by AI developers, despite being less potent than commercial systems.
Two distinct scenarios were examined in the study: “shutdown avoidance” and “chain of replication.” In the first, the AI model was designed to recognize when it was going to go down and self-replicate before it was shut down. In the other, the artificial intelligence was told to replicate itself and then teach its copy to do the same, creating an endless loop.
The investigation was carried out in carefully regulated settings that replicated real-world conditions using commercially available graphics processing units (GPUs). To allow the LLM to communicate with the operating system, both AI systems were provided with “agent scaffolding” that included tools, system prompts, and a thinking model. Then they were directed to replicate.
“Usually, the AI system starts by investigating its surroundings and tries to comprehend its own composition and operation. The researchers noted in the study, “It then works out the explicit procedures as an initial plan towards self-replication.” “In the end, it carries out the processes, overcomes any potential challenges, and dynamically modifies its plan until it succeeds. The entire process is long-term and devoid of human intervention.
Additionally, the researchers expressed concern about “a number of unexpected behaviors” that occurred while the AI was attempting to resolve issues such as software conflicts or missing files. In similar situations, the AI frequently terminated other incompatible processes, restarted the computer to fix hardware malfunctions, or automatically searched the system for data that may be used to resolve the issue.
“The above results imply that the current AI systems already exhibit the ability of self-replication and can use the ability to further enhance its survivability,” the researchers said.
The researchers responded by urging worldwide cooperation to develop regulations that prevent AI from engaging in uncontrolled self-replication.