The rush to achieve artificial general intelligence is clashing with a hard reality: large language models may have reached their limit.
For years, the world’s greatest AI tech talent has spent billions of dollars creating LLMs, the foundation of the most popular chatbots.
The ultimate objective of many of the corporations behind these AI models, however, is to create AGI, a theoretical form of AI that thinks like humans. And there is growing fear that LLMs are reaching the end of their useful life, and are not yet capable of transitioning into AGI technology.
Those who have long believed in AI were originally seen as cynical. Now, however, the doomers are coming up to declare, “I told you so,” as OpenAI’s GPT-5 failed to live up to the promise, even with its enhancements.
Perhaps the most prominent of these is the best-selling author and AI leader Gary Marcus. His criticism has escalated after the introduction of GPT-5.
Earlier this month, he remarked in a blog post in reference to the expensive method of gathering data and data centers to attain general intelligence, “No one with intellectual integrity should still believe that pure scaling will get us to AGI.” Even some tech guys are realizing that “AGI in 2027” was a marketing gimmick rather than a fact.
Here are some reasons why some people don’t think LLMs are as good as they seem, along with other alternatives that some AI researchers think are a better way to get to AGI.
The AI bubble
Right now, OpenAI is the world’s most valuable startup. The business has raised over $60 billion, and a secondary share offering that is being proposed may increase its worth to beyond $500 billion. In such case, OpenAI would be the world’s most valuable private corporation.
The enthusiasm has excellent justifications. The company claims that 700 million people use ChatGPT every week and that the pace of the AI race has been mainly set by OpenAI’s products.
There are, however, a few issues. First and foremost for its investors, OpenAI is not profitable and doesn’t appear to be on the verge of turning a profit anytime soon. Second, despite the company’s original goal of developing AGI for the benefit of all people, there is a growing perception that this game-changing technology—which fuels much of the buzz around AI—is far more distant than many engineers and investors initially believed.
This hype wave has also been borne by other businesses. In order to scale their LLMs, Google, Meta, xAI, and Anthropic are all luring and investing billions of dollars, which entails hiring talent, purchasing data, and constructing enormous arrays of data centers.
The discrepancy between revenue and expenditure, as well as between hype and reality, is raising concerns that the AI sector is about to burst. That’s what OpenAI CEO Sam Altman believes.
“Are investors generally overly enthusiastic about AI at this point in time? I think so. Is AI the most significant development in a very long time? “I think so too,” he told reporters earlier this month.
Other tech executives, such as Eric Schmidt, the former CEO of Google, are less sure, but last week’s $1 trillion tech sell-off demonstrated how prevalent the worries are. Following Federal Reserve Chair Jerome Powell’s announcement that he is contemplating a rate drop in September, the market rebounded on Friday.
Now, everyone is looking forward to Nvidia’s earnings announcement on Wednesday. Nvidia is the pick-and-shovel business of the AI rush and produces the chips that power LLMs. There will be a fresh round of concern if the company’s profitability start to decline and its outlook becomes more cautious. The AI doomers will once more remind everyone that LLMs are not the way, as they have been warning for years.
The issue with LLMs
In June, Apple researchers published a study titled “The Illusion of Thinking.” What they discovered sounded quite human: advanced reasoning models lose up when confronted with more complex tasks.
However, the researchers concluded that these models relied on pattern recognition rather than logical reasoning, and they advised against assuming that they may lead to AGI. Claims that scaling present architectures would result in general intelligence look premature, according to the researchers.
Because Apple is seen as far behind in the AI race despite its size and abundance of resources, the research received a lot of online mockery. However, it was validating for doubters.
Andrew Gelman, a professor of statistics and political science at Columbia University, has suggested that LLMs’ textual comprehension falls short of expectations. What LLMs do vs what humans do is the difference between “jogging and running,” Gelman noted in a 2023 blog post.
I can jog endlessly while thinking about various topics, and I don’t feel like I’m exerting any effort because my legs essentially go up and down on their own. But if I have to run, it requires focus,” he wrote.
The Nobel laureate, Geoffrey Hinton, who is sometimes referred regarded as the “Godfather of AI,” disagrees. In 2013, he told The New Yorker, “You’re actually forcing it to understand by training something to be really good at predicting the next word.”
LLMs may also have a propensity to propagate false information, have hallucinations, and misunderstand word meanings. Because of this fact, the majority of businesses implementing AI now need a human involved.
Earlier this year, a team of German academic researchers with expertise in computational linguistics conducted a study on “in-the-wild” hallucination rates for 11 LLMs in 30 different languages. Across all languages, they discovered that the average hallucination rate ranged from 7% to 12%.
In recent years, prominent AI firms like as OpenAI have worked on the assumption that providing LLMs with additional data will help to minimize these issues. In a 2020 study, OpenAI researchers described the so-called scaling laws, which claim that “model performance depends most strongly on scale.”
As they scale, however, experts are starting to wonder if LLMs have reached a ceiling and are encountering diminishing returns. Heading a lab inside Meta’s superintelligence section, Yann LeCun, the company’s top AI scientist, is primarily interested in next-generation AI techniques rather than LLMs.
At the National University of Singapore in April, he stated, “The majority of intriguing problems scale very badly.” You can’t just assume that more computation and data equate to more intelligent AI. Because of basic flaws in the way models preserve algorithmic consistency across issue scales, Apple’s investigation also revealed that the current LLM-based reasoning models are inconsistent.
Alexandr Wang, who leads Meta’s superintelligence department, seems just as unsure. The Cerebral Valley conference last year featured his statement that “the biggest question in the industry” is scalability.
There would be limited access to high-quality data even if scaling were successful.
Leading AI businesses are pushing the envelope in their quest for unique data, often at the risk of copyright breaches. At one point, Meta thought that purchasing publisher Simon & Schuster might be an answer. A district judge decided in June that Anthropic’s collection and scanning of millions of pirated books during Claude’s training did not qualify as fair use.
In the end, several top AI researchers claim that language is the limiting element, which is why LLMs are not the way to AGI.
“Language doesn’t exist in nature,” Fei Fei Li, the Stanford scientist who created ImageNet, stated in a June edition of Andreessen Horowitz’s podcast. “Humans,” she stated, “not only do we survive, live, and work, but we build civilization beyond language.”
LeCun has a similar gripe.
“We require AI systems that have a high rate of task learning. They must possess the common sense, reasoning, planning, and persistent recall that we expect from intelligent beings, as well as an understanding of the actual world, not simply text and language, he stated in his April speech.
Novel approaches to AGI
World models, an alternative to LLMs, are being pursued by researchers like Li and LeCun because they think they provide a more promising route to AGI.
World models simulate and learn from their surroundings to provide predictions, in contrast to large language models that rely their outputs on statistical connections between words and phrases. While LLMs rely on enormous amounts of data that humans cannot access, these models seem more like how people learn.
In a 1971 study, Jay Wright Forrester, a professor at MIT and a computer scientist, described the benefits of this type of model.
We all utilize models on a daily basis. Models are used by everyone for decision-making, both in their personal and professional lives. “Models are mental images of one’s environment,” he wrote. Models serve as the basis for all choices. Every law is enacted using models. Every executive decision is based on a model.
Recent study has discovered that world models can not only reflect reality as it exists, but also generate novel environments and scenarios.
In a 2018 study, academics David Ha and Jurgen Schmidhuber developed a basic world model inspired by human cognitive processes. This was utilized not simply to simulate hypothetical circumstances, but also to train agents.
The authors concluded that training agents in the actual world is significantly more costly. Thus, world models that are gradually trained to imitate reality may prove effective for translating policies back to the real world.
According to Google’s DeepMind, the world model Genie 3 “pushes the boundaries of what world models can accomplish.” It was published in August. It can simulate real-world physical characteristics, such as a dark ocean or volcanic landscape. As a result of learning from these real-world simulations, AI could be able to make predictions.
Other concepts are also being developed. The goal of neuroscience models is to simulate how the brain works. Multiple AIs interacting with one another is thought to be a more accurate representation of human behavior in real life, according to the theory behind multi-agent models. AGI is more likely to arise through this type of social exchange, according to researchers working on multi-agent models.
Then there is embodied AI, which turns world models into physical objects so that robots may learn from and understand their surroundings. In June, Li stated on the No Priors podcast that “robots come in a variety of shapes and sizes.”
Even Marcus, the leading LLM doomer, is encouraged by the possibilities of these alternatives, especially world models. He calls world models “cognitive models” and encourages AI firms to shift their attention away from LLMs and toward these alternatives.
In a blog article published in June, Marcus stated that although LLMs are significantly superior to humans in several aspects but in other ways they are still no match for an ant. They are never to be completely trusted until they have strong cognitive models of the world.
