The reason for “hallucinations,” or AI models’ propensity to fabricate responses that are factually incorrect, has been identified by OpenAI.
The entire industry is dealing with this serious issue, which significantly reduces the technology’s use. Even worse, experts have discovered that the issue is becoming worse as AI models become more powerful.
Therefore, even with high deployment costs, frontier AI models continue to make false assertions when presented with a prompt for which they are unsure of the answer.
Some experts contend that hallucinations are a natural part of the technology, but whether there is a solution to the issue is still up for dispute. In other words, massive language models may be a dead end in our effort to create AIs that can reliably understand true assertions.
In a report released last week, a team of OpenAI researchers sought to provide an answer. They believe that large language models hallucinate because they are encouraged to guess rather than confess they don’t know the answer.
Hallucinations persist because most assessments are assessed in a way that optimizes language models to be effective test takers, and guessing when uncertain increases test performance.
AI output is often assessed in a binary manner, rewarding the system when it provides the right answer and penalizing it when it provides the wrong one.
In other words, guessing is rewarded since it may be accurate, but an AI that acknowledges it is unsure of the answer will always be marked as incorrect.
As a result, rather than “acknowledging uncertainty,” LLMs are far more likely to hallucinate a response due to “natural statistical pressures.”
Most scoreboards emphasize and rate models based on accuracy, although errors are more harmful than abstentions, OpenAI wrote in an accompanying blog post.
In other words, OpenAI claims that it and all of its imitators in the business have made a serious fundamental mistake in how they have been training AI.
There will be a lot riding on whether the problem can be fixed moving ahead. According to OpenAI, “there is a straightforward fix” to the problem: Penalize confident mistakes more heavily than uncertainty, and grant partial credit for appropriate representations of uncertainty.
According to the blog post, assessments should make sure that their evaluations discourages guesswork in the future. Models will continue to learn to guess if correct guesses are consistently rewarded on the main scoreboards.
In the report, the company’s experts came to the conclusion that straightforward adjustments of mainstream assessments might realign incentives, rewarding legitimate expressions of doubt instead of penalizing them. In addition to removing obstacles to hallucination suppression, this can pave the way for further research on more complex language models, such as those with deeper pragmatic competence.
It is yet unclear how these changes to assessments will manifest in practice. Despite the company’s claims that its most recent GPT-5 model produces less hallucinations, consumers were mostly unsatisfied.
For the time being, the AI sector will need to keep addressing the issue as it explains tens of billions of dollars in capital expenditures and skyrocketing emissions.
According to OpenAI’s blog post, they are working hard to further reduce hallucinations, which remain a fundamental challenge for all large language models.
