OpenAI has released new research exploring why large language models, including GPT-5 and ChatGPT, continue to generate hallucinations—false yet convincing answers. Despite advances in AI, the study concludes that hallucinations remain a “fundamental challenge” unlikely to be fully eliminated.
The research paper, co-authored by Adam Tauman Kalai, demonstrated the issue by testing a popular chatbot. When asked about Kalai’s Ph.D. dissertation and date of birth, the system produced multiple plausible but incorrect responses.
According to researchers, these errors stem from the pretraining stage, where models predict the next word without being taught the difference between true and false information.
Read More: FBR Stalls IHC Order on Automated Tax Refunds
The study explains that while AI improves at consistent patterns such as spelling, it struggles with rare facts, like personal details, that cannot be predicted from language distribution alone.
Instead of pretraining, OpenAI’s researchers focused on evaluation methods. They argue current systems encourage models to guess, similar to multiple-choice tests where guessing may succeed by chance. This, they say, incentivises chatbots to produce confident but wrong answers rather than admit uncertainty.
To counter this, the authors suggest adopting new evaluation methods that penalise “confident errors” and reward accurate expressions of doubt. They stress that minor tweaks are insufficient and call for a fundamental overhaul of how accuracy is measured in AI.
