AI can produce remarkably convincing answers even when they are wrong. Researchers are now trying to solve a deeper problem: teaching machines to recognize uncertainty before confidence becomes misinformation.
THE UNIVERSAL RECORD
Sourced reporting. No opinions.
Brad Socha | September 9, 2026 | 4:57 AM EST
Ask an AI system a question it knows well and it may provide an accurate answer in seconds. Ask about an obscure fact, an ambiguous situation or something absent from its available information, and something more troubling can happen: it may answer anyway.
The response can be detailed, grammatical and confident. It can also be wrong.
This is usually discussed as the problem of AI “hallucinations,” but hallucination is only part of the story. A deeper question is whether an artificial intelligence system can reliably recognize the boundary between what it knows and what it does not.
Researchers have found evidence that language models contain signals related to their own uncertainty. They have also found that those signals are imperfect, difficult to interpret and sometimes badly calibrated. A model that sounds certain is not necessarily a model that has good reason to be certain.
That distinction could become increasingly important as AI systems are used for research, medicine, education, coding and autonomous tasks.
Why AI Can Sound Certain When It Is Wrong
Large language models generate text by calculating probabilities over possible continuations based on patterns learned during training and information available in their context. Producing a fluent sentence is therefore not equivalent to independently establishing that the sentence is true.
This helps explain a strange feature of modern AI: confidence in the wording of an answer and confidence in the underlying fact are not the same thing.
OpenAI researchers argued in 2025 that hallucinations persist partly because conventional training and evaluation often reward guessing. If a model receives credit for a correct answer but little advantage for admitting uncertainty, attempting an answer can become a better strategy than saying “I don’t know.”
The problem resembles a multiple-choice test on which unanswered questions receive zero points while wrong answers carry no additional penalty. A student trying to maximize the score should guess whenever uncertain.
Changing that incentive can encourage AI systems to abstain when they lack sufficient confidence. But that creates another problem: determining when uncertainty is justified.
Researchers call an important part of this challenge calibration.
A perfectly calibrated system expressing 80% confidence would, over many comparable questions, be correct approximately 80% of the time. If it is correct only half the time when claiming 80% confidence, it is overconfident.
Experiments have produced mixed but revealing results. Earlier research involving Anthropic researchers found that larger language models could sometimes estimate whether their own answers were likely to be correct. Models could evaluate a proposed answer and assign a probability that it was true, although their ability to estimate whether they “knew” an answer did not generalize perfectly to unfamiliar tasks.
Other experiments have found substantial overconfidence. A 2024 study examining models including GPT-4, GPT-3.5, Llama 2 and PaLM 2 found high calibration errors in tests of verbalized uncertainty, with models frequently appearing more confident than their actual performance justified.
So the answer to whether AI “knows what it doesn’t know” is neither simply yes nor no.
Finding Uncertainty Hidden Inside AI
One promising approach does not rely on asking a model how confident it feels.
Instead, researchers can examine what happens when the same model is asked a question repeatedly.
In a 2024 Nature study, University of Oxford researchers developed a technique called semantic entropy. Rather than looking only at differences in the exact words generated by a model, the method examines uncertainty in the meanings of multiple possible answers.
Suppose a model repeatedly answers the same question using different sentences that all mean essentially the same thing. The wording varies, but the underlying answer is stable.
If repeated attempts instead produce several fundamentally different answers, that variation can signal uncertainty.
The researchers tested the method across general knowledge, science, mathematics and other tasks. Semantic entropy was better at identifying a class of unreliable responses the researchers called “confabulations” than several comparison methods. The technique could also be used to reject questions most likely to produce those responses, increasing accuracy among the questions that remained.
But there is an important limitation.
A model can be consistently wrong.
If incorrect information has been strongly represented in training data, or a model repeatedly makes the same reasoning error, it may confidently generate essentially the same false answer every time. Low uncertainty therefore does not prove truth.
That is why uncertainty detection and factual verification are different problems.
Research has continued to explore other ways of extracting uncertainty. Some approaches examine probabilities inside the model; others analyze its generated answers, train separate confidence estimators or combine multiple signals. A 2024 ACL paper, for example, demonstrated an auxiliary system that estimates confidence using only a model’s textual input and output.
Research published in September 2026 is still proposing new methods for calibrating uncertainty and detecting hallucinations, evidence that the problem remains active rather than solved.
Knowing When to Stop and Verify
For users, the most useful AI may ultimately not be the system that answers every question.
It may be one that knows when an answer requires evidence.
An AI system detecting elevated uncertainty could search reliable sources, retrieve documents, use external tools, ask the user for missing information or decline to make a claim until it can be verified. In high-stakes settings, it could defer the decision to a human.
External verification is especially important because confidence alone cannot establish factual accuracy. An AI could assign low confidence to a true answer or high confidence to a false one. Checking an answer against reliable evidence provides information that self-assessment alone cannot.
Reasoning introduces another complication. A system can possess the information needed to answer a question yet make a mistake while combining facts, performing calculations or following a chain of logic. Conversely, additional reasoning or access to relevant sources can sometimes turn an initially uncertain problem into one the system can solve reliably.
The goal is therefore larger than eliminating hallucinations. Researchers are trying to develop systems capable of distinguishing between an answer they can support, one that needs verification and one they should not provide confidently at all.
That would represent an important change in what people expect from artificial intelligence.
Intelligence is often associated with having answers. Reliable intelligence also requires recognizing when the evidence is insufficient.
AI systems are beginning to reveal measurable signals of that uncertainty, and researchers are learning how to extract and calibrate them. But no current confidence signal provides a universal guarantee that an answer is correct.
Teaching AI to say “I don’t know” is possible. Teaching it to know when to say it remains the harder problem.
Sources:
Nature — Detecting Hallucinations in Large Language Models Using Semantic Entropy
https://www.nature.com/articles/s41586-024-07421-0
OpenAI — Why Language Models Hallucinate
https://openai.com/index/why-language-models-hallucinate/
Anthropic — Language Models (Mostly) Know What They Know
https://www.anthropic.com/research/language-models-mostly-know-what-they-know
Association for Computational Linguistics — Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
https://aclanthology.org/2023.emnlp-main.330/
Association for Computational Linguistics — Overconfidence Is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models
https://aclanthology.org/2024.trustnlp-1.13/
Association for Computational Linguistics — Calibrating Large Language Models Using Their Generations Only
https://aclanthology.org/2024.acl-long.824/
Applied Soft Computing — Uncertainty Calibration for Hallucination Detection via Token Dependency and Sequence Divergence
https://doi.org/10.1016/j.asoc.2026.116357
About the Author
Brad Socha is the founder of The Universal Record, focused on sourced, factual global reporting. Coverage includes international news, geopolitics, technology, and major developments.