FrontierAugust 28, 2026via Apple Machine Learning
LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs’ Probabilistic Beliefs
Why it matters
LLMs deployed in high-stakes domains (medicine, law, science) claim to handle uncertainty, but Apple's research shows they violate basic probabilistic consistency. This matters for practitioners building systems that must reason reliably as evidence accumulates—and for understanding whether current models can be trusted in domains where calibrated uncertainty is non-negotiable.
Key signals
- Apple research introduces 'information processing gap' metric to measure deviation from Bayesian updates
- Study focuses on LLM consistency in updating beliefs as new evidence arrives
- Evaluated in high-stakes domains: medicine, science, law
- Extensive experiments conducted (scale/sample sizes not specified in excerpt)
- Published by Apple ML Research (peer-reviewed venue signal)
- Apple ML research paper on LLM probabilistic reasoning
- Introduces 'information processing gap' metric to measure deviation from Bayes updates
- Tests LLM belief-updating in domains requiring rational uncertainty management: medicine, science, law
- Extensive experiments on internal consistency of probabilistic beliefs
- Published Aug 2026 by Apple's research team
The hook
Apple researchers expose a fundamental flaw: LLMs systematically fail at Bayesian reasoning—the math that should govern rational belief updates under uncertainty.
Modern AI systems are being deployed in complex domains such as medicine, science, and law, where there is often not a single correct answer given the observed evidence. Such systems must be able to represent and update uncertain beliefs about the world as new evidence arrives to make rational decis…