The subjective confidence we feel in our judgments and predictions, which is determined by the coherence of the story we construct rather than the quality of the evidence supporting it.
Confidence is a poor indicator of accuracy because it reflects narrative coherence, not evidential quality. When we construct a compelling story from available information, we feel confident—regardless of whether that information is sufficient or reliable. This explains why overconfident people are often wrong, why experts can be confident in domains where their predictions have no validity, and why we trust our intuitions even when they've been proven worthless. The illusion persists because System 1 generates coherent stories from whatever information is available (WYSIATI), and cognitive ease from that coherence translates into subjective confidence.
After a 30-minute interview, a hiring manager feels highly confident about a candidate's abilities and cultural fit. This confidence reflects the coherent story constructed from limited information, not the actual predictive validity of interviews (which research shows is low without structure).
If I feel confident in my judgment, it's probably accurate—actually, confidence reflects story coherence, not accuracy; you can be highly confident and completely wrong.
Why is subjective confidence a poor indicator of judgment accuracy?
How do WYSIATI and the illusion of validity work together to create overconfidence in hiring decisions based on interviews?
The slow, deliberate, effortful mode of thinking that allocates attention to complex computations, self-control, and conscious reasoning.
Mental ModelThe fast, automatic, intuitive mode of thinking that operates effortlessly and generates impressions, intuitions, and feelings without conscious control.
Mental ModelJudging the frequency or probability of events by how easily examples come to mind, leading to overestimation of vivid or recent events.
Mental ModelJudging probability by how much something resembles a typical case while ignoring base rates, sample size, and statistical principles.
Mental ModelThe tendency to rely too heavily on an initial piece of information (the anchor) when making subsequent judgments, even when the anchor is arbitrary or irrelevant.
Mental ModelThe principle that losses loom psychologically larger than equivalent gains, with losing something feeling roughly twice as bad as gaining the same thing feels good.
PrincipleA descriptive model of decision-making under risk showing that people evaluate outcomes relative to a reference point, are loss-averse, and weight probabilities non-linearly.
FrameworkSystem 1's tendency to construct the most coherent story possible from currently available information without considering what's missing or questions not asked.
PrincipleThe subjective confidence we feel in our judgments and predictions, which is determined by the coherence of the story we construct rather than the quality of the evidence supporting it.
After a 30-minute interview, a hiring manager feels highly confident about a candidate's abilities and cultural fit. This confidence reflects the coherent story constructed from limited information, not the actual predictive validity of interviews (which research shows is low without structure).
If I feel confident in my judgment, it's probably accurate—actually, confidence reflects story coherence, not accuracy; you can be highly confident and completely wrong.
Illusion of Validity is explored in depth in "Thinking, Fast and Slow" by Daniel Kahneman. Distilo provides a deep AI-powered analysis with key insights, audio narration, and practical frameworks.