Stephen T’s Blog Spot

Try a quick test. Pick ten quantities you cannot know off the top of your head, the length of the Nile, the year a certain invention appeared, the population of a country, and for each give a low and a high value wide enough that you are 90 percent sure the truth falls between them. Not certain, just 90 percent. If you are well calibrated, you should miss about one of the ten. Almost no one manages that: across hundreds of studies, the true value lands inside people’s 90 percent ranges less than half the time, and often far less. We are not as sure as we think. We are much surer.

This effect has a name, overprecision, and it is the most stubborn member of a family of overconfidence biases. Researchers separate three: overestimating your own performance, overrating yourself relative to others, and overprecision, holding your beliefs with too much certainty. The first two come and go with the situation; overprecision is remarkably robust. A striking version of the classic demonstration shows that the range people give when asked to be 98 percent sure is often barely wider than the range they give when asked to be 50 percent sure, as though the extra certainty cost nothing to claim.

This matters because our work is built on ranges and probabilities. A cost estimate, a schedule, a risk rating, a projected effect size, a win probability on a bid, each is really a statement about how sure we are. Draw the range too narrow and you plan as if the future is more pinned down than it is, and then you are surprised, repeatedly and expensively, by outcomes you had quietly ruled out. The overrun that supposedly could not happen was usually just outside a range someone drew too tightly.

The comforting thought is that expertise cures this. It does not. When specialists put ranges on uncertain quantities in their own field, a standard step in formal risk analysis, their ranges are still too narrow, and the tendency is described as pervasive and hard to correct. Knowing more can even make it worse, since confidence tends to rise faster than accuracy, and on genuinely hard questions overprecision is at its strongest.

Part of the reason awareness alone does not fix it is how we build a range in the first place. We start from a best single guess and add a little to each side, and that adjustment is always too small, the anchoring problem this series has covered. We end up answering an easier question, how confident do I feel, in place of the hard one, how wide is the real spread of possibilities. Being told to widen your intervals barely moves the needle.

What does work is structure that forces you to face the ways you could be wrong. Rather than drawing a range around your best guess, set the two ends separately: first a low value you are 90 percent sure the truth sits above, then a high value you are 90 percent sure it sits below. That small change reliably widens the interval. Before you commit, run a premortem: assume the estimate turned out badly wrong and explain how, which drags the ignored scenarios into view. And keep score. Forecasters who become genuinely well calibrated, weather forecasters being the standard example, are the ones who make many predictions and get prompt, unambiguous feedback on what happened. Track your ranges against outcomes and, over time, your ninety percent starts to mean ninety percent.

For a contractor this is not an academic nicety. An overprecise cost range loses money; an overprecise risk assessment misses the event that sinks the project; a confident single number is a promise you cannot keep. An honestly wide range is more useful than a narrow one that is wrong, and in front of a client who has been burned before, it is more credible too.

So here is my question. When you hand over an estimate or a forecast, is the range wide enough to be right ninety percent of the time, or only wide enough to feel confident?