Stephen T’s Blog Spot

Ask an experienced team how long a project will take, and they will study the specifics. They will break the work into phases, estimate each one, add the pieces, and perhaps pad the total for safety. It is a careful, disciplined process, and it produces an answer that is almost always too optimistic. The very act of building the estimate from the inside is what biases it.

This is the planning fallacy, named by Daniel Kahneman and Amos Tversky in 1979. It is the systematic tendency to underestimate the time, cost, and risk of a plan while overestimating its benefits, and its most unsettling feature is that experience does not cure it. People who have watched similar projects run late will still forecast that this one will finish on schedule. The classic demonstration followed students estimating when they would finish their theses; most blew past even their own worst-case predictions, and the pattern holds far beyond students.

The root of the problem is the view you take. When you plan from the inside, you focus on the particulars of this project: your team, your approach, the steps you intend to follow. That view feels rich and relevant, and it is exactly the view that misleads, because it cannot show you what you do not know to look for. The delays that wreck projects are rarely the ones on anyone’s list: the vendor who fell through, the requirement that changed, the clearance that took four months. The inside view has no slot for the unexpected, which is precisely why the unexpected keeps winning.

The alternative is the outside view. Instead of asking how this project will unfold step by step, you ask a different question: how did similar projects actually turn out? You assemble a reference class of past efforts that resemble this one, you look at what they really cost and how long they really took, and you start your estimate from that distribution rather than from your own plan. Your project is not as unique as it feels from the inside, and the track record of its cousins is the best predictor you have.

This is the heart of reference class forecasting, developed by Bent Flyvbjerg from that original insight. Study large infrastructure projects and the numbers are sobering: in one well-known analysis, the large majority of rail projects overshot their cost estimates, by an average of nearly 30 percent. Reference class forecasting starts from that empirical reality and adjusts for the case at hand, rather than the reverse. The approach is now written into official guidance, including the United Kingdom Treasury’s rules for appraising major public spending. Kahneman called taking the outside view the single most valuable thing you can do to improve a forecast.

It is not a cure-all, and it is fair to say so. A useful reference class can be hard to define when a project really is novel, and reasonable people can argue about which past efforts belong in it. The method also corrects the symptom, the optimism, more than it explains the specific causes. But none of that undoes the core point. An estimate anchored to how comparable work actually turned out beats one built purely from the hopeful logic of your own plan.

There is one more force worth naming, because it lives in my corner of the world. Optimistic estimates are not only a cognitive error; they are often rewarded. The confident schedule wins the bid, and the lean budget secures the approval. That pressure quietly pushes forecasts toward best-case thinking exactly when a sober number matters most. The outside view is a discipline against both the bias and the incentive, which is part of why it is hard to adopt and valuable when you do.

So here is my question. The next time you estimate a cost or a timeline, do you start from the details of your own plan, or from the record of how similar efforts actually turned out?