Stephen T’s Blog Spot

Picture a simple bar chart. Two bars, one taller than the other. Your eye settles the matter in an instant: this group is higher than that one. But the chart has withheld the single fact you need in order to trust that reading, which is how much each bar could have come out differently by chance. Strip the uncertainty out of a picture and you have not made the result simpler. You have made it look more certain than it is.

This is the quiet failure of most default charts. A bar is a single number. A line connects single numbers. A dashboard tile shows one figure in bold. Nothing on the page signals that each of those numbers is one draw from a distribution, that a rerun of the same study would have landed somewhere nearby but not identical. The crisp, confident geometry of a clean chart is a claim, and often an overclaim, made by omission.

The consequences are concrete. Two bars that differ by a sliver can be statistically indistinguishable, their intervals heavily overlapping, and yet the picture shouts that one is bigger. A line that drifts upward can be indistinguishable from flat once you allow for noise, and yet a trend line reads as momentum. The viewer walks away with a strong conclusion the data does not support, and no one had to lie with a number to produce it. The chart did the overstating on its own, by leaving the uncertainty out.

The remedy is to put the uncertainty back on the page. Add error bars or confidence intervals to the bars and points. Draw a shaded band around a fitted line or a forecast. Where you can, show the spread of the underlying data and not only the summary, because a mean conceals the distribution it came from. The aim is for the eye to take in the estimate and its uncertainty at the same time, so that the strength of the visual impression matches the strength of the evidence, no more and no less.

Showing uncertainty well has its own traps, and two are worth naming. First, an error bar is meaningless until you say what it is. A standard deviation, a standard error, and a confidence interval describe three different things with three different widths, and readers routinely confuse them, so label them every time. Second, you cannot judge a difference by whether two intervals overlap. Overlapping confidence intervals do not prove there is no real difference, and separated ones do not prove there is one; the eyeball test is unreliable. When the difference between two groups is the point, the honest move is to show the uncertainty of the difference itself, not to leave readers comparing two bars by sight.

Step back and this is simply the communication end of everything the series has said about uncertainty. A p-value, a confidence interval, an effect size with its range, these are all efforts to be honest about how much we do not know. A chart that shows only the point estimate throws that honesty away at the last step, exactly when the result reaches the people who will act on it. The picture is often the only part of a study a decision-maker ever sees, which makes it the most important place to tell the truth about what is uncertain.

For those of us who build dashboards, briefings, and one-page summaries, the pressure to present a clean and confident picture is real, and it runs the wrong way. A bar chart of outcomes by site, with nothing to signal uncertainty, quietly invites a manager to rank the sites and act on gaps that may be noise. Adding the intervals is not hedging or clutter. It is the difference between informing a decision and inviting a mistake.

So here is my question. When you put a result in front of someone who will act on it, does your chart show how much you do not know, or does it quietly promise a certainty you cannot back up?