Stephen T’s Blog Spot

In the previous post I argued that screening can do harm, and that the usual evidence offered for it, more cases caught and higher survival rates, is exactly the evidence that misleads. That naturally raises the next question. If survival and cases found are the wrong measures, what are the right ones? How do you tell a screening program that earns its place from one that does not? There is an actual toolkit, and it comes down to a handful of demands you should make before you believe a program works.

Demand mortality, not survival. Survival measured from diagnosis is inflated by the lead-time and length biases described last time, so it cannot settle the question. The honest endpoint is the death rate in the whole group offered screening compared with a similar group that was not. And there is a stricter version: all-cause mortality. A program can lower deaths from the target disease while leaving total deaths unchanged, because the workup and treatment carry their own risks and the disease-specific gain is often small. This is not hypothetical. The randomized trials of breast screening have shown reductions in breast-cancer deaths, yet neither the individual trials nor the pooled analysis has demonstrated a reduction in deaths from all causes. That deserves a long pause before anyone calls a program lifesaving.

Insist on absolute risk, not relative risk. A claim that screening cuts deaths from a cancer by 20 percent sounds overwhelming, but 20 percent off a small baseline risk is a small number, and the relative figure is built to hide that. The Cochrane review of mammography put the benefit at roughly a 15 percent reduction in breast-cancer death in relative terms, which works out to about 0.05 percent in absolute terms. Both numbers describe the same trials. One feels decisive; the other tells you what a given woman should actually expect. Always ask for the absolute number.

Translate the benefit into number needed to screen, and put the harms in the next column. Number needed to screen asks the plain question: how many people must be screened, for how long, to prevent one death? For breast cancer, decision models suggest that screening 1,000 women in their forties every other year over their lifetimes prevents on the order of 8 breast-cancer deaths, at a cost of roughly 1,500 false alarms, around 200 unnecessary biopsies, and about 20 overdiagnosed cancers, every one of which is then treated. State only the first column and you are advertising. State both and you are evaluating.

Compute the positive predictive value at the real base rate. This is the base-rate lesson from earlier in the series applied directly. A test’s sensitivity and specificity are properties of the test, not of the answer a patient receives. What matters after a positive result is the positive predictive value, and that depends on how common the disease is. Suppose 5 of every 1,000 screened women truly have the cancer, the test catches about 87 percent of real cases, and it is about 89 percent specific. A positive result then comes with roughly 4 true cancers for every 110 false alarms, so a positive mammogram carries something like a 4 percent chance of cancer. The other positives face anxiety and follow-up for a disease they do not have, and for a rarer disease the predictive value is worse still.

Finally, run the whole thing through the Wilson and Jungner checklist. The 1968 criteria still organize the judgment: the condition should matter, there should be a recognizable early stage, the test should be acceptable and accurate enough, the natural history should be understood, and the benefits should outweigh the harms and the costs. The criterion that quietly does the most work is whether treating the disease earlier actually changes its course. A program can have a fine test and still fail here, and when it does, it is detecting disease without helping anyone.

Notice what these demands have in common. None of them asks how clever the test is or how much disease it finds. They ask about absolute benefit, measured at the right endpoint, weighed against the full ledger of harm imposed on the many people who can never benefit because they were never going to be harmed. A good screening program is not the one that detects the most. It is the one that prevents the most suffering per unit of harm it creates.

That is general evaluation logic wearing a lab coat. Demand the right endpoint, express the effect in absolute terms, count the harms next to the benefits, and judge against criteria set in advance. Replace screening program with any program, and the discipline does not change at all.

So here is my question. The next time a program is offered to you with an impressive relative number and a count of successes, do you ask for the absolute benefit, the right endpoint, and the full list of harms before you are convinced?