Stephen T’s Blog Spot

Much of this series has been about establishing cause by comparison. You find a control group, a counterfactual, a similar case that did not get the treatment, and you reason from the difference. But a great deal of real evaluation offers no such luxury. There is one program, in one place, and no comparison to be had. The instinct is to retreat to storytelling, to describe what happened and call it a narrative rather than a finding. There is a more disciplined option, and it comes from an unlikely source: detective work.

The method is called process tracing, and its logic is the logic of a good investigator. A causal explanation is not just a claim that A produced B. It is a claim about a mechanism, a chain of events that had to occur, in order, for A to produce B. If that mechanism really operated, it would have left traces: documents, sequences, testimony, intermediate steps that must be present if the story is true. Process tracing is the disciplined search for those traces within a single case. You do not compare the case to another. You interrogate the case against what your explanation requires to be true.

What keeps this from being mere storytelling is that not all evidence counts equally, and process tracing is explicit about why. It sorts evidence by how much it can actually do, using four kinds of test. A hoop test is a necessary condition: failing it eliminates the explanation, though passing proves little on its own. An alibi is the classic example, if the suspect was elsewhere, the case collapses. A smoking-gun test is the mirror image: passing it strongly confirms, though failing does not eliminate, like a weapon found in a suspect’s hand. Weakest are straw-in-the-wind tests, merely suggestive either way; rarest are doubly decisive tests, which confirm one explanation and eliminate its rivals at once.

The engine underneath is Bayesian, even when no numbers appear. What gives a clue its force is not how dramatic it is, but how much more likely it would be if your explanation were true than if a rival were. A fact that any explanation would predict tells you almost nothing. A fact that only your explanation would predict is powerful, precisely because a competitor cannot easily account for it. This is why a single well-chosen observation can outweigh a pile of ordinary ones. Quality of evidence, not quantity, carries the inference. The dog that did not bark mattered to Sherlock Holmes because silence was expected under exactly one story and surprising under the rest.

For evaluation, this reframes what a single-case study can do. Instead of asking whether the outcome appeared after the program, which almost any account would predict, you lay out the causal chain your theory of change requires, then hunt for the steps that must exist if the program truly caused the result. Just as important, you look for the evidence that rival explanations would leave if they were the real cause: a funding change, a national trend, a parallel initiative. You test both. Contribution is established not by comparison but by surviving the search for disconfirmation, a theme this series keeps returning to.

None of this makes process tracing easy or foolproof. It demands a well-specified theory, genuine effort to imagine rival explanations rather than only your favored one, and honesty about which tests the evidence actually passed. Doubly decisive evidence is rare, and most conclusions rest on an accumulation of hoop and smoking-gun tests. But done well, it lets you say something disciplined and defensible about causation in exactly the situations where a comparison group was never available, which is much of the work.

So here is my question. When you have only a single case and no comparison, do you retreat to narrative, or do you specify the fingerprints your explanation must have left and go looking for them?