Stephen T’s Blog Spot

A great deal of modern research and evaluation runs on linked data. We connect a program’s enrollment file to earnings records, a survey to health claims, a benefits roster to death records, and suddenly we can follow people across systems we could never afford to track ourselves. The catch is in the joining. When two datasets do not share a clean, reliable identifier, and usually they do not, we match on names, dates of birth, addresses, and partial identifiers. That matching is never perfect, and its imperfections are not random.

Linkage produces two kinds of error, pulling in opposite directions. A false match joins two records that belong to different people, when common names, shared birthdates, or transcription errors make strangers look identical. A missed match fails to join two records that belong to the same person, when a name changed at marriage, an address moved, a field was blank, or a digit was mistyped. One invents a person who is not there; the other loses a person who is.

The heart of the problem is that these errors are not spread evenly. Whether a record links correctly depends on the quality and stability of its identifiers, and those depend on who the person is. People who move frequently, change their names, hold common names, or appear in lower-quality records are systematically harder to link, and those traits travel with age, income, housing stability, immigration status, and race. This is not hypothetical: in large administrative systems, missed matches have been shown to fall hardest on ethnic minorities, people in deprived areas, foreign nationals, and those with no fixed address. The people who fail to link are not a random slice; they are disproportionately the mobile and the marginalized, often exactly the people an equity-focused evaluation most needs to see.

That turns the linked dataset into a biased sample, tilted toward the easy to link, and the bias enters your results in two ways. If linkage is related to the outcome, whether someone is employed, or has died, or was readmitted, then the linked sample misrepresents that outcome, and you can manufacture or erase an effect through nothing more than who matched. And false matches attach the wrong record to a person, misclassifying an outcome in ways a clean-looking analysis never reveals. Because match rates differ by group, comparisons across groups are contaminated by differential linkage rather than real differences, which is how a study of disparities can end up measuring its own linkage.

What makes this easy to miss is that the linked file arrives looking authoritative, one tidy row per person, often assembled by someone other than the analyst. The match rate may be reported as a single reassuring number, but a high overall rate can hide large gaps by subgroup, and the errors are baked in before analysis begins. Standard errors computed on linked data usually assume the linkage was perfect, so they come out falsely small. It is a fitness-for-use problem sitting upstream of everything you do.

The discipline is to treat linkage as part of how the data were made, not a preprocessing footnote. Ask how records were matched, deterministic or probabilistic, on which fields, at what threshold. Examine match rates by subgroup, not just overall, and treat a differential rate as a bias to investigate rather than a nuisance to note. Where you can, check the linkage against a gold-standard sample and estimate which way the error pushes. And report honestly that your analytic sample is the linked sample, which may differ from the population in patterned ways.

This is the nonresponse question from earlier in this series, moved to a new step. There the issue was who failed to answer, and whether it related to what you measured; here it is who failed to link, and whether it relates to what you measure. Same logic, same danger, further upstream. An impact estimate from linked data inherits every bias of the join, whether or not anyone looked.

So here is my question. When you analyze linked data, do you know who did not link, and whether the reason they are missing is related to the very thing you are trying to estimate?