Stephen T’s Blog Spot
A blog aimed at issues only data scientists, data analysts, statisticians, evaluators, and researchers care about.
Category: Uncategorized
-
Here is a scene familiar to anyone who has done this work. A program has run for two years. The funder wants an evaluation, a contract is awarded, and a few months in the team hits a wall. Nobody quite agrees on what the program was supposed to achieve. The goals are vague. The data…
-
One of the biggest shifts in how evidence gets made is the rise of real-world data and real-world evidence. It began at the FDA and is spreading across health agencies and into evidence-based policy more broadly. The appeal is obvious: instead of slow, narrow, expensive clinical trials, use the mountain of data the system already…
-
The most famous cautionary tale in observational research goes like this. For years, study after study found that women on hormone replacement therapy had healthier hearts, and it was widely prescribed partly for that reason. Then a large randomized trial, the Women’s Health Initiative, found the opposite: the therapy raised cardiac risk. A great deal…
-
Across several posts on causal inference, propensity-score matching, regression discontinuity, target trial emulation, I kept hitting the same wall. Each method could adjust for the confounders you had measured, but none could touch the ones you had not. Unmeasured confounding was the shared limit. There is one classic method built to climb that wall, with…
-
We pour enormous effort into getting the methods right: the design, the analysis, the careful caveats. Then the report is delivered, politely thanked, and set on a shelf, where nothing happens to it. In evaluation, the most common failure is not a flawed method. It is irrelevance. A technically flawless study that changes no decision…
-
Across several previous posts on causal inference, propensity-score matching, regression discontinuity, target trial emulation, I kept hitting the same wall. Each method could adjust for the confounders you had measured, but none could touch the ones you had not. Unmeasured confounding was the shared limit. There is one classic method built to climb that wall,…
-
We live in the age of evidence-based everything: evidence-based policy, evidence-based practice, evidence-based design. When someone says “the research shows,” it is meant to end the argument. But a quiet problem is buried in that phrase. The research you get to see is not a fair sample of the research that was actually done. The…
Stephen T’s Blog Spot 