Stephen T’s Blog Spot

An evaluation comes back positive. The program works, the effect is real, and the instinct is to call it a success and recommend expanding it. But whether it works is only half of the question a decision-maker actually faces. The other half is whether it is worth it, and a program can clear the first bar and fail the second.

The reason is simple and unforgiving. Resources are finite, so every dollar spent on this program is a dollar not spent on something else. That makes the useful question not merely whether the program produces an effect, but how much effect it produces per dollar, and whether that rate beats the alternatives you could have funded instead. Effectiveness without cost is half an answer. A program that works but costs a fortune to move the needle a little may be a worse use of public money than a cheaper one that works less well.

There are two main ways to bring cost into the picture. Cost-effectiveness analysis expresses the result as cost per unit of outcome: dollars per additional graduate, per case prevented, per job placement, per healthy year gained. Because the outcome stays in its natural units, you can compare programs that share a goal. Cost-benefit analysis goes a step further and puts a dollar value on the outcomes themselves, so benefits and costs are measured in the same units and you can ask whether the benefits exceed the costs outright. Cost-benefit is more ambitious and more contestable, because assigning a dollar figure to a year of schooling or a life saved requires assumptions many people will dispute.

Two ideas do most of the real work. The first is that the comparison must be incremental. What matters is the extra cost and the extra effect relative to the next-best alternative, not relative to doing nothing. The same program can look like a bargain against no action and a poor deal against the cheaper option already available, so the honest analysis uses the right comparator. The second is opportunity cost: the true cost of a choice is the best thing you gave up to make it. Money spent here is benefits foregone elsewhere, and a program that looks affordable on its own can be expensive once you count what the same funds would have bought.

None of this is as clean in practice as it sounds, and the traps deserve naming. A cost-effectiveness number is only as good as the costs you remember to count. It is easy to tally the visible program budget and omit the participants’ time, the administrative burden, or costs shifted onto other systems, and the answer moves with the perspective you take. It is easy to credit the program with benefits really produced by something else, which is the causal-inference problem the rest of this series wrestles with. And a single ratio can hide who pays and who gains, a question of fairness that efficiency alone does not answer. The number informs judgment; it does not replace it.

One confusion is worth killing directly. Cost-effective does not mean cheap. The cheapest program is not the most cost-effective if it accomplishes little, and the most expensive can be the best value if it accomplishes a great deal. Cost-effectiveness is a ratio of outcome to cost, not a price tag, and the lowest sticker price and the best value for money are often not the same choice.

For those of us who work in and around federal programs, this is close to daily life. Public budgets are effectively zero-sum in the short run, and agencies are increasingly asked to show value for money, not just an effect. An evaluation that reports an impact but says nothing about cost hands the decision-maker half a picture. And in a business case or a proposal, the value-for-money argument is often what separates a fundable idea from a merely interesting one. Bringing cost in, and bringing it in honestly, is part of the work.

So here is my question for the group. When you judge whether a program should grow, do you ask what its results cost and what the same money could have bought elsewhere, or does a positive effect settle the matter on its own?