Crying "Fire! Fire!" in Noah's Flood Watch: the OECD
New York Times FAIL: Gardiner Harris and Reed Abelson

Henry Aaron of Brookings on Abelson and Harris of the New York Times: "Rarely Have I Begun an Article with Such High Hopes That The Public Would Be Educated on a Difficult Subject and Ended With Such a Feeling of Complete Let-Down..."

The extremely intelligent and thoughtful Henry Aaron:

Rarely have I begun an article with such high hopes that the public would be educated on a difficult subject and ended with such a feeling of complete let-down. It isn't that the Times authors of the critique of Dartmouth were wrong. It is that they said so little with so many words. The Dartmouth research, showing huge variation in the use of various medical procedures and large variations in per patient spending under Medicare, has been a revelation and a useful one. There is no way to explain such variation on medical grounds and it is problematic...

That disposes of Abelson's and Harris's claim that:

Data Used to Justify Health Savings Effort Is Sometimes Shaky: For all anyone knows, patients could be dying in far greater numbers in hospitals in the beige regions than hospitals in the brown ones, and Dartmouth’s maps would not pick up that difference. As any shopper knows, cheaper does not always mean better...

Aaron, however, is not a big fan of Dartmouth or of its use by the Obama Administration. Here is the right way to write a critique:

[R]eaders, including my former colleague Orszag, have taken an oversimplistic view of what the numbers mean and what to do about them. There are three really big problems with the common interpretations, none of which the Times authors [Abelson and Harris] explained clearly-- or, indeed, at all:

  1. [W]hat appear to be high [Medicare] spending areas are, in many cases, high [total health] cost-shifting areas.... What counts is total medical spending if one wants to relate spending and quality.

  2. Knowing that spending is high and poorly correlated with measures of quality (if such a correlation exists!) is not the same as knowing how to pare spending in a way that increases net social benefit from health care.... There are some egregious situations where ex-ante waste can be demonstrated. But for the most part, we just do not have good enough research to predict with sufficient accuracy who will benefit and who will not from doing more of some procedure. That will take a lot of research. It hasn't yet been done.

  3. Some research has shown that even the most glaring cost differences noted in the Dartmouth research are associated with improved outcomes.... Joseph Doyle at MIT. If the Times article had described some of his work, readers would have learned something.... [Y]ou can be pretty sure that people who suffer coronaries while on vacation do not choose in which county to have the heart attack... they did better in Florida counties that spent most... preemies who were a couple of grams under the threshold at which they are described as in need of intenseive care did materially better than did preemies a couple of grams over that threshold and, hence, treated as normal births.... [This] means not that the Dartmouth people are wrong, but that any savings will be very hard to achieve without doing more harm than good and will be very slow in coming...

Comments