Can you imagine a news article with that title? Certainly not. How about  this one: Abstaining from alcohol significantly shortens life. There, that’s more sensational, isn’t it? (To be fair, that’s the title on the page, not the title of the article. I’m not sure why they aren’t the same.)

I’ve seen news articles circulating about a recent study from the University of Texas at Austin that followed 1,824 adults between the ages of 55 and 65, and compared how likely they were to die over a 20-year period depending on whether they abstained from drinking alcohol, drank moderately, or drank heavily. The results indicated that moderate drinkers had the  greatest longevity, followed by heavy drinkers, with abstainers being the most likely to die.

This is where the science reporters stopped paying attention, and started writing sensationalist “alcohol is good for you!” news articles.

I’m skeptical of this interpretation. The technique being used by the researchers is one that is very common in health and social science studies, whereby the researcher measures many real-world variables, and uses linear regression to tease out the effects of each variable on the variable under study.

To compare this to a controlled experiment, suppose a researcher wants to know the effect of B on A, but he’s worried that C’s effect on A might mess up his results. He can set up a controlled experiment whereby he keeps C fixed, varies B, and observes the effect on A.

In health and social science research, performing the research in a controlled lab is not usually an option. Those 1,824 people probably wouldn’t have signed up for the study if the researchers had asked them to stay in a controlled laboratory environment for the full two decades. So what health and social science researchers do is measure the variations in A, B, and C in the real world and attempt to tease out the effect of B on A by a mathematical procedure that accounts for the variation in C. The problem with this is, first, that they have to make certain assumptions about the relationships between A, B, and C, and these assumptions are often violated. Second, even if the researchers succeed in controlling for C, they might be unaware of some other factor, D, that is related to both A and B. Then what looks like the effect of B on A might actually just be the effect of D on both.

The irony of journalists taking this study and proclaiming that “drinking makes you live longer!” is that this study’s real contribution is to show that earlier studies had overestimated the very effect journalists are trumpeting. Previous studies that controlled for some factors but not others showed a stronger positive relationship between abstention from alcohol and death. This study controlled for more factors: previous heavy drinking, health conditions, etc. Each additional factor the researchers controlled for made the observed relationship between abstention and death smaller.

It’s no stretch to think that, for all the researchers’ hard work, they didn’t manage to control for everything. For instance, if a person abstains from drinking because he’s worried about some potential health problem, that would only show up in the data if his health problem was diagnosed. So undiagnosed health problems are almost certainly having some effect on the result, although we don’t know how large the effect is because they don’t show up in the data.

When you think about it, if each factor researchers were able to measure and control for made the result smaller, is it so unreasonable to expect that the things they couldn’t measure and control for would make the result smaller still? We can’t know. What we can know is that it’s best to take studies like this with a grain of salt.