We’re a little tired of watching the media bandy about the supposed death of dietary supplements, based on a study on older women and multivitamins published earlier this month. Fox News and The Wall Street Journal ran the headline “Studies suggest case for dietary supplements is collapsing,” and we had to shake our heads.
The problem with this idea is that the study itself was so flawed that it’s hard to draw anything but flawed conclusions from it.
Published in the Archives of Internal Medicine—a very good journal—the study looked at older women who take multivitamins, and concluded that those who do take them are more likely to die than those who don’t. This is what was reported in the press, along with much wrinkling of brows, wringing of hands, and gnashing of teeth.
What the study itself did not address—and the stories in the press ignored this, or didn’t even realize it—is that the study looked for factors that might have confused the results, such as smoking, weight, blood pressure, diabetes, weight, eating fruits and vegetables. For every one of these risk factors, the women who took vitamins were in the healthier category.
So, the researchers performed some statistical sleight of hand: they adjusted their data, allowing them to analyze their data as if the only difference between the vitamin takers and not vitamin takers was the vitamins. But it wasn’t! They were healthier in almost every way imaginable. This means that the researchers had to artificially, statistically increase the apparent likelihood of the vitamin takers dying.
Scratching your head? Now you know why Mark Twain said: “There are lies, damn lies, and then there are statistics.”
In other words, we don’t know what actually happened to the women who took multivitamins. We only know that, after statistical manipulation, the authors got a certain answer—maybe even the one they were looking for.
One other note: the women in this study were 62 years old when they began the study. Among the many things this study doesn’t tell us is what happens if you start taking vitamins when you are 25 or 35 or 45.
OK, a second note: this type of study, called an observational study, is never able to tell us anything about cause and effect. It’s only able to tell us that two factors, such as vitamins and living longer, may or may not be related. The results from such a study must then be confirmed, or refuted, by a Randomized Controlled Trial (RCT.) We’d bet that if this study had said vitamins make you live longer and healthier, the media and the medical establishment would have warned people against taking vitamins until an RCT confirmed the results. But when a flawed observational study suggests vitamins might be bad, those same authorities urge us to rush for the exits.
