All Studies Are Not Created Equal

Monday, November 19, 2012 // Uncategorized

We are bombarded daily with medical news with often conflicting conclusions making it difficult to know what to do.  In medicine, we don’t weigh

 all studies equally.  There are different kinds as outlined in this recent articl from the San Antonio Express News by Dr. Steven Austad from the UTHSCSA.


Observational studies can lead to false conclusions

Steven Austad

Updated 3:24 p.m., Saturday, November 17, 2012

Page 1 of 1

People who eat diets rich in fruits and vegetables are healthier and live longer than people who don’t. Fruits and vegetables contain high levels of antioxidants, chemicals that neutralize damaging free radicals that our bodies produce. Therefore, consuming massive amounts of antioxidants to soak up the free radicals must be good for your health. From this logic, a lucrative industry was born — selling pills packed with the holy trinity of antioxidant vitamins, A, C, and E.

This scenario all made sense based on the sort of weak evidence that I’ve previously described — and on which most human health studies are based. That is, asking people questions, then following up what happens to them. People who take antioxidant vitamins are healthier and live longer than those who don’t. But is it the vitamins that do it or other lifestyle factors? We don’t have to depend only on weak evidence. Science has spent the last 500 years learning how to gather strong evidence. It’s called “doing experiments.” Experiments, if properly done (which is a whole other topic), are the gold standard of science.

Human health experiments are called clinical trials. They are pretty straightforward. Divide a group of people randomly into one part that gets the treatment (such as a daily supplement of antioxidant vitamins) and another part that doesn’t (the controls). Probably the most informative thing you can do when reading the health news is try to figure out whether the study was weak and observational (that is, it merely observed what happened to people), or strong and experimental, imposing a treatment on a randomly selected group of people and comparing them with who don’t get the treatment.

The advantage of experiments is that by randomly dividing people into groups, you are guarding against the possibility that the people you are studying differ in some systematic way that you weren’t aware of. People who take antioxidant vitamins, for instance, are likely to differ from people who don’t in lots of other health habits that no one asked about. If so, then the effect you thought was due to their antioxidants could be due to something else.

The strong evidence is now in on the holy trinity of antioxidants. It turned the weak evidence on its head. Taking massive amounts of vitamins A, C, or E has no obvious health benefits and in some cases may actually harm your health. Something similar happened with hormone replacement therapy for postmenopausal women. A large body of weak observational evidence indicated that HRT reduced heart disease by 40 percent to 50 percent. Yet when the Women’s Health Initiative experimentally tested one common type of HRT, it found quite the reverse: a slight increase in heart disease. Don’t take this to mean that you should avoid vitamins and postmenopausal hormones. Results of the studies I’ve mentioned are complex and depend on other factors that I don’t have room to discuss. I’ve focused on the unexpected results of these studies simply to illustrate that observational studies, even well-done ones, can sometimes lead to false conclusions.


Steven Austad is a professor and interim director for the Barshop Institute for Longevity & Aging Studies at the UT Health Science Center San Antonio. His column appears every other Sunday in S.A. Life.

[email protected]


Leave a Reply

Your email address will not be published. Required fields are marked *