Implausible results in human nutrition research

I agree with Wiseman and Jackcson that the World Cancer Research Fund (WCRF)/American Institute for Cancer Research efforts are worthy initiatives. However, as they themselves acknowledge publication bias can distort the available information. For small effects, bias in a fragmented literature can wreak havoc even to the most comprehensive systematic efforts by the very best scientists. I don’t agree that it is much more difficult to interpret negative results in randomized controlled trials (RCTs) of nutrition than positive results. Given that the prior chances of single nutrients being highly effective are slim, a negative result in a well-designed RCT practically settles the matter. A positive result, conversely, may still be tenuous, especially when it comes from exploratory, secondary analyses, even in an otherwise well-designed RCT. Wiseman and Jackson apparently believe that observational epidemiology is so complex that the hypotheses that they generate cannot be refuted by RCTs. This surrounds observational epidemiology with some sort of sacrosanct aura. Conversely, I think that observational epidemiology is valuable and it can generate interesting hypotheses that are interesting precisely because they can be tested, validated, and refuted. If observational epidemiology created hypotheses that were untouchable, I would not be interested in it at all. I fully agree with Wiseman and Jackson that mechanistic evidence is very interesting and I applaud their efforts to standardize and systematize it. However, biases in this evidence are sometimes even stronger than those of observational epidemiology. Finally, I am extremely supportive of the need to tackle systematically the entire exposurome (1,2) rather than single nutrients. Eventually, I have no problem with making decisions and acting when we only know 1% of what we could know, but it is good to acknowledge what we don’t know.

Fraser suggests that tentative conclusions that important effects do exist for some diets (e.g. Mediterranean diet with virgin olive oil or nuts) seem more prudent than a dismissal. My intention was not to dismiss these effects. Conversely, I did say that I do believe that there is some beneficial effect and this could also have major public health implications. However, I do dismiss some very large effects that are occasionally claimed, simply because they don’t satisfy common sense criteria. E.g. if a serving of nuts per day decreased mortality by half, then I would expect to live for 140 years, simply because I do love nuts, but I don’t have such expectations. Fraser also mentions his experience with American Adventist vegetarians adding that “these results seem un-confounded by other non-dietary factors”. Personally, I cannot make this statement about lack of confounding for any observational study that I have been involved in. I cannot dismiss confounding when typically our studies in nutritional epidemiology capture only the minority of the risk variance, even when all measured variables are carefully considered. Fraser also lists a number of caveats and real-life difficulties about designing and running long-term RCTs. I don’t want to diminish the challenges in designing useful mega-trials, however, if as Fraser says “we do not know how to effectively intervene on such a complex behavior for such a long period”, then we should acknowledge that whatever we learn from observational nutritional epidemiology currently is useless, since we won’t be able to implement it in the long-term anyhow. I don’t share this viewpoint. Conversely, I believe that research should shift focus on understanding how to implement such long-term changes and interventions, rather than simply continuing with more of the same. Finally, I disagree with Fraser that consistent results across observational studies will certainly suggest causal effects. Consistent effects may indeed demonstrate causality in some cases. However, in other cases, they may just reflect a literature that is written, peer-reviewed, and edited by fervent believers who will not accept any other result other than what perpetuates their beliefs.

John P.A. Ioannidis, MD, DSc
Professor and Director, Stanford Prevention Research Center, Stanford, CA

E-mail: jioannid@stanford.edu

References:
1. Tzoulaki I, Patel CJ, Okamura T, Chan Q, Brown IJ, Miura K, Ueshima H, Zhao L, Van Horn L, Daviglus ML, Stamler J, Butte AJ, Ioannidis JP, Elliott P. A nutrient-wide association study on blood pressure. Circulation 2012;126(21):2456-64.
2. Ioannidis JP, Loy EY, Poulton R, Chia KS. Researching genetic versus nongenetic determinants of disease: a comparison and proposed unification. Sci Transl Med 2009;1:7ps8.