Separating Sound Science from Junk Science

It can be frustrating to sort through all the nutrition news to find what is credible, and what is not. Apply these evaluation tools and tips to help sort through all the nutrition news.

Scientific studies, traditionally sheltered in insulated professional journals, are now quickly publicized in press releases, posted on Twitter and Facebook, and disseminated on YouTube, TED talks and TV. We are inundated with food and nutrition headlines, recommendations, claims and counter claims asserting the evidence is based on the latest scientific research.

Results of scientific studies are often exaggerated, and are promoted by the researchers, or the public relations departments of their academic institutions, or picked up and misinterpreted by the media. “If it bleeds, it leads” — and fear of food sells. Unfounded scares, embellishments and “what-ifs?” terrify people and can create fear of foods, ingredients and food production technologies. It is challenging for consumers, and even health professionals, to tease out reputable science-based data versus fraudulent nutrition information and claims.

Science is the pursuit and application of knowledge and understanding of natural and social worlds. It follows systematic methodology based on evidence. Sound science refers to a consensus of experts and a weight of evidence accumulated over time, using the best practices of the scientific method, and particularly high quality, peer-reviewed and replicated tests.

Junk science however, is bad science. It is inaccurate analysis, and misinterpretation or embellishment of data, which is then used to skew opinion or push an agenda. It is science that is done or interpreted badly, and comes in many different forms. A few examples are:

• Misreporting and exaggeration of results. This can occur at many levels: by researchers, by their institutions or by the media. It often results in touting a beneficial effect, or amplification of harm, that is not supported by the study. For example, a meta-analysis was introduced in a press release entitled “Little to no association between butter consumption and chronic disease or total mortality.” The study actually reported that butter consumption was weakly associated with increased total mortality, not associated with cardiovascular disease, and slightly protective against diabetes.¹ Yet the research garnered media attention misstating “the case for eating butter just got stronger,” and headlines that declared “Butter is Back.”

• Food Myths: Ideas about products or ingredients that become popular without facts or research to back them up. This translates into widely held but false beliefs. The demonization of monosodium glutamate (MSG) is an example, where a letter to the editor — not a research study — in a medical journal initiated a cascade of conviction that MSG caused human health harm, despite hundreds of well-conducted studies that show MSG is a safe product to use. Additionally, MSG is approved by food safety authorities worldwide, including those in the U.S., Europe, Japan and other Asian countries, North and South America, Africa and Australia and New Zealand.²

• Pseudo-science is based on beliefs, theories, or practices considered scientific, but actually have no basis in scientific fact. This “science” lacks evidence and plausibility, or has been disproved scientifically, or is untestable. An example is the “Science of Astrology.” A famous TV doctor tweeted, “For centuries, we have used astrological signs to examine our personality and how we interact with those around us, however, these signs may reveal a great deal about our health as well. Discover what your astrological sign can tell you about your health.”³ Uh, no…

• Pop science is science that is intended for broader general audience. It is promoted in a variety of formats: books, magazines, YouTube, and Ted Talks by recognized personalities. People are informed while entertained, often on science subjects they do not know anything about. The content is simplified, and focuses on one aspect of the topic. Factual information, contradictory research, opposing views and nuance are lost.

How to differentiate between sound science and junk science?

Almost two decades ago, the Food and Nutrition Science Alliance (FANSA), a partnership between the American Dietetic Association (now the Academy of Nutrition and Dietetics), American Society for Clinical Nutrition, American Society for Nutritional Sciences, and Institute of Food Technologists, published a list “Ten Red Flags of Junk Science.” These ten assessments still resonate today to help evaluate whether a science report is credible or not.⁴

In addition, consider these points that are tip-offs that the nutrition science claim is anywhere from over-simplified, to misinterpreted, to a sales job, to outright fraudulent.

• Cherry-picking. A headline proclaiming a single individual nutrient or ingredient can increase or decrease a risk of a disease. In a study published in the American Journal of Clinical Nutrition, researchers randomly selected 50 ingredients, and looked into studies evaluating the relation of each to cancer risk. Most of the foods had contradictory studies behind them claiming both positive and negative results.⁵ Single studies contradict one another. That is why it is important to look at the totality of the research – that is, the “weight of the evidence.”

• The science claims to be the new wave of nutrition research, discounting mainstream thought and recommendations. It states it has left dieticians and nutritionists behind. Their pet theory is that (science) (doctors) (medicine) (health establishment) has a secret or cure they do not want you to know. They want you to believe they are ahead of the curve and the scientific community just hasn’t caught up.

• Celebrity advocacy appeals to both the media and consumers. A scientist or health professional may be an expert in an unrelated field, but not nutrition. MD’s and PhD’s are highly educated in their field, but decide to “sell health.” Or a famous celebrity or actor endorses or advocates products. Their message: Living healthy doesn’t have to be hard. Just do this thing/eat this food/buy my book. They entice followers to visit a website packed with ads and sales pitches.

FANSA’s “Ten Red Flags of Junk Science”:

1. Recommendations that promise a quick fix
2. Dire warnings of danger from a single product or regimen
3. Claims that sound too good to be true
4. Simplistic conclusions drawn from a complex study
5. Recommendations based on a single study
6. Statements refuted by reputable scientific organizations
7. Lists of ‘good’ and ‘bad’ foods
8. Recommendations made to help sell a product
9. Recommendations based on studies that have not been peer reviewed
10. Recommendations from studies that ignore differences among individuals and groups

The Bottom Line: It can be frustrating to sort through all the nutrition news to find what is credible, and what is not. Apply these evaluation tools and tips to help sort through all the nutrition news.

References

1. Pimpin L, Wu JHY, Haskelberg H, Del Gobbo L, Mozaffarian D. Is Butter Back? A Systematic Review and Meta-Analysis of Butter Consumption and Risk of Cardiovascular Disease, Diabetes, and Total Mortality. 2016. PLoS ONE 11(6): e0158118. doi:10.1371/journal.pone.0158118
2. Walker R, Lupien JR. The Safety Evaluation of Monosodium Glutamate, The Journal of Nutrition.2000.130(4):1049S–1052S. https://doi.org/10.1093/jn/130.4.1049S
3. @Dr. Oz June 6, 2018. 
4. Ayoob KT et al. Position of the American Dietetic Association: Food and Nutrition Misinformation. Journal of the American Dietetic Association. 2002. 102(2):260 – 266.
5. Schoenfeld, JD and Ioannidis, JPA. Is everything we eat associated with cancer? A systematic cookbook review. 2013. The American Journal of Clinical Nutrition. 97(1): 127-134,  https://doi.org/10.3945/ajcn.112.047142