Here's how you evaluate a nutritional study
Confused by all the contradictory results being put out by nutrition research? Sloppy science may be to blame
- Published 18.11.18, 4:40 PM
- Updated 30.01.19, 11:49 AM
- 3 mins read
Confused about what to eat and drink to protect your health? I’m not surprised. For example, after decades of research-supported dietary advice to reduce saturated fats to minimise the risk of heart disease and stroke, along comes a new observational study of 1,36,384 people in 21 countries linking consumption of full-fat (read saturated) dairy foods to a lower risk of death from cardiovascular disease.
But without dissecting each study included in this meta-analysis, it is not possible to say what might be behind this surprising result and whether you should now resume putting cream in your coffee and whole milk in your cereal bowl. The study may simply mean that consuming the equivalent of three servings of dairy products a day is healthful, not saturated fat per se.
Caution is in order, especially since another new study, this one a randomly assigned clinical trial, found that three weeks on a diet rich in saturated fat caused liver fat and insulin resistance to rise far higher than diets high in sugar or unsaturated fat.
Or maybe you bought into the hype about pomegranate juice as an antioxidant superfood, only to learn that the health-promoting evidence for this expensive drink derives mainly from $20 million of company-sponsored research.
In the book, Unsavory Truth: How Food Companies Skew the Science of What We Eat, Marion Nestle, emerita professor of nutrition, food studies and public health at New York University, points out that “pomegranates might have high antioxidant activity,” then asks “compared to what?” Are they more healthful than (much cheaper) grapes?
The POM Wonderful company responded: “Comparing the health benefits of our product to other juices is not a key objective of our extensive research programme.” To which I would ask, “If you’re selling ‘health’, why wouldn’t it be?”
The answer, as Nestle’s extensive research shows, is that the unstated interest of most company-sponsored studies is the bottom line. “It’s marketing research, not science,” she said in an interview. It matters not whether the food in question is considered healthy, like wild blueberries and avocados, or it’s laden with health-robbing calories from fats, sugars and refined starches.
Noting that nutrition research, especially that funded by industry, “requires careful interpretation,” she suggests an approach that all consumers would be wise to follow: “Whenever I see studies claiming benefits for a single food, I want to know three things: whether the results are biologically plausible; whether the study controlled for other dietary, behavioural, or lifestyle factors that could have influenced it; and who sponsored it.”
Consider the studies sponsored by the soft-drink industry, in which Coca-Cola has led an effort to undermine the contribution of sugar-laden carbonated water to the United State’s obesity epidemic. For example, the company funded a study of childhood obesity that, without looking for a possible link between overweight and sugary soft drinks, concluded that low physical activity, inadequate sleep and lots of television watching were most important. To make such conclusions appear valid, Coca-Cola enlisted the participation of university-based scientists who all stood to profit financially from their association with the research.
The “who sponsored it” issue forms the crux of Nestle’s book. It is a critically important question to ask, not just with regard to foods, but also drugs, supplements, exercise regimens and any other product or service that may — or may not — affect the health of consumers.
Increasingly, actual or potential conflicts of interests — factors that can consciously or subconsciously influence the outcomes of research — are being brought to public attention. In September, the director of clinical research at Memorial Sloan Kettering Cancer Center relinquished his post after failing to disclose the millions of dollars he received from pharmaceutical companies whose drugs he studied. An investigation revealed he had put a positive spin on results that other researchers found wanting.
In September, a piece in the Upshot section of The New York Times described yet another type of conflict prominent in the reporting of drug trials: what studies get published (most negative findings never see the light of day), how their results are reported and spin that casts negative or non-significant findings in a positive light. The problem is often confounded when these spurious study results are cited over and over again by other researchers.
“Fifty years of research has demonstrated the influence of pharmaceutical companies on physicians’ behaviour. Even giving doctors pads or pens printed with the brand name of a drug can prompt doctors to ignore a generic or competing brand,” Nestle said.
However, Nestle said, while there have been thousands of studies of conflicts of interest involving drug studies and industry-touted medications, she could identify only 11 such studies of the influence of industry funding on the outcome of food and beverage research in relation to health.
Nestle documents how commercial sources have co-opted organisations that members of the public, and even science writers like me, rely on for unbiased assessments of research results. Even experts in organisations and on panels who formulate health advice for the public can be tainted by commercial influences, she found.
There is an inherent conflict, for example, within the US Department of Agriculture, which has two potentially opposing mandates. The agency is supposed to support production of all kinds of agricultural products — meat, poultry and dairy, as well as fruits and vegetables — and it participates with the Department of Health and Human Services in formulating national dietary guidelines and nutrient requirements for school lunches and other public nutrition programmes.