George Hasara – Columnist
According to a study done in 2016 by the journal Nature, science is facing a “reproducibility crisis.” Nearly 1,600 scientists of various disciplines from cancer treatment to climate change, concluded that 70 percent of published scientific studies could not be replicated. Since replicating experiments is a critical component of the scientific method, the idea that most reports are fudging the data is a quantum quandary.
The top reasons given for flawed studies are pressure to publish, selective reporting, and poor understanding of statistics. As Mark Twain is famous for saying: there are “lies, damned lies, and statistics.” Statistics are a numerical expression of information. Just as sophistry or wordplay can distort the truth, so can the manipulation or misunderstanding of statistics lead to a false conclusion. Unfortunately, most scientists, like most people, are not statisticians.
Research is typically funded by entities that prefer a particular result. (Have you noticed that in business, people are “paid,” while in academia, they are funded?) There is pressure to publish something sensational or at least newsworthy. You typically don’t see headlines such as, “Study shows no correlation between cell phone use and thumb arthritis,” or “Dark chocolate is NOT a miracle food.” If a “conclusion” is desired, it will be found, even if documentation of the alleged causation is flimsy.
Another key element cited by Nature is that about two-thirds of the scientists rated fraud as a factor in irreproducible research. We instinctively understand in areas such as personal relationships, politics, or business, that there are plenty of incentives for deception. However, a blind spot tends to emerge when the subgroup of humanity are scientists. Why would a scientist lie, aren’t they the guardians of truth? It’s not a surprise that scientists are flawed like the rest of us, but it is refreshing to hear that those in the field, know that high IQs don’t necessarily correlate with truthfulness.
Selective reporting, aka cherry picking, is perhaps the simplest way to reach a particular conclusion. Data doesn’t need to be falsified, but rather carefully selected or omitted. In research, causation is often confused with correlation. These two words are interchangeable in casual conversation but are significantly different in logic and scientific terms. There have been a slew of studies in recent years touting the health benefits of drinking red wine, from increasing bone density to reducing the risk of stroke. However, I don’t believe wine consumption necessarily makes you healthier but rather that health-conscious people (actual causation) are statistically more likely to be red wine drinkers. I bet BMW owners have a lower BMI (Body Mass Index) but it has nothing to do with German engineering.
I enjoy quoting studies, it gives the air of expertise, even if I don’t know what I’m talking about. But what if those who author the studies don’t really know what they are talking about either? Perhaps, the Nature report itself is one of those scientific studies that can’t be reproduced but I have a hunch that that is not the case. As always, if you think the subject matter is important enough, it’s best to do your own fact-checking and have a cautious eye toward studies that could very well have a problem with reproducibility.
Contact George at firstname.lastname@example.org.