Is most published research wrong?
Monday October 17, 2005
Long before Copernicus suffered the wrath of the Catholic church for pointing out that Earth orbits the sun, researchers were publishing results that contradicted, and often completely overturned, prevailing scientific beliefs. The process continues to this day.
Scientific consensus builds by testing hypotheses. Those results that are successfully repeated by others gradually form part of the scientific basis of our understanding of the world and the universe.
So far, all observations suggest Copernicus was spot on. For much research being published today, however, it’s only a matter of time before another scientist does an experiment that contradicts, clarifies or proves it plain wrong. In turn, that paper will probably be corrected by later research. The hope, and the reality, is that science as a whole becomes less wrong as time goes by.
It’s not hard to come by examples of duff research. In 2003, researchers at one US university reported that small amounts of ecstasy could kill. Wrongly, as it happens: their conclusion was based on injecting animals with a wholly different drug. And after British scientists spent five years trying to find out if sheep were infected with BSE, someone noticed their results were nonsense. They had been studying cows’ brains by mistake.
But Ioannidis’s paper, which appeared in the online journal Public Library of Science Medicine, argues that "false findings may be the vast majority of published research claims".
There are good reasons for this, he explains: studies that are too small, too much flexibility in designs and definitions, the possibility of financial interests at work. And don’t think having dozens of teams at work on a problem is likely to produce a more reliable set of findings: competition to be first to publish on hot topics is so intense that "each team may prioritise on pursuing and disseminating its most impressive ‘positive’ results", prompting their rivals then to find negative results to get their share of the glory.
Incorrect research is a millstone that will hold research back, but ultimately it is safe to believe that future studies will bring science closer to the truth.