Scientific investigation relies on the publication of peer-reviewed studies to communicate advances in research, including medical research. Over the course of the last decade, however, several important paradigms have been found baseless. Now, a new project, the Reproducibility Initiative, announced last Tuesday by the company Science Exchange, will provide researchers with an opportunity to replicate their studies in order to double-check the validity of their results. The initiative is promising, in that it aims to combat the apparently growing trend of publishing studies that are later found to be invalid.
And it seems that an overwhelming number of published studies are invalid; that is, they can t be reproduced. For instance, Bayer Healthcare reported last year that about 75 percent of their published work in cardiovascular disease, cancer, and women s health could not be replicated. Similarly, C. Glenn Begley, former head of global cancer research at Amgen, and Lee Ellis of M.D. Anderson Cancer Center reported that the Science Exchange scientists were able to confirm the results of only 6 out of 53 prominent studies in basic cancer biology.
Why such a prevalence of invalid study findings? Well for one, a researcher s incentive to publish may compromise the integrity of the research. In addition, flashy studies promoted by both the researchers institution and the journal often grab the attention of the media, making even dubious studies tempting.
Yet as ACSH s Dr. Gilbert Ross observes, It is detrimental to scientific research and to the public to publish flawed or exaggerated data. It not only hinders the development of new technologies and treatments, he says, but it also hurts less sophisticated consumers who trust the media to give them the facts.
ACSH s Dr. Josh Bloom is not terribly surprised that many experiments or studies are difficult to reproduce. Once you start working with living systems like animals and humans, reproducibility is often a problem, even in the same lab, he says. And when other labs try to repeat the same work, it is not at all uncommon for them to be unable to do so. The further you get away from test tubes, the more variability you get. This does not necessarily make the science or the scientists bad, but it can inadvertently lead to misguided policies and standards not supported by reliable evidence.
The Reproducibility Initiative hopes to limit the number of invalid studies published, thus facilitating the self-correcting nature of scientific research. Researchers can apply to the initiative to have their data and results confirmed by another, qualified lab, as appointed by its 10-member board of prominent scientists.