A symposium was held a few weeks ago in the U.K., sponsored by several academic institutions, with the participation of journal editors, academics, business interests (pharmaceutical), among others. The topic was of utmost importance and interest: Symposium on the reproducibility and reliably of biomedical research. Nothing less!
The sponsors were (all based in the UK): The Academy of Medical Sciences, Medical Research Council, and Biotechnology and Biological Sciences Research Council, and the Wellcome Trust. In The Lancet s April 11th edition, editor-in-chief Dr. Richard Horton published his own perspectives on the symposium.
In what amounts to an editorial, entitled What is medicine s 5 sigma, he wrote:
A lot of what is published is incorrect. I m not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Why the paranoid concern for secrecy and non-attribution? Because this symposium on the reproducibility and reliability of biomedical research touched on one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations. The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, poor methods get results.
This is chilling information: the sum and substance of it is that our edifice of scientific progress, the peer-reviewed medical/scientific literature, is a can of worms rather than the gold standard we thought it to be. Well, not all of us: we here at ACSH have often taken pains to skewer published studies in respected journals which are clearly flawed, data-dredged junk, and/or clearly devoted to propagating the researcher s career-oriented agenda, facts be damned.
Indeed, a news article on this symposium referred to a citation from yet another former editor-in-chief and her opinion of the sad state of peer-reviewed publication: Dr. Marcia Angell, a physician and longtime Editor in Chief of the New England Medical Journal (NEMJ)[sic], which is considered to another one of the most prestigious peer-reviewed medical journals in the world, makes her view of the subject quite plain:
It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine.
These editors, charged with making the decisions of such publications, should clearly be pointing the finger of guilt at themselves; yet, I saw no solutions being offered. Dr. Horton s comments in particular are nothing if not ironic, given that he is responsible for possibly the worst publication decision in recent times: Andrew Wakefield's fraudulent study in The Lancet in 1998 alleging some link between the MMR vaccine and autism, which led to a significant decline in vaccinations rates in the UK and globally. Another scientist notable for pointing out the lack of support for much of the published medical literature has been Stanford s Dr. John Ioannidis, who tried to reproduce many published studies and found that less than half were reproducible.
