Eminence over Evidence: The Lancet's COVID-19 Retraction

For those of you needing to catch up on the story, you can look here. But the gist is that The Lancet published an article on COVID-19 and hydroxychloroquine based on data that no one other than one of the study's four authors had seen or analyzed. The data itself turns out to be possibly wrong, or worse, and no one else can see it. 

The authors

Only one of the four authors of the paper had access to the data, Dr. Sepan Desai, who is the founder of the company Surgisphere that allegedly collected the data. The Lancet article indicates that he provided the statistical analysis of the dataset, which was subsequently discussed in the paper by the other authors. Since they had known one another for several years and had no reason to doubt his veracity, they accepted the analysis sight unseen. And while that approach frequently works in one-to-one relationships, should that be the standard for co-authorship?

"Mandeep Mehra, a Harvard University doctor who was a co-author on that study, said: "It is now clear to me that in my hope to contribute this research during a time of great need, I did not do enough to ensure that the data source was appropriate for this use. For that, and for all the disruptions — both directly and indirectly — I am truly sorry." [1]

While this explanation is not quite "the dog ate my homework," it is not an apology. None of the three co-authors that retracted the paper, Dr. Desai, did not participate in the retraction, even looked at the data, and these are individuals that work in academic medical centers and who should be familiar with datasets from registries.

The Lancet

"The Lancet is a world-leading medical journal. We have a Journal Impact Factor of 59·102® (2018 Journal Citation Reports®, Clarivate Analytics 2019) and are currently ranked second out of 160 journals in the Medicine, General & Internal subject category." [2]

You would think that peer review of such an august journal would be a significant hurdle to publication, you would be wrong. Like the co-authors, the peer reviewers clearly had no access to the dataset. Peer review meant assuming the integrity of the data is the paper, and merely determining with the subsequent discussion was logical and relevant. It was the rapid and intense objection to the data by Lancet readers that lead to "serious scientific questions have been brought to our attention." The retraction, written by the co-authors, indicates that the independent reviewers they brought in to respond to the serious questioning "were not able to conduct an independent and private peer review and therefore notified us of their withdrawal from the peer-review process." How refreshing, perhaps, this is a standard that The Lancet might wish to emulate. But more importantly, where is the mea culpa or at least introspection on the part of the journal? Perhaps they want to mount a Facebook defense, they are only a "platform" and have no control over the content of their publications.

Innocent victims

The work of the World Health Organization has come in for a lot of scrutiny, often politically motivated, in the last few weeks. So I understand why in the face of the publication of the now-retracted paper, they put their hydroxychloroquine study on pause – first do no harm is an important principle.

"Before the retraction, Soumya Swaminathan, the WHO's chief scientist, told the Financial Times: "In hindsight, you can say maybe we should have asked for the database, we should have examined [it], but that's not normal, especially when it's published in The Lancet." [1]

Ah, there it is, eminence over evidence. Just as some people believe that if it is on the Internet, it must be true, evidently we all fall for that fallacy. If the Lancet publishes it, then it must be true. Perhaps that was a strong belief in the past, but publication by the second most impactful journal now merits questioning. Swaminathan does indicate that "in hindsight, one could criticise "the failure to examine the database supporting the Lancet study, "but the decision was based on an expert group considering all evidence and making an informed decision with a view to protecting patients." And while that might be true, and I am sure that it is, the experts were considering the evidence available to them, primarily Lancet's peer-reviewed publication. They were not considering all the evidence; they did not have independent verification of the data – they too assumed that this was the Lancet's or the authors' responsibility. 

The other great victims, of course, are you and I. The paper, when published quickly spread through the medical and mainstream media. It was from a reputable journal, and it fed or denied the now politicized narrative that hydroxychloroquine was or was not an effective treatment for COVID-19. It sowed doubt; it makes us less trusting in those we have entrusted with our scientific thought and publication. It fuels a belief for anti-vaxxers that there "Big Medicine," and they can't be trusted. It makes people that trusted the Lancet look foolish. 

We need to take a hard look at this event. The Lancet needs to decide if they are a responsible medical journal or just a Facebook wannabe. The authors need to consider how they collaborate with others; the chain of integrity is no stronger than its weakest link, what checks and balances must be present to put your name on a paper?  

[1] Financial Times, WHO says it did not see Surgisphere data that halted virus drug trial

[2] About the Lancet