A Massive Study Flagged Concerns About Hydroxychloroquine Last Month. The Authors Just Retracted It.

Cadu Rolim/Fotoarena/ZUMA

The coronavirus is a rapidly developing news story, so some of the content in this article might be out of date. Check out our most recent coverage of the coronavirus crisis, and subscribe to the Mother Jones Daily newsletter.

Over the past few months, there has been a pretty heated debate raging in both scientific and political circles over the antimalarial drug hydroxychloroquine. The broad outlines are likely familiar at this point: President Trump has touted its effectiveness to treat COVID-19, even claiming to be taking it himself as a preventative measure, without scientific evidence to back him up and even though the FDA has warned against its use outside of hospital settings. 

Now there’s a new development: On Thursday, researchers retracted a major study that suggested a link between hydroxychloroquine and a substantially increased risk of death. Another leading journal has issued a statement that questions the integrity of related research.

The now-retracted study, which published last month in the Lancet, a respected research journal, included analysis of 96,032 patients’ records and was at the time the largest study to investigate the drug’s potential in fighting COVID-19. According to the study, patients who received a cocktail of hydroxychloroquine and an antibiotic saw a 45 percent increased risk of death. The news was a big deal. Several media organizations, including Mother Jones, picked up the study. The World Health Organization, which had been investigating hydroxychloroquine’s potential as a coronavirus treatment, paused recruitment for its trial. And Italy, France, and Belgium banned the drug from being used to treat COVID-19.

Still, questions about the study started popping up shortly after it published; more than 180 researchers issued a letter raising potential problems with the data behind it, and on May 29, the Lancet corrected some of the data. Now, just shortly after the Lancet and the New England Journal of Medicine published “expressions of concern” about the research, the authors have officially retracted the paper. 

“Today, three of the authors of the paper, ‘Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis’, have retracted their study,” the Lancet announced in a statement. “They were unable to complete an independent audit of the data underpinning their analysis. As a result, they have concluded that they ‘can no longer vouch for the veracity of the primary data sources.'”

NEJM, which published a study on May 1 drawing from the same data source, wrote in an expression of concern, “Recently, substantive concerns have been raised about the quality of the information in that database. We have asked the authors to provide evidence that the data are reliable.”

Without access to the data, it’s unclear what exactly, if anything, went wrong, though I recommend you read this excellent investigation by the Guardian into the company that provided the dataset. But what we do know now is that the controversy is concerning—if not unsurprising, given how fast the science is moving around COVID-19 and how desperate people are for information on the disease that’s claimed more than 100,000 American lives.

As I’ve written before, experts say the pandemic has put a strain on an already flawed scientific review process. The coronavirus has brought on a deluge of manuscripts for journals to review. Multiple journals, including NEJM, told me in April that their peer-review process has been expedited. But even in normal times, peer-reviewed journals often don’t review raw data as part of the vetting process in the first place. “I would argue maybe they should more often because that might prevent cases like this,” Ivan Oransky, a founder of Retraction Watch, a blog that tracks academic retractions, told me earlier this week after the Lancet study had been flagged for concern, but before it was retracted; Oransky also serves as the vice president of editorial at medical news and reference site Medscape and teaches medical journalism at New York University. “Journals hardly ever look at the underlying data.” Other times bias and error simply slip by peer-reviewers.

When Mother Jones asked the Lancet via email on Thursday how this paper made it past peer-review, the journal did not answer directly. It instead underscored that the paper was, in fact, peer-reviewed. “All research articles published in The Lancet journals undergo independent, external peer review, including statistical review,” it wrote. (NEJM simply told Mother Jones that the expression of concern “must speak for itself for now.”)

The Lancet study in question is one of a handful of COVID-related papers that have been retracted already, though this might be the most high-profile. So far, Retraction Watch has documented 14 retractions (including today’s) and two temporary retractions for recent coronavirus-related research, as well as two expressions of concern about COVID-19 papers (including NEJM‘s notice this week).

Generally speaking, expressions of concern are not common in the science world, but they’re not unheard of either. “An expression of concern is used when editors have a reasonable sense that there might be something wrong but they’re not sure yet. So it’s not a retraction, but it’s grander than a correction,” says Oransky. Retractions are more serious. On average, according to Oransky, the amount of time between when a paper is published and withdrawn is about three years. Again, the pandemic has put this entire timeline into super-speed.

Both papers using this now-disputed data were authored by a team of researchers led by Dr. Mandeep Mehra, a professor of medicine and a cardiovascular specialist at Harvard Medical School. A co-author is Sapan Desai, who leads Surgisphere, the data firm which supplied the COVID dataset for the studies. The Guardian investigation raises a slew of concerns about the validity of the data, including significant data discrepancies and conversations with hospitals that said they did not participate. It also reports that Desai “has been named in three medical malpractice suits, unrelated to the Surgisphere database” and that several of the company’s employees have “little or no data or scientific background.” (Again, it’s worth reading the Guardian‘s investigation in full.) Despite pleas from the scientific community, the authors of the initial study have yet to supply the raw data from the studies, citing confidentiality concerns. 

Complicating matters on hydroxychloroquine even further, the first major controlled clinical trial of the drug was published on Wednesday in the New England Journal of Medicine and found that it does not help prevent infection. As the New York Times reports:

The malaria drug hydroxychloroquine did not prevent Covid-19 in a rigorous study of 821 people who had been exposed to patients infected with the virus, researchers from the University of Minnesota and Canada are reporting on Wednesday…. Conducted in the United States and Canada, this trial was also the first to test whether the drug could prevent illness in people who have been exposed to the coronavirus.

“The take-home message for the general public is that if you’re exposed to someone with Covid-19, hydroxychloroquine is not an effective post-exposure preventive therapy,” the lead author of the study, Dr. David R. Boulware, from the University of Minnesota, said in an interview.

Though both dispute the effectiveness of hydroxychloroquine, this new NEJM study and the now-retracted Lancet study have highly different methodologies. The Lancet paper was observational: The researchers had no control over who received or did not receive treatment. A controlled clinical trial, on the other hand, which happened in the new NEJM study, is a much more rigorous experiment (though it’s still not foolproof).

If there’s one definitive takeaway this week it’s that the research is moving quickly. And a lot of it is incomplete. At a time when we need good science the most, experts tell me bad science is slipping through the cracks at an alarming rate. “What [the pandemic] has done is just made everyone rush to publication and rush to judgment, frankly,” Oransky told me in April. “You’re seeing papers published in the world’s leading medical journals that probably shouldn’t have even been accepted in the world’s worst medical journals.”

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with the Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

AN IMPORTANT UPDATE

We’re falling behind our online fundraising goals and we can’t sustain coming up short on donations month after month. Perhaps you’ve heard? It is impossibly hard in the news business right now, with layoffs intensifying and fancy new startups and funding going kaput.

The crisis facing journalism and democracy isn’t going away anytime soon. And neither is Mother Jones, our readers, or our unique way of doing in-depth reporting that exists to bring about change.

Which is exactly why, despite the challenges we face, we just took a big gulp and joined forces with the Center for Investigative Reporting, a team of ace journalists who create the amazing podcast and public radio show Reveal.

If you can part with even just a few bucks, please help us pick up the pace of donations. We simply can’t afford to keep falling behind on our fundraising targets month after month.

Editor-in-Chief Clara Jeffery said it well to our team recently, and that team 100 percent includes readers like you who make it all possible: “This is a year to prove that we can pull off this merger, grow our audiences and impact, attract more funding and keep growing. More broadly, it’s a year when the very future of both journalism and democracy is on the line. We have to go for every important story, every reader/listener/viewer, and leave it all on the field. I’m very proud of all the hard work that’s gotten us to this moment, and confident that we can meet it.”

Let’s do this. If you can right now, please support Mother Jones and investigative journalism with an urgently needed donation today.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate