We Have No Idea Which Interventions Work Against COVID-19

Handwashing definitely works against COVID-19. But does anything else?Kevin Drum

Facts matter: Sign up for the free Mother Jones Daily newsletter. Support our nonprofit reporting. Subscribe to our print magazine.

A while back I mentioned that we were in new territory with COVID-19: we’ve had similar or bigger pandemics before, but we’ve never had one where we adopted widespread countermeasures and then studied how well they worked. This was an offhand comment and I got curious today if it was really true. Spoiler alert: It is.

I went looking for empirical studies done before 2020 since I was interested only in pandemics prior to COVID-19. It didn’t take long to come across “Effectiveness of workplace social distancing measures in reducing influenza transmission: a systematic review,” which was published in 2018. It’s a review of previous studies of social distancing and other countermeasures in “socially dense community settings, such as schools or workplaces,” and it’s far more comprehensive than anything I could do myself. Here’s what they found:

  • A grand total of 15 studies.
  • Of which, only three were epidemiological (the others were modeling studies).
  • Of which, the overall risk of bias was rated “critical” in one, meaning the “study is too problematic to provide any useful evidence on the effects of intervention.”
  • That leaves two studies. The risk of bias was rated “serious” in both, but at least they weren’t completely useless.
  • Of those, one was a study of ordinary seasonal influenza in 2007-08 and looked only at employees who could work at home vs. those who couldn’t.

The sole remaining study was conducted on a group of 1,015 miltary personnel in Singapore during the 2009 H1N1 pandemic. It compared a control group (no intervention) to a “normal” group (individuals were provided general health education on respiratory and hand hygiene and were advised to seek medical care if ill) and an “essential” group (which received enhanced surveillance with isolation, segregation, and personal protective equipment). The study found that 44 percent of the control group contracted the flu compared to 17 percent of the normal group and 11 percent of the essential group. It also found that the pandemic peaked earlier in normal units compared to the control group.

So that’s it: a single study that compared groups to each other but didn’t try to establish the efficacy of individual interventions. Normal hygeine advice apparently had a big effect, reducing incidence from 44 percent to 17 percent, but only the “essential” group practiced any kind of social distancing, and it reduced incidence of the flu only from 17 percent to 11 percent. This may sound worthwhile even though it’s small, but don’t forget that this study had “serious” problems with possible bias.

Effectively, this means we’re flying blind. We have loads of modeling estimates of various interventions:

  • Quarantines
  • Stay-at-home orders
  • School closings
  • Social distancing
  • Mask wearing
  • Bans on large gatherings
  • Closure of restaurants
  • Closure of non-essential businesses
  • Mass testing and contact tracing

For practical purposes, however, we have no reliable empirical data at all on any of these measures. It makes sense that some or all of them have an effect—and it’s probably safe to say that all of them put together have an effect—but we have no idea which particular ones have a large effect vs. which ones have a small effect. And given the vast range of assumptions used in various models, it’s not clear to me that we can trust models to tell us anything useful at the level of specific interventions.

It’s astonishing how much we’re learning about COVID-19 on literally a daily basis. In some ways it works like any other pandemic, but in other ways it truly appears to be unique. Given this, and given the complete lack of good empirical studies of past pandemics, we should be very, very cautious about insisting on any particular intervention as either critical or dispensable. We just don’t know.

REAL QUICK, REAL URGENT

Minority rule, corruption, disinformation, attacks on those who dare tell the truth: There is a direct line from what's happening in Russia and Ukraine to what's happening here at home. And that's what MoJo's Monika Bauerlein writes about in "Their Fight Is Our Fight" to unpack the information war we find ourselves in and share a few examples to show why the power of independent, reader-supported journalism is such a threat to authoritarians.

Corrupt leaders the world over can (and will) try to shut down the truth, but when the truth has millions of people on its side, you can't keep it down for good. And there's no more powerful or urgent argument for your support of Mother Jones' journalism right now than that. We need to raise about $450,000 to hit our online fundraising budget in these next few months, so please read more from Monika and pitch in if you can.

payment methods

REAL QUICK, REAL URGENT

Minority rule, corruption, disinformation, attacks on those who dare tell the truth: There is a direct line from what's happening in Russia and Ukraine to what's happening here at home. And that's what MoJo's Monika Bauerlein writes about in "Their Fight Is Our Fight" to unpack the information war we find ourselves in and share a few examples to show why the power of independent, reader-supported journalism is such a threat to authoritarians.

Corrupt leaders the world over can (and will) try to shut down the truth, but when the truth has millions of people on its side, you can't keep it down for good. And there's no more powerful or urgent argument for your support of Mother Jones' journalism right now than that. We need to raise about $450,000 to hit our online fundraising budget in these next few months, so please read more from Monika and pitch in if you can.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate