Clinical Trial in Bangladesh Shows Masks Stop COVID-19 (Sort of)

“Massive randomized study is proof that surgical masks limit coronavirus spread,” the Washington Post said of a recent clinical trial conducted in Bangladesh. “Its authors say this provides conclusive, real-world evidence for what laboratory work and other research already strongly suggest: mask-wearing can have a significant impact on limiting the spread of symptomatic covid-19 ...”

If you survey the research on the efficacy of masks, it's clear that what the public is told about the evidence and what the evidence actually shows are very different things. See my story Fact-Checking The Fact-Checkers: What Do Studies Say About Masks And COVID-19? for more on that. While this study confirms that wearing a surgical mask is better than nothing, we're again watching the media oversimplify the study's results.

First, let's look at the Post's summary of the study.

About 178,000 Bangladeshi villagers were in an intervention group and encouraged to use masks. An additional 163,000 were in a control group, where no interventions were made … The researchers estimate that ... mask-wearing increased by 28.8 percent after the intervention. When tracked, this group saw a 9.3 percent reduction in symptomatic covid-19 seroprevalence, meaning the virus was confirmed by bloodwork, as well as a further 11.9 percent reduction in covid-19 symptoms.

The newspaper only reported the total number of participants alongside the percentages for mask-wearing, seroprevalence and the reduction in symptoms. Who knows why they did that, but the other numbers are also worth noting, even if just for the sake of completeness.

According to the study, “The proportion of individuals with COVID-like symptoms was 7.62% (N=13,273) in the intervention arm and 8.62% (N=13,893) in the control arm.” In other words, there were 620 fewer people with COVID-like symptoms in an experimental group of 178,288 individuals. Symptoms were self-reported by each household, the authors wrote:

The owner of the household’s primary phone completed surveys by phone or in-person at weeks 5 and 9 after the start of the intervention. They were asked to report symptoms experienced by any household member that occurred in the previous week and over the previous month.

Additionally, based on blood samples collected from 9,977 symptomatic participants (5,006 from the masked group and 4,971 controls), the study found that masking “led to a 9.3% reduction in symptomatic SARS-CoV-2 seroprevalence (which corresponds to a 103 fewer symptomatic seropositives) and an 11.9% reduction in the prevalence of COVID-like symptoms, corresponding to 1,587 fewer people reporting these symptoms.”

Cloth masks had essentially no effect, what the authors called “an imprecise zero.” The 200 villages (out of 600 in the study) randomized to wear surgical masks saw somewhat better results, though the risk reduction was statistically insignificant for anyone under 50 years old.

Why the older age groups would see significantly better results isn't clear. If masks work, we should have seen some effect in the under-50 group. My colleague Dr. Chuck Dinerstein has alternatively argued in his analysis that the alpha variant, the primary strain in circulation during the study, didn't spread well among younger people, which may explain the divergent impact in different age groups.

Important limitations

The experiment also had several important limitations. Perhaps most importantly, it's difficult to track compliance in a study of this nature. The researchers tried to do so “through direct observation in public locations,” including a village's market, restaurant entrances, main road, tea stalls, and mosque. But they only monitored these public locations for an hour at a time “at baseline and then once per week on weeks 1, 2, 4, 6, 8, and 10 after the intervention.”

Crucially, “observations were conducted outside except at the mosque, where surveillance was conducted inside.” This is because rural Bangladesh isn't home to many of the indoor venues people frequent in more developed locales. Of course, there is very little evidence that SARS-COV-2 spreads well outdoors; that means the study doesn't tell us much about the efficacy of masking indoors, where most governments recommend or require it.

Finally, since alpha was the primary variant in circulation during the study period, we have to be careful about applying these results to the delta strain currently causing all the chaos. We wouldn't assess a vaccine's efficacy against delta based on its ability to control alpha. The same rule ought to apply to masking. 

None of this is the researchers' fault; indeed, the study was well-conducted given the number of people and villages involved. And they had no control of which variant they were up against. But a study's limitations, unsurprisingly, limit its significance.

Trust is paramount, as always

With all that in mind, consider this quote from one of the study authors included in the Washington Post story:

'I think this should basically end any scientific debate about whether masks can be effective in combating covid at the population level,' Jason Abaluck, an economist at Yale who helped lead the study, said in an interview, calling it a 'a nail in the coffin' of the arguments against masks.

This is technically correct, but the key question is, "how effective?" Later in the story, the same researcher gave readers a little more context:

The authors found that while cloth masks clearly reduced symptoms, they 'cannot reject' the idea that unlike surgical masks, they may have only a small effect on symptomatic coronavirus infections, and possibly none at all.  

Abaluck emphasized, however, that research did not produce evidence that cloth masks are ineffective.  The results 'don’t necessarily show that surgical masks are much, much better than cloth masks, but we find much clearer evidence of the effectiveness in surgical masks,' he said.

In sum, certain masks have a limited effect on transmission. As Dr. Dinerstein explained, this is probably important in a country like Bangladesh, where COVID vaccines are not widely available. But reporting the results of this paper as if they settle the dispute around masking more generally is misleading. Skeptics are smart; they can look up study results just as well as WaPo reporters. 

When you tell people without qualification that a study is "proof that surgical masks limit coronavirus spread," they begin ignoring everything you say. We've seen this phenomenon all throughout the pandemic. If you're dumbfounded by the lingering vaccine hesitancy in the US, or the thousands of people protesting lockdowns and vaccine passports around the world, one of the key reasons for their resistance is on full display right now.