That Cochrane review
Mar. 10th, 2023 01:47 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
livSo there was an update to the Cochrane review on masks to reduce the spread of respiratory viruses. It concluded that
The reception of this review has been fascinating, though extremely depressing. The lead author, Tom Jefferson, gave an interview informally summarizing the conclusion as,
Very quickly, all of public-facing social media was absolutely full of highly networked, bot-like accounts posting a link to the study on every thread that referenced Covid precautions, saying, LOL, masks don't work! (I don't think all the comments to this effect were posted by literal automated bots, but by the kind of mix of human and automated postings that swarm Twitter repeating the same message when it's convenient to someone influential.) There were even a few anonymous posts on my quiet backwater little DW with a similar message, the Cochrane review shows masks don't work, you can stop bothering. Some people I actually know also mentioned that the review had convinced them to give up on masks.
There were also a bunch of debunkings which seemed very knee-jerk, again, particularly on Twitter but all over the place. A much retweeted thread which actively lied about the studies included in the meta-analysis and all but libelled the authors. I read a lot of stuff. I dismissed very extreme refutations which amounted to little more than, well, everybody knows masks work so if this meta-analysis showed they don't it must be biased and wrong. That's not very scientific, that's just confirming your pre-existing views. I skimmed the review itself but didn't read all the studies cited in it, but gave myself enough of a sense of when people referring to it were reporting broadly true facts about its findings, rather than making up nonsense to support either the pro- or anti-mask cause.
[Sidebar: This, by the way, is an example of why I disagree withsiderea that
The conclusion I tentatively reached was that there are, essentially, two well-powered, high quality RCTs of mask wearing for prevention of actual Covid. The Bangladesh study which found a modest but consistent positive effect, and the DANMASK study which found little to no effect (and like the review that relies on it, was widely cited as definite proof for the anti-mask case when it came out). You can't do a meta-analysis by combining only two studies, so the Cochrane team had a choice: they could include studies that aren't RCTs, or they could include studies that are not on Covid. They chose the latter, which is not a completely unreasonable decision and I don't think I'm better able to judge than actual experts in medical evidence which way they should have gone. Also, it was an update of a pre-Covid review about respiratory illness in general, so it makes sense to include studies from before the emergence of Covid.
But to me this meant that the meta-analysis added little to what I already knew, that the Bangladesh study found masks to be somewhat effective in a real-world setting, and the Danish study found that in a situation where case levels are already extremely low, they weren't able to measure much difference between almost no cases in the control group and almost no cases in the intervention group. I did a lot of reflecting about my own attitudes; I certainly trust Cochrane reviews on the whole, and I certainly want to be the sort of person who updates my opinions based on new evidence. I was pretty sure I had a bias towards looking for flaws in the Cochrane review, and I tried to compensate for that.
For some people, "masks don't work" is good news, because it means they can push back against anyone asking them to wear a mask when they don't want to. But I don't feel like that at all; I would be very unhappy to learn that masks don't work, because it wouldn't mean time-travelling back to 2019 as I'm sure many of us would love to be able to do, rather it would mean that the main precaution I'm taking to be able to do any in-person activities at all is essentially worthless. Obviously my unhappiness is completely the opposite of evidence about what is actually the reality out there. So I wanted to be very careful not to over-weight evidence that aligns with what I want to be true.
I had some amount of trust in this rebuttal by Macintyre et al. It contains a great deal of non-RCT, but compelling, evidence in favour of masks, and some convincing arguments why RCTs based purely on outcomes for the mask-wearer may not be the only good quality evidence for this question. It also has what looks to me like a valid, not hysterical, critique of the Cochrane review, namely that they are comparing a lot of different things that aren't really comparable, and also that many of the studies which came out not showing much effect were studies of whether telling people to wear masks is effective and didn't take into account compliance or lack thereof. I generally trust the last author, Trish Greenhalgh, but she has been passionately pro-mask throughout and indeed it's reading her research that convinced me wearing masks was worthwhile even when the official advice was against it. Macintyre herself declares a conflict of interest in that she works for mask manufacturers, which is a pretty glaring source of potential bias!
My conclusion until today was: telling people to wear masks in the community, regardless of Covid levels, and regardless of mask quality, doesn't strongly predict better outcomes (less Covid, in this case). But there is pretty good evidence that well-fitted masks worn by most people do actually provide some protection when Covid levels are high, even if there isn't much in the way of well-powered RCTs showing benefit. I held that conclusion pretty lightly, because I knew I was biased and because I hadn't actually read all the original studies in detail, mostly summaries.
The latest news is that the editor-in-chief of the Cochrane library has semi-retracted the notorious review. As reported by Zeynep Tufekci in the New York Times, Soares-Weiser has issued a statement that
The remainder of the article summarizes some of the (non RCT) evidence in favour of masks, and notes that the Cochrane review lead author has a history of anti-mask views and even pandemic minimizing. I have a lot of time for Tufekci, but some people don't like her because she doesn't come from a scientific or medical background, and she tends to take into account evidence from a range of sources and in particular doesn't accept some of the most extreme Covid catastrophizing. IMO she's been pretty consistently right, supporting masks and opposing restrictions on outdoor socializing from very early on, campaigning for Long Covid sufferers but rejecting absurdly high projections of levels of severe long-term organ and immune damage. In my view the important thing is the statement by Soares-Weiser where she is very clear that the Cochrane review has been misinterpreted as being definitively anti-mask.
So there you go, that takes us more or less back to the status quo ante, before the January update of the Cochrane Review. We have some evidence in favour of some types of masks in some situations, but we can't definitively say 'masks work' or 'masks don't work'. I think that may be some of the reason why some of the mask debate has become so painful and polarized; a lot of people don't feel comfortable with the idea of a mitigation that has some effect but isn't perfect, they want interventions to either "work" or "not work". Same as the vaccination debate. Vaccines, like masks, give some protection, but they're not, as Tufecki puts, it,
Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks.
The reception of this review has been fascinating, though extremely depressing. The lead author, Tom Jefferson, gave an interview informally summarizing the conclusion as,
There is just no evidence that [masks] make any difference.. That's not a totally ridiculous single sentence summary of the conclusions of the meta-analysis, but it's in the context of a very inflammatory article. Some mainstream media outlets ran the story as if it were, new study proves masks make ZERO difference. (Not linking to any of those directly, I'm sure you'll have seen them around. But deeply misleading by confusing a study with a meta-analysis of previous studies, and assuming that a finding of no evidence counted as "proof" of no effect.)
Very quickly, all of public-facing social media was absolutely full of highly networked, bot-like accounts posting a link to the study on every thread that referenced Covid precautions, saying, LOL, masks don't work! (I don't think all the comments to this effect were posted by literal automated bots, but by the kind of mix of human and automated postings that swarm Twitter repeating the same message when it's convenient to someone influential.) There were even a few anonymous posts on my quiet backwater little DW with a similar message, the Cochrane review shows masks don't work, you can stop bothering. Some people I actually know also mentioned that the review had convinced them to give up on masks.
There were also a bunch of debunkings which seemed very knee-jerk, again, particularly on Twitter but all over the place. A much retweeted thread which actively lied about the studies included in the meta-analysis and all but libelled the authors. I read a lot of stuff. I dismissed very extreme refutations which amounted to little more than, well, everybody knows masks work so if this meta-analysis showed they don't it must be biased and wrong. That's not very scientific, that's just confirming your pre-existing views. I skimmed the review itself but didn't read all the studies cited in it, but gave myself enough of a sense of when people referring to it were reporting broadly true facts about its findings, rather than making up nonsense to support either the pro- or anti-mask cause.
[Sidebar: This, by the way, is an example of why I disagree with
siderea that I think you should trust me (within limits) because I cite my sources.. Lots of not at all trustworthy people cite sources all the time. They post lists and lists of links to highly technical academic articles, and make completely untrue claims about what those articles show, and hope that you won't have time or the right skills to go back to the cited source and notice for yourself that the summary is a distortion, or even the opposite, of what the purported citation shows. Like, there's literally a playbook circulating in the nasty bits of the internet with a bibliography of peer reviewed journal articles that purport to show that intelligence is genetic and correlated with race. Transphobes regularly cite "studies" which apparently show that trans people are a danger to normal children and cis women, when the studies either have been debunked or show no such thing. Even Wikipedia, with its
citation neededculture, can easily be used for disinformation because a pile of little blue numbers make false articles look authoritative, and people believe the air of academic-ness without noticing that the citations don't support what is claimed on the page. So anyway, lots of people "cite" the Cochrane review or the studies within it, but are wrong, intentionally or accidentally, about what such sources say.]
The conclusion I tentatively reached was that there are, essentially, two well-powered, high quality RCTs of mask wearing for prevention of actual Covid. The Bangladesh study which found a modest but consistent positive effect, and the DANMASK study which found little to no effect (and like the review that relies on it, was widely cited as definite proof for the anti-mask case when it came out). You can't do a meta-analysis by combining only two studies, so the Cochrane team had a choice: they could include studies that aren't RCTs, or they could include studies that are not on Covid. They chose the latter, which is not a completely unreasonable decision and I don't think I'm better able to judge than actual experts in medical evidence which way they should have gone. Also, it was an update of a pre-Covid review about respiratory illness in general, so it makes sense to include studies from before the emergence of Covid.
But to me this meant that the meta-analysis added little to what I already knew, that the Bangladesh study found masks to be somewhat effective in a real-world setting, and the Danish study found that in a situation where case levels are already extremely low, they weren't able to measure much difference between almost no cases in the control group and almost no cases in the intervention group. I did a lot of reflecting about my own attitudes; I certainly trust Cochrane reviews on the whole, and I certainly want to be the sort of person who updates my opinions based on new evidence. I was pretty sure I had a bias towards looking for flaws in the Cochrane review, and I tried to compensate for that.
For some people, "masks don't work" is good news, because it means they can push back against anyone asking them to wear a mask when they don't want to. But I don't feel like that at all; I would be very unhappy to learn that masks don't work, because it wouldn't mean time-travelling back to 2019 as I'm sure many of us would love to be able to do, rather it would mean that the main precaution I'm taking to be able to do any in-person activities at all is essentially worthless. Obviously my unhappiness is completely the opposite of evidence about what is actually the reality out there. So I wanted to be very careful not to over-weight evidence that aligns with what I want to be true.
I had some amount of trust in this rebuttal by Macintyre et al. It contains a great deal of non-RCT, but compelling, evidence in favour of masks, and some convincing arguments why RCTs based purely on outcomes for the mask-wearer may not be the only good quality evidence for this question. It also has what looks to me like a valid, not hysterical, critique of the Cochrane review, namely that they are comparing a lot of different things that aren't really comparable, and also that many of the studies which came out not showing much effect were studies of whether telling people to wear masks is effective and didn't take into account compliance or lack thereof. I generally trust the last author, Trish Greenhalgh, but she has been passionately pro-mask throughout and indeed it's reading her research that convinced me wearing masks was worthwhile even when the official advice was against it. Macintyre herself declares a conflict of interest in that she works for mask manufacturers, which is a pretty glaring source of potential bias!
My conclusion until today was: telling people to wear masks in the community, regardless of Covid levels, and regardless of mask quality, doesn't strongly predict better outcomes (less Covid, in this case). But there is pretty good evidence that well-fitted masks worn by most people do actually provide some protection when Covid levels are high, even if there isn't much in the way of well-powered RCTs showing benefit. I held that conclusion pretty lightly, because I knew I was biased and because I hadn't actually read all the original studies in detail, mostly summaries.
The latest news is that the editor-in-chief of the Cochrane library has semi-retracted the notorious review. As reported by Zeynep Tufekci in the New York Times, Soares-Weiser has issued a statement that
Many commentators have claimed that a recently updated Cochrane Review shows that ‘masks don’t work’, which is an inaccurate and misleading interpretation ... [our] wording was open to misinterpretation, for which we apologize. (I am aware that some people are boycotting the NYT due to its horrible transphobia; right now they are the outlet breaking this news, so feel free to wait until other press wires pick it up if you don't want to give them clicks. Also, I bypassed the paywall by turning off JavaScript; up to you whether you're comfortable with doing that.)
The remainder of the article summarizes some of the (non RCT) evidence in favour of masks, and notes that the Cochrane review lead author has a history of anti-mask views and even pandemic minimizing. I have a lot of time for Tufekci, but some people don't like her because she doesn't come from a scientific or medical background, and she tends to take into account evidence from a range of sources and in particular doesn't accept some of the most extreme Covid catastrophizing. IMO she's been pretty consistently right, supporting masks and opposing restrictions on outdoor socializing from very early on, campaigning for Long Covid sufferers but rejecting absurdly high projections of levels of severe long-term organ and immune damage. In my view the important thing is the statement by Soares-Weiser where she is very clear that the Cochrane review has been misinterpreted as being definitively anti-mask.
So there you go, that takes us more or less back to the status quo ante, before the January update of the Cochrane Review. We have some evidence in favour of some types of masks in some situations, but we can't definitively say 'masks work' or 'masks don't work'. I think that may be some of the reason why some of the mask debate has become so painful and polarized; a lot of people don't feel comfortable with the idea of a mitigation that has some effect but isn't perfect, they want interventions to either "work" or "not work". Same as the vaccination debate. Vaccines, like masks, give some protection, but they're not, as Tufecki puts, it,
a talisman or a magic wand.
(no subject)
Date: 2023-03-14 03:28 pm (UTC)I don't think shouting at people about the exact numbers of how much protection you get from masks in exactly what circumstances is at all productive; people who have made anti-mask part of their identity aren't going to accept any evidence anyway, much less emotional pressure from people they perceive as angry and threatening. The group worth persuading, I think, are people who basically accept the consensus that masks are protective, but believe that the downsides of masking are worse than repeatedly catching Covid.
Like you I wear a mask when it's practical; I don't find them particularly unpleasant, and also: masks are a physical barrier. The level of evidence I need in order to believe that a physical barrier which prevents viruses from entering your lungs is effective, is much lower than the level of evidence I need to believe in a pharmaceutical intervention. I might get Covid while wearing a mask, but I probably won't ever know whether it was because the mask wasn't sealed well enough, or because I was unlucky and got exposed when I took the mask off briefly, or when I intentionally removed the mask because outdoors is IMO 'safe enough'. But I am mostly relying on trying to minimize my exposures as much as I can.