That Cochrane review
Mar. 10th, 2023 01:47 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
livSo there was an update to the Cochrane review on masks to reduce the spread of respiratory viruses. It concluded that
The reception of this review has been fascinating, though extremely depressing. The lead author, Tom Jefferson, gave an interview informally summarizing the conclusion as,
Very quickly, all of public-facing social media was absolutely full of highly networked, bot-like accounts posting a link to the study on every thread that referenced Covid precautions, saying, LOL, masks don't work! (I don't think all the comments to this effect were posted by literal automated bots, but by the kind of mix of human and automated postings that swarm Twitter repeating the same message when it's convenient to someone influential.) There were even a few anonymous posts on my quiet backwater little DW with a similar message, the Cochrane review shows masks don't work, you can stop bothering. Some people I actually know also mentioned that the review had convinced them to give up on masks.
There were also a bunch of debunkings which seemed very knee-jerk, again, particularly on Twitter but all over the place. A much retweeted thread which actively lied about the studies included in the meta-analysis and all but libelled the authors. I read a lot of stuff. I dismissed very extreme refutations which amounted to little more than, well, everybody knows masks work so if this meta-analysis showed they don't it must be biased and wrong. That's not very scientific, that's just confirming your pre-existing views. I skimmed the review itself but didn't read all the studies cited in it, but gave myself enough of a sense of when people referring to it were reporting broadly true facts about its findings, rather than making up nonsense to support either the pro- or anti-mask cause.
[Sidebar: This, by the way, is an example of why I disagree withsiderea that
The conclusion I tentatively reached was that there are, essentially, two well-powered, high quality RCTs of mask wearing for prevention of actual Covid. The Bangladesh study which found a modest but consistent positive effect, and the DANMASK study which found little to no effect (and like the review that relies on it, was widely cited as definite proof for the anti-mask case when it came out). You can't do a meta-analysis by combining only two studies, so the Cochrane team had a choice: they could include studies that aren't RCTs, or they could include studies that are not on Covid. They chose the latter, which is not a completely unreasonable decision and I don't think I'm better able to judge than actual experts in medical evidence which way they should have gone. Also, it was an update of a pre-Covid review about respiratory illness in general, so it makes sense to include studies from before the emergence of Covid.
But to me this meant that the meta-analysis added little to what I already knew, that the Bangladesh study found masks to be somewhat effective in a real-world setting, and the Danish study found that in a situation where case levels are already extremely low, they weren't able to measure much difference between almost no cases in the control group and almost no cases in the intervention group. I did a lot of reflecting about my own attitudes; I certainly trust Cochrane reviews on the whole, and I certainly want to be the sort of person who updates my opinions based on new evidence. I was pretty sure I had a bias towards looking for flaws in the Cochrane review, and I tried to compensate for that.
For some people, "masks don't work" is good news, because it means they can push back against anyone asking them to wear a mask when they don't want to. But I don't feel like that at all; I would be very unhappy to learn that masks don't work, because it wouldn't mean time-travelling back to 2019 as I'm sure many of us would love to be able to do, rather it would mean that the main precaution I'm taking to be able to do any in-person activities at all is essentially worthless. Obviously my unhappiness is completely the opposite of evidence about what is actually the reality out there. So I wanted to be very careful not to over-weight evidence that aligns with what I want to be true.
I had some amount of trust in this rebuttal by Macintyre et al. It contains a great deal of non-RCT, but compelling, evidence in favour of masks, and some convincing arguments why RCTs based purely on outcomes for the mask-wearer may not be the only good quality evidence for this question. It also has what looks to me like a valid, not hysterical, critique of the Cochrane review, namely that they are comparing a lot of different things that aren't really comparable, and also that many of the studies which came out not showing much effect were studies of whether telling people to wear masks is effective and didn't take into account compliance or lack thereof. I generally trust the last author, Trish Greenhalgh, but she has been passionately pro-mask throughout and indeed it's reading her research that convinced me wearing masks was worthwhile even when the official advice was against it. Macintyre herself declares a conflict of interest in that she works for mask manufacturers, which is a pretty glaring source of potential bias!
My conclusion until today was: telling people to wear masks in the community, regardless of Covid levels, and regardless of mask quality, doesn't strongly predict better outcomes (less Covid, in this case). But there is pretty good evidence that well-fitted masks worn by most people do actually provide some protection when Covid levels are high, even if there isn't much in the way of well-powered RCTs showing benefit. I held that conclusion pretty lightly, because I knew I was biased and because I hadn't actually read all the original studies in detail, mostly summaries.
The latest news is that the editor-in-chief of the Cochrane library has semi-retracted the notorious review. As reported by Zeynep Tufekci in the New York Times, Soares-Weiser has issued a statement that
The remainder of the article summarizes some of the (non RCT) evidence in favour of masks, and notes that the Cochrane review lead author has a history of anti-mask views and even pandemic minimizing. I have a lot of time for Tufekci, but some people don't like her because she doesn't come from a scientific or medical background, and she tends to take into account evidence from a range of sources and in particular doesn't accept some of the most extreme Covid catastrophizing. IMO she's been pretty consistently right, supporting masks and opposing restrictions on outdoor socializing from very early on, campaigning for Long Covid sufferers but rejecting absurdly high projections of levels of severe long-term organ and immune damage. In my view the important thing is the statement by Soares-Weiser where she is very clear that the Cochrane review has been misinterpreted as being definitively anti-mask.
So there you go, that takes us more or less back to the status quo ante, before the January update of the Cochrane Review. We have some evidence in favour of some types of masks in some situations, but we can't definitively say 'masks work' or 'masks don't work'. I think that may be some of the reason why some of the mask debate has become so painful and polarized; a lot of people don't feel comfortable with the idea of a mitigation that has some effect but isn't perfect, they want interventions to either "work" or "not work". Same as the vaccination debate. Vaccines, like masks, give some protection, but they're not, as Tufecki puts, it,
Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks.
The reception of this review has been fascinating, though extremely depressing. The lead author, Tom Jefferson, gave an interview informally summarizing the conclusion as,
There is just no evidence that [masks] make any difference.. That's not a totally ridiculous single sentence summary of the conclusions of the meta-analysis, but it's in the context of a very inflammatory article. Some mainstream media outlets ran the story as if it were, new study proves masks make ZERO difference. (Not linking to any of those directly, I'm sure you'll have seen them around. But deeply misleading by confusing a study with a meta-analysis of previous studies, and assuming that a finding of no evidence counted as "proof" of no effect.)
Very quickly, all of public-facing social media was absolutely full of highly networked, bot-like accounts posting a link to the study on every thread that referenced Covid precautions, saying, LOL, masks don't work! (I don't think all the comments to this effect were posted by literal automated bots, but by the kind of mix of human and automated postings that swarm Twitter repeating the same message when it's convenient to someone influential.) There were even a few anonymous posts on my quiet backwater little DW with a similar message, the Cochrane review shows masks don't work, you can stop bothering. Some people I actually know also mentioned that the review had convinced them to give up on masks.
There were also a bunch of debunkings which seemed very knee-jerk, again, particularly on Twitter but all over the place. A much retweeted thread which actively lied about the studies included in the meta-analysis and all but libelled the authors. I read a lot of stuff. I dismissed very extreme refutations which amounted to little more than, well, everybody knows masks work so if this meta-analysis showed they don't it must be biased and wrong. That's not very scientific, that's just confirming your pre-existing views. I skimmed the review itself but didn't read all the studies cited in it, but gave myself enough of a sense of when people referring to it were reporting broadly true facts about its findings, rather than making up nonsense to support either the pro- or anti-mask cause.
[Sidebar: This, by the way, is an example of why I disagree with
siderea that I think you should trust me (within limits) because I cite my sources.. Lots of not at all trustworthy people cite sources all the time. They post lists and lists of links to highly technical academic articles, and make completely untrue claims about what those articles show, and hope that you won't have time or the right skills to go back to the cited source and notice for yourself that the summary is a distortion, or even the opposite, of what the purported citation shows. Like, there's literally a playbook circulating in the nasty bits of the internet with a bibliography of peer reviewed journal articles that purport to show that intelligence is genetic and correlated with race. Transphobes regularly cite "studies" which apparently show that trans people are a danger to normal children and cis women, when the studies either have been debunked or show no such thing. Even Wikipedia, with its
citation neededculture, can easily be used for disinformation because a pile of little blue numbers make false articles look authoritative, and people believe the air of academic-ness without noticing that the citations don't support what is claimed on the page. So anyway, lots of people "cite" the Cochrane review or the studies within it, but are wrong, intentionally or accidentally, about what such sources say.]
The conclusion I tentatively reached was that there are, essentially, two well-powered, high quality RCTs of mask wearing for prevention of actual Covid. The Bangladesh study which found a modest but consistent positive effect, and the DANMASK study which found little to no effect (and like the review that relies on it, was widely cited as definite proof for the anti-mask case when it came out). You can't do a meta-analysis by combining only two studies, so the Cochrane team had a choice: they could include studies that aren't RCTs, or they could include studies that are not on Covid. They chose the latter, which is not a completely unreasonable decision and I don't think I'm better able to judge than actual experts in medical evidence which way they should have gone. Also, it was an update of a pre-Covid review about respiratory illness in general, so it makes sense to include studies from before the emergence of Covid.
But to me this meant that the meta-analysis added little to what I already knew, that the Bangladesh study found masks to be somewhat effective in a real-world setting, and the Danish study found that in a situation where case levels are already extremely low, they weren't able to measure much difference between almost no cases in the control group and almost no cases in the intervention group. I did a lot of reflecting about my own attitudes; I certainly trust Cochrane reviews on the whole, and I certainly want to be the sort of person who updates my opinions based on new evidence. I was pretty sure I had a bias towards looking for flaws in the Cochrane review, and I tried to compensate for that.
For some people, "masks don't work" is good news, because it means they can push back against anyone asking them to wear a mask when they don't want to. But I don't feel like that at all; I would be very unhappy to learn that masks don't work, because it wouldn't mean time-travelling back to 2019 as I'm sure many of us would love to be able to do, rather it would mean that the main precaution I'm taking to be able to do any in-person activities at all is essentially worthless. Obviously my unhappiness is completely the opposite of evidence about what is actually the reality out there. So I wanted to be very careful not to over-weight evidence that aligns with what I want to be true.
I had some amount of trust in this rebuttal by Macintyre et al. It contains a great deal of non-RCT, but compelling, evidence in favour of masks, and some convincing arguments why RCTs based purely on outcomes for the mask-wearer may not be the only good quality evidence for this question. It also has what looks to me like a valid, not hysterical, critique of the Cochrane review, namely that they are comparing a lot of different things that aren't really comparable, and also that many of the studies which came out not showing much effect were studies of whether telling people to wear masks is effective and didn't take into account compliance or lack thereof. I generally trust the last author, Trish Greenhalgh, but she has been passionately pro-mask throughout and indeed it's reading her research that convinced me wearing masks was worthwhile even when the official advice was against it. Macintyre herself declares a conflict of interest in that she works for mask manufacturers, which is a pretty glaring source of potential bias!
My conclusion until today was: telling people to wear masks in the community, regardless of Covid levels, and regardless of mask quality, doesn't strongly predict better outcomes (less Covid, in this case). But there is pretty good evidence that well-fitted masks worn by most people do actually provide some protection when Covid levels are high, even if there isn't much in the way of well-powered RCTs showing benefit. I held that conclusion pretty lightly, because I knew I was biased and because I hadn't actually read all the original studies in detail, mostly summaries.
The latest news is that the editor-in-chief of the Cochrane library has semi-retracted the notorious review. As reported by Zeynep Tufekci in the New York Times, Soares-Weiser has issued a statement that
Many commentators have claimed that a recently updated Cochrane Review shows that ‘masks don’t work’, which is an inaccurate and misleading interpretation ... [our] wording was open to misinterpretation, for which we apologize. (I am aware that some people are boycotting the NYT due to its horrible transphobia; right now they are the outlet breaking this news, so feel free to wait until other press wires pick it up if you don't want to give them clicks. Also, I bypassed the paywall by turning off JavaScript; up to you whether you're comfortable with doing that.)
The remainder of the article summarizes some of the (non RCT) evidence in favour of masks, and notes that the Cochrane review lead author has a history of anti-mask views and even pandemic minimizing. I have a lot of time for Tufekci, but some people don't like her because she doesn't come from a scientific or medical background, and she tends to take into account evidence from a range of sources and in particular doesn't accept some of the most extreme Covid catastrophizing. IMO she's been pretty consistently right, supporting masks and opposing restrictions on outdoor socializing from very early on, campaigning for Long Covid sufferers but rejecting absurdly high projections of levels of severe long-term organ and immune damage. In my view the important thing is the statement by Soares-Weiser where she is very clear that the Cochrane review has been misinterpreted as being definitively anti-mask.
So there you go, that takes us more or less back to the status quo ante, before the January update of the Cochrane Review. We have some evidence in favour of some types of masks in some situations, but we can't definitively say 'masks work' or 'masks don't work'. I think that may be some of the reason why some of the mask debate has become so painful and polarized; a lot of people don't feel comfortable with the idea of a mitigation that has some effect but isn't perfect, they want interventions to either "work" or "not work". Same as the vaccination debate. Vaccines, like masks, give some protection, but they're not, as Tufecki puts, it,
a talisman or a magic wand.
(no subject)
Date: 2023-03-15 05:13 pm (UTC)I want everyone to be able to avoid Covid, including young children, including people with customer-facing jobs and bosses who aren't willing to accommodate them, including people who can't stand having something pressing on their face. You might in fact be better than 99% of the population at taking the right precautions – that's a really good thing, and I admire you for your ability to assess the evidence and implement an effective solution, but protecting the best 1% of the population isn't public health, it doesn't do much to stop Covid.
(no subject)
Date: 2023-03-15 07:37 pm (UTC)Oh, absolutely. I would like all the people I know who wear masks sometimes, and the people I know who never wear masks (regardless of the reason), to have the benefit of better ventilation standards (starting with schools and hospitals), better sick pay policies, better access to working remotely where that's feasible, free testing, proper contact tracing (insofar as that can be done without infringing civil liberties -- I think it likely can be, but I don't think it's trivial). All of those things substantially impact the numbers game too. Hopefully, in time, better vaccines would also change the numbers, and better treatments would make those numbers matter less because the risks would be lower (and these are... less scary than they were in, say, March 2020, but still Not Great).
In the meantime... I know not everyone can take the precautions I do, but that makes it feel more important, not less, that I do it, because then it's less likely that I'll pass covid (or influenza or whatever) on to someone who can't take my precautions. And then I really get annoyed by the "well, not everyone can avoid catching covid, and most people recover, so nobody should bother to try" narrative that seems to be assumed in a lot of the "this single intervention isn't perfect" media coverage. I don't think that's where you were going at all, though, and I think we largely agree that better public health interventions would be desirable and effective, and that individual mitigations in the meantime are imperfect but still often worthwhile (subject to individual context), and that ideally individuals wouldn't be put in the position of having to decide whether to wear a mask for activities where it's clearly easier not to but the infection risk is high or unknown, or having to research which masks work well for them, or having to learn how to wear them properly. It's pretty absurd that any of us are in this position at all.
I'm also still really angry at being told for most of my life that handwashing and not touching my face and not getting sneezed or coughed on directly were The Important Things To Focus On To Not Get Sick, only to find that I haven't had so much as a cold since February 2020 because I stopped exposing myself to aerosol transmission of viruses. Don't get me wrong, I do still wash my hands, but... not all that much more than I did pre-pandemic, I think. And I'm angry that as a society we appear to be turning around and ignoring that; I'm angry that I asked a choir I would love to sing in about CO2 monitoring and didn't even get a response; I'm angry that we can't afford to do the kind of air filtration we would need to at church to keep the space safer when the soup kitchen is operating, there are no grants available for improving it, and we don't want to just open the doors because we're also advertising it as a "warm space" for people who don't have any heating at home (...and yeah, if the energy bills were lower, maybe we could afford a consultation and the equipment to do the filtration, but... that's not where we are, and I don't exactly think artificially lowering energy bills is a sustainable approach either given the climate situation. During the times when there were restrictions on gathering we offered an outdoor-only service, but it's not necessarily better to make people sit in the cold). So I get a bit spiky in my replies sometimes, because all of that feels intractable and impossible, even though I do see signs of improvement (like this news about air quality regulations in Belgium and elsewhere: https://www.nature.com/articles/d41586-023-00642-9 and something similar I read a month or two ago regarding air in schools in France -- though if the government has actually rolled out CO2 monitors to every UK classroom, as that article claims it "started to" in August 2021, it's news to me). I'm sorry if I am coming across as unreasonably cantankerous as a result.