On the importance of being careful with meta-analyses on the effectiveness of NPIs
Adrian Lison and colleagues systematically reviewed studies that assess the effectiveness of non-pharmaceutical interventions. Their comment about the limited comparability of current studies was recently featured on a well-known blog for statistical modeling.
Throughout the COVID-19 pandemic, different non-pharmaceutical interventions have been implemented to reduce the spread of SARS-CoV-2. Consequently, numerous studies have attempted to empirically assess their effectiveness by using epidemiological data. Over the last 1.5 years, Adrian and his colleagues conducted a systematic literature review of over 240 such studies. They concluded that current studies are not standardized in their methodologies and reporting practices, making it difficult to compare results. Nevertheless, several meta-analysis on the topic have been published, with some drawing strong conclusions.
Recently, Adrian and his colleagues expressed their concerns with one such meta-analysis in the form of a scientific comment, describing the general difficulties of such analyses. Their comment was shared on social media and was recently featured in a blog post by Andrew Gelman's on the "Statistical Modeling, Causal Inference, and Social Science" blog.
You can read their systematic literature review external page here, their scientific comment external page here and the corresponding blog post external page here.