Methods
Data sources
Briefly, the instantaneous reproduction number is estimated based on the daily counts of confirmed COVID-19 cases reported by the European Centre for Disease Prevention and Control. The instantaneous reproduction number represents the average number of secondary cases that would arise from a primary case infected at a given time if the conditions remained identical after that time, and thus measures the instantaneous transmissibility.
The modelling framework accounts for reporting delay between symptom onset and case notification, right truncation of notification dates, and the delay between onset and infection based on empirical data to ensure that temporal variations in R can be compared directly with the times at which NPIs were implemented.
OxCGRT was established by a dedicated team of public policy and governance experts, who collect publicly available information on indicators of government response. In OxCGRT, NPIs are grouped into the following eight categories: closure of schools, closure of workplaces, public events bans (eg, sports, festive, and religious events), restrictions on the size of gatherings, closure of public transport, stay at home orders, restrictions on internal movement, and restrictions on international travel. Country-specific information on each of the NPIs is available on a daily basis (since Jan 1, 2020). We also included data on testing policy and contact tracing of each country from OxCGRT for sensitivity analyses.
Data processing
Data analysis
Figure 1Schematic presenting calculation of the R ratio
Day i is defined as the ith day of the phase (ie, since NPI status changed). Day N represents the last day of the phase. Note that different phases could have different numbers of days. NPI=non-pharmaceutical intervention. R=time-varying reproduction number.
In the main analysis, we modelled the R ratio using a log-linear regression, with the following equation, for each day of the first 28 days following the change in the corresponding NPI (ie, a total of 28 separate models):
where Yt represents the R ratio on day t (t=1, 2, …, 28);
are binary indicators of whether each of the eight NPIs are introduced and lifted, respectively; and
are binary indicators of whether multiple NPIs are introduced and lifted simultaneously, respectively. Hence,
represents the baseline change in R on day t in the absence of changes in NPI status;
represent the individual effects of introducing and lifting NPIs on day t, respectively; and
represent the interaction between introducing and lifting, respectively, multiple NPIs as they are introduced and lifted simultaneously. No days beyond the first 28 days following the change were included due to limited data availability.
We did a series of sensitivity analyses. First, we replaced the NPI of a ban on gatherings of more than ten people with a ban on gatherings of more than 100 people in the model to understand how limiting public gatherings of different sizes could affect the transmission. Second, we presented the effect of individual NPIs by only including phases in which just one NPI was changed. Third, we used a different comparator, the mean R for the 7 days before NPI status change (rather than R for the day before NPI status change), when calculating the R ratio. Fourth, we excluded early phases in which the country’s first NPI was introduced. Fifth, we excluded large countries that could have greater regional variability in NPI policies: Brazil, Canada, China, India, Russia, and the USA. Sixth, we did 20 sets of analyses, each of which randomly excluded ten countries from the dataset, to understand how our estimates had been affected by possible outliers. Seventh, we included only the phases with comprehensive testing (defined as the requirement to test anyone with COVID-19 symptoms) in the analysis, since testing practice could affect the estimate of R. Eighth, we included only the phases with comprehensive contact tracing (defined as the requirement to trace contacts for all COVID-19 cases) to understand how contact tracing could modify the effect of NPIs in our model.
In addition, based on the modelled effect of individual NPIs from our main analysis, we did an ad-hoc analysis to estimate the effect of reintroducing multiple NPIs (those with the greatest effects and following the observed sequence of introducing NPIs) to tackle the possible resurgence of SARS-CoV-2. We considered four candidate strategies for the reintroduction: candidate 1 included a ban on public events and gatherings of more than ten people; candidate 2 included workplace closure as well as a ban on public events and gatherings of more than ten people; candidate 3 included workplace closure, a ban on public events and gatherings of more than ten people, and internal movement limits; and candidate 4 included school and workplace closure, a ban on public events and gatherings of more than ten people, internal movement limits, and requirements to stay at home.
Role of the funding source
The funders of the study had no role in study design, data collection, data analysis, data interpretation, writing of the manuscript, or the decision to submit for publication. All authors had full access to all the data in the study and were responsible for the decision to submit the manuscript for publication.
Results
Figure 2Frequency (A) and order (B) of introducing and lifting NPIs
(A) Each number denotes the frequency of the co-occurrence of NPIs in the x and y axes. Numbers on the diagonal (from bottom left to top right) denote the frequency of the occurrence of NPIs (with and without co-occurrence). (B) Each number in the graph denotes the percentage of NPI in the y axis that occurred earlier than the NPI in the x axis among countries with both NPIs ordered or lifted. NPIs are ranked from earliest to latest based on the mean percentage of the row. NPI=non-pharmaceutical intervention.
Figure 3Change over time in the R ratio following the introduction and relaxation of individual NPIs
For each NPI, the reference period is the day before introduction or relaxation of that NPI. An R ratio of more than 1 indicates increased transmission, and an R ratio of less than 1 indicates decreased transmission. The error bars present the 95% CIs of the R ratios derived from the model. NPI=non-pharmaceutical intervention. R=time-varying reproduction number.
Table 1Change in the R ratio over time on day 7, day 14, and day 28 after the introduction and relaxation of each NPI
Data are R ratio (95% CI). For each NPI, the reference period is the day before introduction or relaxation of that NPI. An R ratio of more than 1 indicates increased transmission, and an R ratio of less than 1 indicates decreased transmission. NPI=non-pharmaceutical intervention. R=time-varying reproduction number.
Figure 4Change over time in the R ratio following the introduction and relaxation of a ban on public gatherings of different sizes
The error bars present the 95% CIs of the R ratios derived from the model. R=time-varying reproduction number.
Table 2Modelled change in the R ratio over time on day 7, day 14, and day 28 after the introduction of different composites of NPIs
Data are R ratio (95% CI). The reference period is the day before introduction of an NPI. An R ratio of more than 1 indicates increased transmission, and an R ratio of less than 1 indicates decreased transmission. NPI=non-pharmaceutical intervention. R=time-varying reproduction number.
Discussion
To the best of our knowledge, this study is the first to assess the temporal association between changing the status of a range of NPIs and the transmission of SARS-CoV-2, as measured by R, for all countries for which data were available. On the basis of the empirical data from 131 countries, we found that individual NPIs, including school closure, workplace closure, public events bans, requirements to stay at home, and internal movement limits, were associated with reductions in R of 3–24% on day 28 after their introduction, compared with the day before their introduction. Reopening schools, lifting bans on public events, lifting bans on public gatherings of more than ten people, lifting requirements to stay at home, and lifting internal movement limits were associated with increases in R of 11–25% on day 28 after the relaxation. The effects of introducing and lifting NPIs were not immediate; it took around 1 week following the introduction of an NPI to observe 60% of the maximum reduction in R and even longer (almost 3 weeks) following the relaxation of an NPI to observe 60% of the maximum increase in R. Our analysis suggests that, in the context of a resurgence of SARS-CoV-2, a control strategy of banning public events and public gatherings of more than ten people would be associated with a reduction in R of 6% on day 7, 13% on day 14, and 29% on day 28; if this strategy also included closing workplaces, the overall reduction in R would be 16% on day 7, 22% on day 14, and 38% on day 28. These findings provide additional evidence that can inform policy makers’ decisions on the timing of introducing and lifting different NPIs.
Flaxman and colleagues reported that several NPIs (eg, school closure and public events ban), and lockdown in particular, had a large effect (81%) on reducing transmission.
However, Flaxman and colleagues did not assess changes over time in the effect of lockdown and assumed that the effect was immediate. In this study, we estimated that an extreme intervention similar to lockdown, consisting of school and workplace closure, bans on public events and gatherings, requirements to stay at home, and limits on internal movement, could reduce R by 35% on day 7, 42% on day 14, and 52% on day 28. Our findings on the effects of introducing NPIs were also qualitatively similar to those from a study by Islam and colleagues that modelled the incidence rate ratio of COVID-19 with OxCGRT NPI data,
although that study did not assess the effects of lifting NPIs.
Our analysis demonstrates that the effect of introducing and lifting NPIs was not immediate and that the time required to reach certain levels of effect differed by NPI. This finding provides important evidence to policy makers on when to expect a notable effect from introducing or lifting an NPI. The observed delay of effect could be explained by behavioural inertia, which is supported by the similar immediacy results of NPIs between using R and using Google mobility data.
,
For SARS-CoV-2, the role of children in its transmission is still unclear. A modelling study from China showed that school closure alone could not interrupt transmission, but it could potentially reduce peak incidence by 40–60% and delay the epidemic of COVID-19.
In this study, we showed that closing schools alone could decrease transmission by 15% (R ratio 0·85, 95% CI 0·66–1·10) on day 28 and reopening schools could increase transmission by 24% (1·24, 1·00–1·52) on day 28. It should be acknowledged that in our analysis, we were unable to account for different precautions regarding school reopening that were adopted by some countries, such as physical distancing within classrooms (eg, limiting class sizes and placing transparent dividers between students) and outside classrooms (eg, physical distancing during meal times, recreation, and transportation), enhanced hygiene (eg, routine deep cleaning and personal handwashing and face masks), and others (eg, thermal temperature checks on arrival).
,
Such precautions are imperative for safer school reopening. A COVID-19 outbreak was reported in a high school in Israel 10 days after its reopening; students were in crowded classrooms and were not instructed to wear face masks due to high temperatures.
In addition, it should be noted that we did not consider the normal school holidays in some countries. We were also unable to assess the effect of reopening different levels of school (eg, elementary vs middle schools) since the effect might differ by finer age bands within school-age children and adolescents.
,
A report found that children younger than 5 years with mild to moderate COVID-19 had high viral loads in their nasopharynx compared with older children and adults, and thus could potentially be important drivers of transmission in the general population.
Another explanation for the high reduction is that a ban on public events was often the first NPI to be introduced in countries; our sensitivity analysis that excluded NPIs that were introduced first showed a non-significant reduction of transmission with banning public events, with an R ratio of 0·80 (95% CI 0·57–1·11) on day 28.
Our findings also suggest that, within 28 days, lifting public events bans could increase transmission by 21%, although the finding was not significant, and lifting bans on gatherings of more than ten people could increase transmission by 25%, which was the highest increase among all NPIs. We did not observe a substantial reduction in transmission after introduction of bans on gatherings of more than ten or more than 100 people, especially for more than 100 people, which showed an increase in transmission after day 14; possible explanations for this finding include low adherence and, for the ban on gatherings of more than 100 people, an increase in smaller-scale gatherings. In addition, it should be noted that for bans on physical gatherings, we were unable to further stratify our analysis by indoor versus outdoor settings due to scarcity of data.
However, our data lacked the necessary granularity to further explore timeliness of testing and tracing. Additionally, similar to the findings by Islam and colleagues,
we did not observe substantial effects of public transport closure on the R ratio.
There are some advantages to our study. First, both the method for the R calculation and the method for recording NPIs remained consistent over time among different countries, which ensured comparability between different phases in different countries in our analysis. Second, by dividing the timeline into different phases according to the changes in NPIs, we were able to assess the effect of individual NPIs. Third, we were able to estimate the change in the effect of NPIs over time.
Seventh, we only assessed the effect of introducing and lifting NPIs for the first 28 days after introduction and relaxation, and the findings (including the trend) should not be generalised to beyond 28 days. Finally, although our study could essentially be regarded as a natural experiment study,
our findings do not necessarily imply causation.
We acknowledge several limitations of the methodology for the R estimate used in our analysis. First, the adjustment for reporting delays was only done globally and not specific to each country due to the scarcity of available data on reporting delays. This could lead to temporal inaccuracy of R, which could bias our findings on the immediacy of changes in R associated with NPIs. Nonetheless, our findings on the immediacy of changes associated with NPIs were consistent with the results of the analysis using Google mobility data, indicating that the possible temporal inaccuracy of R might have had little impact on the overall findings. Second, the R estimate was subject to the specification of parameters (eg, incubation period and generation time of SARS-CoV-2) in the model and could be biased upwards or downwards. However, we believe it unlikely that this bias affected the main findings of our study because we used the R ratio as the output metric (which cancels out all time-invariant elements related to the R estimate). Third, the modelling framework for R was unable to account for the change over time in eligibility for testing, method of testing, or case definition in different countries. This could bias both the R estimate and the R ratio in our analysis for the dates during which the changes were ongoing. For example, we are likely to observe an artificial increase in R if a country increases the testing capacity within a short period. Last, the uncertainty range of the national R estimate was based on the number of national reported cases and therefore did not reflect any variations in R within the country.
,
In summary, our findings provide additional evidence that can inform policy makers’ decisions on the timing of introducing and lifting different NPIs. The decisions to reintroduce and relax restrictions should be informed by various factors, including the capacity and resilience of the health-care system, and might be best made at provincial or district rather than national levels in some countries.
YL, HC, and HN conceptualised the study. YL led data acquisition, analysis, and visualisation. HN, HC, and YL led the data interpretation with substantial contribution from DK, AH, MN, and XW. YL wrote the draft report, and all other authors revised the report critically for important intellectual content. All authors have read and approved the final version of the report. YL and HN verified the data linkage of two publicly available datasets and had full access to the linked data.
YL reports grants from WHO, outside the submitted work. HC reports grants from the Innovative Medicines Initiative, UK National Institute for Health Research, and Bill & Melinda Gates Foundation, and grants and personal fees from WHO and Sanofi, outside the submitted work. HN reports grants from the Innovative Medicines Initiative, WHO, and the National Institute for Health Research; personal fees from the Bill & Melinda Gates Foundation, Janssen, and AbbVie; and grants and personal fees from Sanofi and the Foundation for Influenza Epidemiology, outside the submitted work. All other authors declare no competing interests.
