Science Translation During the COVID-19 Pandemic: An Academic-Public Health Partnership to Assess Capacity Limits in California
Abstract
On the basis of an extensive academic–public health partnership around COVID-19 response, we illustrate the challenge of science-policy translation by examining one of the most common nonpharmaceutical interventions: capacity limits. We study the implementation of a 20% capacity limit in retail facilities in the California Bay Area.
Through a difference-in-differences analysis, we show that the intervention caused no material reduction in visits, using the same large-scale mobile device data on human movements (mobility data) originally used in the academic literature to support such limits. We show that the lack of effectiveness stems from a mismatch between the academic metric of capacity relative to peak visits and the policy metric of capacity relative to building code.
The disconnect in metrics is amplified by mobility data losing predictive power after the early months of the pandemic, weakening the policy relevance of mobility-based interventions. Nonetheless, the data suggest that a better-grounded rationale for capacity limits is to reduce risk specifically during peak hours. To enhance the connection between science, policy, and public health in future times of crisis, we spell out 3 strategies: living models, coproduction, and shared metrics. (Am J Public Health. 2022;112(2):308–315. https://doi-org.laneproxy.stanford.edu/10.2105/AJPH.2021.306576)
Public health responses to COVID-19 have faced serious challenges in light of rapid changes in the scientific understanding of both the virus and the effectiveness of policy responses. This article discusses lessons from an academic–public health partnership around COVID-19 response. We present findings based on a collaboration with the Public Health Department of Santa Clara County, California, one of the largest counties in the United States. In conjunction with 5 other Bay Area counties, Santa Clara was the first jurisdiction in the country to issue a shelter-in-place order in response to COVID-19.1 We illustrate challenges that can arise for evidence-based policy during times of crisis using a case study of a prominent nonpharmaceutical intervention—namely, the implementation of capacity limits on businesses (i.e., restricting businesses to some percent of capacity).
A main contribution of our work is to identify 3 tangible strategies for mutually enhancing science, policy, and public health, based on this partnership. We illustrate the gains to such a model in studying the implementation of a 20% capacity limit starting December 6, 2020, on the main affected sectors—namely, grocery stores, pharmacies, and general merchandise stores. (Indoor restaurant dining was already prohibited at this time.) Using data on human movements (mobility data) from mobile devices in a difference-in-differences framework,2,3 we show that the 20% capacity limit had no significant impact on decreasing the number of visits or peak hour visits, or the length of visits to businesses in those sectors compared with prepandemic time periods. These are the same measures and data employed in the scientific literature to support capacity limits. The puzzle then is how to reconcile the existing scientific literature, which appears to support such limits, with an intervention that proved ineffective in practice.
To resolve this puzzle, we show that capacity limits were ineffective because of disparate definitions of maximum occupancy adopted by researchers as opposed to policymakers. Although scientists used measures available in retrospective data (e.g., 20% of peak capacity reported after a week from mobility data), policymakers require definitions that can be implemented and enforced on the ground in real time. The result was a limit that did not bind: most businesses were already below the enforced limit at baseline.
This disconnect highlights how profoundly human behavior had already shifted prior to the implementation of the capacity limit. Consistent with other evidence,4,5 we show that mobility loses predictive power of case spread as public health orders are put into place. Scientific studies that anchored capacity limits in associations between human mobility and COVID-19 case rates from the first few months of the pandemic may therefore lose their policy relevance over time.
The effort to reduce the spread of COVID-19 through capacity limits holds valuable lessons for future policy responses to crises. Through our collaboration with public-sector partners, we identified 3 specific strategies for improvement: ensuring that models used to inform policy are dynamic (living) rather than static, improving collaboration between scientists and policymakers through coproduction (not merely science translation), and shifting to more targeted and enforceable metrics in science.
This article assesses the impact of capacity limits and explains how the limits were mistranslated from academic literature, and then reflects on broader lessons for academic–public health collaborations to improve crisis response.