-
Doctors and Nurses Social Media Ads Reduced Holiday Travel and COVID-19 infections: A cluster randomized controlled trial in 13 States
Authors:
Emily Breza,
Fatima Cody Stanford,
Marcela Alsan,
M. D. Ph. D.,
Burak Alsan,
Abhijit Banerjee,
Arun G. Chandrasekhar,
Sarah Eichmeyer,
Traci Glushko,
Paul Goldsmith-Pinkham,
Kelly Holland,
Emily Hoppe,
Mohit Karnani,
Sarah Liegl,
Tristan Loisel,
Lucy Ogbu-Nwobodo,
Benjamin A. Olken Carlos Torres,
Pierre-Luc Vautrey,
Erica Warner,
Susan Wootton,
Esther Duflo
Abstract:
During the COVID-19 epidemic, many health professionals started using mass communication on social media to relay critical information and persuade individuals to adopt preventative health behaviors. Our group of clinicians and nurses developed and recorded short video messages to encourage viewers to stay home for the Thanksgiving and Christmas Holidays. We then conducted a two-stage clustered ra…
▽ More
During the COVID-19 epidemic, many health professionals started using mass communication on social media to relay critical information and persuade individuals to adopt preventative health behaviors. Our group of clinicians and nurses developed and recorded short video messages to encourage viewers to stay home for the Thanksgiving and Christmas Holidays. We then conducted a two-stage clustered randomized controlled trial in 820 counties (covering 13 States) in the United States of a large-scale Facebook ad campaign disseminating these messages. In the first level of randomization, we randomly divided the counties into two groups: high intensity and low intensity. In the second level, we randomly assigned zip codes to either treatment or control such that 75% of zip codes in high intensity counties received the treatment, while 25% of zip codes in low intensity counties received the treatment. In each treated zip code, we sent the ad to as many Facebook subscribers as possible (11,954,109 users received at least one ad at Thanksgiving and 23,302,290 users received at least one ad at Christmas). The first primary outcome was aggregate holiday travel, measured using mobile phone location data, available at the county level: we find that average distance travelled in high-intensity counties decreased by -0.993 percentage points (95% CI -1.616, -0.371, p-value 0.002) the three days before each holiday. The second primary outcome was COVID-19 infection at the zip-code level: COVID-19 infections recorded in the two-week period starting five days post-holiday declined by 3.5 percent (adjusted 95% CI [-6.2 percent, -0.7 percent], p-value 0.013) in intervention zip codes compared to control zip codes.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
-
Selecting the Most Effective Nudge: Evidence from a Large-Scale Experiment on Immunization
Authors:
Abhijit Banerjee,
Arun G. Chandrasekhar,
Suresh Dalpath,
Esther Duflo,
John Floretta,
Matthew O. Jackson,
Harini Kannan,
Francine Loza,
Anirudh Sankar,
Anna Schrimpf,
Maheshwor Shrestha
Abstract:
Policymakers often choose a policy bundle that is a combination of different interventions in different dosages. We develop a new technique -- treatment variant aggregation (TVA) -- to select a policy from a large factorial design. TVA pools together policy variants that are not meaningfully different and prunes those deemed ineffective. This allows us to restrict attention to aggregated policy va…
▽ More
Policymakers often choose a policy bundle that is a combination of different interventions in different dosages. We develop a new technique -- treatment variant aggregation (TVA) -- to select a policy from a large factorial design. TVA pools together policy variants that are not meaningfully different and prunes those deemed ineffective. This allows us to restrict attention to aggregated policy variants, consistently estimate their effects on the outcome, and estimate the best policy effect adjusting for the winner's curse. We apply TVA to a large randomized controlled trial that tests interventions to stimulate demand for immunization in Haryana, India. The policies under consideration include reminders, incentives, and local ambassadors for community mobilization. Cross-randomizing these interventions, with different dosages or types of each intervention, yields 75 combinations. The policy with the largest impact (which combines incentives, ambassadors who are information hubs, and reminders) increases the number of immunizations by 44% relative to the status quo. The most cost-effective policy (information hubs, ambassadors, and SMS reminders but no incentives) increases the number of immunizations per dollar by 9.1% relative to status quo.
△ Less
Submitted 12 September, 2022; v1 submitted 19 April, 2021;
originally announced April 2021.
-
Fisher-Schultz Lecture: Generic Machine Learning Inference on Heterogenous Treatment Effects in Randomized Experiments, with an Application to Immunization in India
Authors:
Victor Chernozhukov,
Mert Demirer,
Esther Duflo,
Iván Fernández-Val
Abstract:
We propose strategies to estimate and make inference on key features of heterogeneous effects in randomized experiments. These key features include best linear predictors of the effects using machine learning proxies, average effects sorted by impact groups, and average characteristics of most and least impacted units. The approach is valid in high dimensional settings, where the effects are proxi…
▽ More
We propose strategies to estimate and make inference on key features of heterogeneous effects in randomized experiments. These key features include best linear predictors of the effects using machine learning proxies, average effects sorted by impact groups, and average characteristics of most and least impacted units. The approach is valid in high dimensional settings, where the effects are proxied (but not necessarily consistently estimated) by predictive and causal machine learning methods. We post-process these proxies into estimates of the key features. Our approach is generic, it can be used in conjunction with penalized methods, neural networks, random forests, boosted trees, and ensemble methods, both predictive and causal. Estimation and inference are based on repeated data splitting to avoid overfitting and achieve validity. We use quantile aggregation of the results across many potential splits, in particular taking medians of p-values and medians and other quantiles of confidence intervals. We show that quantile aggregation lowers estimation risks over a single split procedure, and establish its principal inferential properties. Finally, our analysis reveals ways to build provably better machine learning proxies through causal learning: we can use the objective functions that we develop to construct the best linear predictors of the effects, to obtain better machine learning proxies in the initial step. We illustrate the use of both inferential tools and causal learners with a randomized field experiment that evaluates a combination of nudges to stimulate demand for immunization in India.
△ Less
Submitted 23 October, 2023; v1 submitted 13 December, 2017;
originally announced December 2017.
-
Double/Debiased/Neyman Machine Learning of Treatment Effects
Authors:
Victor Chernozhukov,
Denis Chetverikov,
Mert Demirer,
Esther Duflo,
Christian Hansen,
Whitney Newey
Abstract:
Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016) provide a generic double/de-biased machine learning (DML) approach for obtaining valid inferential statements about focal parameters, using Neyman-orthogonal scores and cross-fitting, in settings where nuisance parameters are estimated using a new generation of nonparametric fitting methods for high-dimensional data, called machin…
▽ More
Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016) provide a generic double/de-biased machine learning (DML) approach for obtaining valid inferential statements about focal parameters, using Neyman-orthogonal scores and cross-fitting, in settings where nuisance parameters are estimated using a new generation of nonparametric fitting methods for high-dimensional data, called machine learning methods. In this note, we illustrate the application of this method in the context of estimating average treatment effects (ATE) and average treatment effects on the treated (ATTE) using observational data. A more general discussion and references to the existing literature are available in Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, and Newey (2016).
△ Less
Submitted 30 January, 2017;
originally announced January 2017.
-
Double/Debiased Machine Learning for Treatment and Causal Parameters
Authors:
Victor Chernozhukov,
Denis Chetverikov,
Mert Demirer,
Esther Duflo,
Christian Hansen,
Whitney Newey,
James Robins
Abstract:
Most modern supervised statistical/machine learning (ML) methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal parameters. Examples of such parameters include individual regression coefficients, average treatment effects, average lifts, and demand or supply elasticities. In fact,…
▽ More
Most modern supervised statistical/machine learning (ML) methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal parameters. Examples of such parameters include individual regression coefficients, average treatment effects, average lifts, and demand or supply elasticities. In fact, estimates of such causal parameters obtained via naively plugging ML estimators into estimating equations for such parameters can behave very poorly due to the regularization bias. Fortunately, this regularization bias can be removed by solving auxiliary prediction problems via ML tools. Specifically, we can form an orthogonal score for the target low-dimensional parameter by combining auxiliary and main ML predictions. The score is then used to build a de-biased estimator of the target parameter which typically will converge at the fastest possible 1/root(n) rate and be approximately unbiased and normal, and from which valid confidence intervals for these parameters of interest may be constructed. The resulting method thus could be called a "double ML" method because it relies on estimating primary and auxiliary predictive models. In order to avoid overfitting, our construction also makes use of the K-fold sample splitting, which we call cross-fitting. This allows us to use a very broad set of ML predictive methods in solving the auxiliary and main prediction problems, such as random forest, lasso, ridge, deep neural nets, boosted trees, as well as various hybrids and aggregators of these methods.
△ Less
Submitted 12 December, 2017; v1 submitted 29 July, 2016;
originally announced August 2016.
-
Using Gossips to Spread Information: Theory and Evidence from a Randomized Controlled Trial
Authors:
Abhijit Banerjee,
Arun G. Chandrasekhar,
Esther Duflo,
Matthew O. Jackson
Abstract:
Is it possible to identify individuals who are highly central in a community without gathering any network information, simply by asking a few people? If we use people's nominees as seeds for a diffusion process, will it be successful? We explore these questions theoretically, via surveys, and via field experiments. We show via a model of information flow how members of a community can, just by tr…
▽ More
Is it possible to identify individuals who are highly central in a community without gathering any network information, simply by asking a few people? If we use people's nominees as seeds for a diffusion process, will it be successful? We explore these questions theoretically, via surveys, and via field experiments. We show via a model of information flow how members of a community can, just by tracking gossip about others, identify highly central individuals in their network. Asking villagers in rural Indian villages to name good seeds for diffusion, we find that they accurately nominate those who are central according to a measure tailored for diffusion - not just those with many friends or in powerful positions. Finally, we run a randomized field experiment in 213 other villages that tests how effective it is to use such nominations as seeds for a diffusion process. Relative to random seeds or those with high social status, hitting at least one seed nominated by villagers leads to more than a 65% increase in the spread of information.
△ Less
Submitted 8 May, 2017; v1 submitted 9 June, 2014;
originally announced June 2014.
