Showing 1–28 of 28 results for author: Zubizarreta, J R

Design-based inference for generalized network experiments with stochastic interventions

Authors: Ambarish Chattopadhyay, Kosuke Imai, Jose R. Zubizarreta

Abstract: A growing number of scholars and data scientists are conducting randomized experiments to analyze causal relationships in network settings where units influence one another. A dominant methodology for analyzing these network experiments has been design-based, leveraging randomization of treatment assignment as the basis for inference. In this paper, we generalize this design-based approach so that… ▽ More A growing number of scholars and data scientists are conducting randomized experiments to analyze causal relationships in network settings where units influence one another. A dominant methodology for analyzing these network experiments has been design-based, leveraging randomization of treatment assignment as the basis for inference. In this paper, we generalize this design-based approach so that it can be applied to more complex experiments with a variety of causal estimands with different target populations. An important special case of such generalized network experiments is a bipartite network experiment, in which the treatment assignment is randomized among one set of units and the outcome is measured for a separate set of units. We propose a broad class of causal estimands based on stochastic intervention for generalized network experiments. Using a design-based approach, we show how to estimate the proposed causal quantities without bias, and develop conservative variance estimators. We apply our methodology to a randomized experiment in education where a group of selected students in middle schools are eligible for the anti-conflict promotion program, and the program participation is randomized within this group. In particular, our analysis estimates the causal effects of treating each student or his/her close friends, for different target populations in the network. We find that while the treatment improves the overall awareness against conflict among students, it does not significantly reduce the total number of conflicts. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Scalable kernel balancing weights in a nationwide observational study of hospital profit status and heart attack outcomes

Authors: Kwangho Kim, Bijan A. Niknam, José R. Zubizarreta

Abstract: Weighting is a general and often-used method for statistical adjustment. Weighting has two objectives: first, to balance covariate distributions, and second, to ensure that the weights have minimal dispersion and thus produce a more stable estimator. A recent, increasingly common approach directly optimizes the weights toward these two objectives. However, this approach has not yet been feasible i… ▽ More Weighting is a general and often-used method for statistical adjustment. Weighting has two objectives: first, to balance covariate distributions, and second, to ensure that the weights have minimal dispersion and thus produce a more stable estimator. A recent, increasingly common approach directly optimizes the weights toward these two objectives. However, this approach has not yet been feasible in large-scale datasets when investigators wish to flexibly balance general basis functions in an extended feature space. For example, many balancing approaches cannot scale to national-level health services research studies. To address this practical problem, we describe a scalable and flexible approach to weighting that integrates a basis expansion in a reproducing kernel Hilbert space with state-of-the-art convex optimization techniques. Specifically, we use the rank-restricted Nyström method to efficiently compute a kernel basis for balancing in {nearly} linear time and space, and then use the specialized first-order alternating direction method of multipliers to rapidly find the optimal weights. In an extensive simulation study, we provide new insights into the performance of weighting estimators in large datasets, showing that the proposed approach substantially outperforms others in terms of accuracy and speed. Finally, we use this weighting approach to conduct a national study of the relationship between hospital profit status and heart attack outcomes in a comprehensive dataset of 1.27 million patients. We find that for-profit hospitals use interventional cardiology to treat heart attacks at similar rates as other hospitals, but have higher mortality and readmission rates. △ Less

Submitted 1 November, 2023; originally announced November 2023.

Fair and Robust Estimation of Heterogeneous Treatment Effects for Policy Learning

Authors: Kwangho Kim, José R. Zubizarreta

Abstract: We propose a simple and general framework for nonparametric estimation of heterogeneous treatment effects under fairness constraints. Under standard regularity conditions, we show that the resulting estimators possess the double robustness property. We use this framework to characterize the trade-off between fairness and the maximum welfare achievable by the optimal policy. We evaluate the methods… ▽ More We propose a simple and general framework for nonparametric estimation of heterogeneous treatment effects under fairness constraints. Under standard regularity conditions, we show that the resulting estimators possess the double robustness property. We use this framework to characterize the trade-off between fairness and the maximum welfare achievable by the optimal policy. We evaluate the methods in a simulation study and illustrate them in a real-world case study. △ Less

Submitted 20 December, 2023; v1 submitted 6 June, 2023; originally announced June 2023.

Journal ref: Proceedings of the 40 th International Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 16997--17014, 2023

Notes on Causation, Comparison, and Regression

Authors: Ambarish Chattopadhyay, Jose R. Zubizarreta

Abstract: Comparison and contrast are the basic means to unveil causation and learn which treatments work. To build good comparison groups, randomized experimentation is key, yet often infeasible. In such non-experimental settings, we illustrate and discuss diagnostics to assess how well the common linear regression approach to causal inference approximates desirable features of randomized experiments, such… ▽ More Comparison and contrast are the basic means to unveil causation and learn which treatments work. To build good comparison groups, randomized experimentation is key, yet often infeasible. In such non-experimental settings, we illustrate and discuss diagnostics to assess how well the common linear regression approach to causal inference approximates desirable features of randomized experiments, such as covariate balance, study representativeness, interpolated estimation, and unweighted analyses. We also discuss alternative regression modeling, weighting, and matching approaches and argue they should be given strong consideration in empirical work. △ Less

Submitted 28 January, 2024; v1 submitted 23 May, 2023; originally announced May 2023.

Risk Set Matched Difference-in-Differences for the Analysis of Effect Modification in an Observational Study on the Impact of Gun Violence on Health Outcomes

Authors: Eric R. Cohn, Zirui Song, Jose R. Zubizarreta

Abstract: Gun violence is a major source of injury and death in the United States. However, relatively little is known about the effects of firearm injuries on survivors and their family members and how these effects vary across subpopulations. To study these questions and, more generally, to address a gap in the causal inference literature, we present a framework for the study of effect modification or het… ▽ More Gun violence is a major source of injury and death in the United States. However, relatively little is known about the effects of firearm injuries on survivors and their family members and how these effects vary across subpopulations. To study these questions and, more generally, to address a gap in the causal inference literature, we present a framework for the study of effect modification or heterogeneous treatment effects in difference-in-differences designs. We implement a new matching technique, which combines profile matching and risk set matching, to (i) preserve the time alignment of covariates, exposure, and outcomes, avoiding pitfalls of other common approaches for difference-in-differences, and (ii) explicitly control biases due to imbalances in observed covariates in subgroups discovered from the data. Our case study shows significant and persistent effects of nonfatal firearm injuries on several health outcomes for those injured and on the mental health of their family members. Sensitivity analyses reveal that these results are moderately robust to unmeasured confounding bias. Finally, while the effects for those injured vary largely by the severity of the injury and its documented intent, for families, effects are strongest for those whose relative's injury is documented as resulting from an assault, self-harm, or law enforcement intervention. △ Less

Submitted 31 May, 2024; v1 submitted 6 May, 2023; originally announced May 2023.

lmw: Linear Model Weights for Causal Inference

Authors: Ambarish Chattopadhyay, Noah Greifer, Jose R. Zubizarreta

Abstract: The linear regression model is widely used in the biomedical and social sciences as well as in policy and business research to adjust for covariates and estimate the average effects of treatments. Behind every causal inference endeavor there is a hypothetical randomized experiment. However, in routine regression analyses in observational studies, it is unclear how well the adjustments made by regr… ▽ More The linear regression model is widely used in the biomedical and social sciences as well as in policy and business research to adjust for covariates and estimate the average effects of treatments. Behind every causal inference endeavor there is a hypothetical randomized experiment. However, in routine regression analyses in observational studies, it is unclear how well the adjustments made by regression approximate key features of randomized experiments, such as covariate balance, study representativeness, sample boundedness, and unweighted sampling. In this paper, we provide software to empirically address this question. We introduce the lmw package for R to compute the implied linear model weights and perform diagnostics for their evaluation. The weights are obtained as part of the design stage of the study; that is, without using outcome information. The implementation is general and applicable, for instance, in settings with instrumental variables and multi-valued treatments; in essence, in any situation where the linear model is the vehicle for adjustment and estimation of average treatment effects with discrete-valued interventions. △ Less

Submitted 20 April, 2024; v1 submitted 15 March, 2023; originally announced March 2023.

Doubly Robust Counterfactual Classification

Authors: Kwangho Kim, Edward H. Kennedy, José R. Zubizarreta

Abstract: We study counterfactual classification as a new tool for decision-making under hypothetical (contrary to fact) scenarios. We propose a doubly-robust nonparametric estimator for a general counterfactual classifier, where we can incorporate flexible constraints by casting the classification problem as a nonlinear mathematical program involving counterfactuals. We go on to analyze the rates of conver… ▽ More We study counterfactual classification as a new tool for decision-making under hypothetical (contrary to fact) scenarios. We propose a doubly-robust nonparametric estimator for a general counterfactual classifier, where we can incorporate flexible constraints by casting the classification problem as a nonlinear mathematical program involving counterfactuals. We go on to analyze the rates of convergence of the estimator and provide a closed-form expression for its asymptotic distribution. Our analysis shows that the proposed estimator is robust against nuisance model misspecification, and can attain fast $\sqrt{n}$ rates with tractable inference even when using nonparametric machine learning approaches. We study the empirical performance of our methods by simulation and apply them for recidivism risk prediction. △ Less

Submitted 15 January, 2023; originally announced January 2023.

Journal ref: 36th Conference on Neural Information Processing Systems (NeurIPS 2022)

Counterfactual Mean-variance Optimization

Authors: Kwangho Kim, Alan Mishler, José R. Zubizarreta

Abstract: We study a new class of estimands in causal inference, which are the solutions to a stochastic nonlinear optimization problem that in general cannot be obtained in closed form. The optimization problem describes the counterfactual state of a system after an intervention, and the solutions represent the optimal decisions in that counterfactual state. In particular, we develop a counterfactual mean-… ▽ More We study a new class of estimands in causal inference, which are the solutions to a stochastic nonlinear optimization problem that in general cannot be obtained in closed form. The optimization problem describes the counterfactual state of a system after an intervention, and the solutions represent the optimal decisions in that counterfactual state. In particular, we develop a counterfactual mean-variance optimization approach, which can be used for optimal allocation of resources after an intervention. We propose a doubly-robust nonparametric estimator for the optimal solution of the counterfactual mean-variance program. We go on to analyze rates of convergence and provide a closed-form expression for the asymptotic distribution of our estimator. Our analysis shows that the proposed estimator is robust against nuisance model misspecification, and can attain fast $\sqrt{n}$ rates with tractable inference even when using nonparametric methods. This result is applicable to general nonlinear optimization problems subject to linear constraints whose coefficients are unknown and must be estimated. In this way, our findings contribute to the literature in optimization as well as causal inference. We further discuss the problem of calibrating our counterfactual covariance estimator to improve the finite-sample properties of our proposed optimal solution estimators. Finally, we evaluate our methods via simulation, and apply them to problems in healthcare policy and portfolio construction. △ Less

Submitted 20 September, 2022; originally announced September 2022.

Balanced and Robust Randomized Treatment Assignments: The Finite Selection Model for the Health Insurance Experiment and Beyond

Authors: Ambarish Chattopadhyay, Carl N. Morris, Jose R. Zubizarreta

Abstract: The Finite Selection Model (FSM) was developed by Carl Morris in the 1970s for the design of the RAND Health Insurance Experiment (HIE) (Morris 1979, Newhouse et al. 1993), one of the largest and most comprehensive social science experiments conducted in the U.S. The idea behind the FSM is that each treatment group takes its turns selecting units in a fair and random order to optimize a common ass… ▽ More The Finite Selection Model (FSM) was developed by Carl Morris in the 1970s for the design of the RAND Health Insurance Experiment (HIE) (Morris 1979, Newhouse et al. 1993), one of the largest and most comprehensive social science experiments conducted in the U.S. The idea behind the FSM is that each treatment group takes its turns selecting units in a fair and random order to optimize a common assignment criterion. At each of its turns, a treatment group selects the available unit that maximally improves the combined quality of its resulting group of units in terms of the criterion. In the HIE and beyond, we revisit, formalize, and extend the FSM as a general tool for experimental design. Leveraging the idea of D-optimality, we propose and analyze a new selection criterion in the FSM. The FSM using the D-optimal selection function has no tuning parameters, is affine invariant, and when appropriate, retrieves several classical designs such as randomized block and matched-pair designs. For multi-arm experiments, we propose algorithms to generate a fair and random selection order of treatments. We demonstrate FSM's performance in a case study based on the HIE and in ten randomized studies from the health and social sciences. On average, the FSM achieves 68% better covariate balance than complete randomization and 56% better covariate balance than rerandomization in a typical study. We recommend the FSM be considered in experimental design for its conceptual simplicity, efficiency, and robustness. △ Less

Submitted 4 July, 2023; v1 submitted 19 May, 2022; originally announced May 2022.

One-Step weighting to generalize and transport treatment effect estimates to a target population

Authors: Ambarish Chattopadhyay, Eric R. Cohn, Jose R. Zubizarreta

Abstract: The problem of generalization and transportation of treatment effect estimates from a study sample to a target population is central to empirical research and statistical methodology. In both randomized experiments and observational studies, weighting methods are often used with this objective. Traditional methods construct the weights by separately modeling the treatment assignment and study sele… ▽ More The problem of generalization and transportation of treatment effect estimates from a study sample to a target population is central to empirical research and statistical methodology. In both randomized experiments and observational studies, weighting methods are often used with this objective. Traditional methods construct the weights by separately modeling the treatment assignment and study selection probabilities and then multiplying functions (e.g., inverses) of their estimates. In this work, we provide a justification and an implementation for weighting in a single step. We show a formal connection between this one-step method and inverse probability and inverse odds weighting. We demonstrate that the resulting estimator for the target average treatment effect is consistent, asymptotically Normal, multiply robust, and semiparametrically efficient. We evaluate the performance of the one-step estimator in a simulation study. We illustrate its use in a case study on the effects of physician racial diversity on preventive healthcare utilization among Black men in California. We provide R code implementing the methodology. △ Less

Submitted 15 June, 2023; v1 submitted 16 March, 2022; originally announced March 2022.

Privacy-Preserving, Communication-Efficient, and Target-Flexible Hospital Quality Measurement

Authors: Larry Han, Yige Li, Bijan A. Niknam, Jose R. Zubizarreta

Abstract: Integrating information from multiple data sources can enable more precise, timely, and generalizable decisions. However, it is challenging to make valid causal inferences using observational data from multiple data sources. For example, in healthcare, learning from electronic health records contained in different hospitals is desirable but difficult due to heterogeneity in patient case mix, diffe… ▽ More Integrating information from multiple data sources can enable more precise, timely, and generalizable decisions. However, it is challenging to make valid causal inferences using observational data from multiple data sources. For example, in healthcare, learning from electronic health records contained in different hospitals is desirable but difficult due to heterogeneity in patient case mix, differences in treatment guidelines, and data privacy regulations that preclude individual patient data from being pooled. Motivated to overcome these issues, we develop a federated causal inference framework. We devise a doubly robust estimator of the mean potential outcome in a target population and show that it is consistent even when some models are misspecified. To enable real-world use, our proposed algorithm is privacy-preserving (requiring only summary statistics to be shared between hospitals) and communication-efficient (requiring only one round of communication between hospitals). We implement our causal estimation and inference procedure to investigate the quality of hospital care provided by a diverse set of 51 candidate Cardiac Centers of Excellence, as measured by 30-day mortality and length of stay for acute myocardial infarction (AMI) patients. We find that our proposed federated global estimator improves the precision of treatment effect estimates by 59% to 91% compared to using data from the target hospital alone. This precision gain results in qualitatively different conclusions about the estimated effect of percutaneous coronary intervention (PCI) compared to medical management (MM) in 63% (32 of 51) of hospitals. We find that hospitals rarely excel in both PCI and MM, which highlights the importance of assessing performance on specific treatment regimens. △ Less

Submitted 6 February, 2023; v1 submitted 1 March, 2022; originally announced March 2022.

Comments: 49 pages of main text + 28 pages of supplemental material

Using Cardinality Matching to Design Balanced and Representative Samples for Observational Studies

Authors: Bijan A. Niknam, Jose R. Zubizarreta

Abstract: Cardinality matching is a computational method for finding the largest possible number of matched pairs of exposed and unexposed individuals from an observational dataset, with specified patterns of baseline characteristics that represent a target population for analysis. This article explains the process of cardinality matching and how it simultaneously addresses the concerns of balance, sample s… ▽ More Cardinality matching is a computational method for finding the largest possible number of matched pairs of exposed and unexposed individuals from an observational dataset, with specified patterns of baseline characteristics that represent a target population for analysis. This article explains the process of cardinality matching and how it simultaneously addresses the concerns of balance, sample size, and representativeness of matched samples in observational studies. △ Less

Submitted 11 January, 2022; originally announced January 2022.

Journal ref: JAMA. 2022;327(2):173-174

The Balancing Act in Causal Inference

Authors: Eli Ben-Michael, Avi Feller, David A. Hirshberg, José R. Zubizarreta

Abstract: The idea of covariate balance is at the core of causal inference. Inverse propensity weights play a central role because they are the unique set of weights that balance the covariate distributions of different treatment groups. We discuss two broad approaches to estimating these weights: the more traditional one, which fits a propensity score model and then uses the reciprocal of the estimated pro… ▽ More The idea of covariate balance is at the core of causal inference. Inverse propensity weights play a central role because they are the unique set of weights that balance the covariate distributions of different treatment groups. We discuss two broad approaches to estimating these weights: the more traditional one, which fits a propensity score model and then uses the reciprocal of the estimated propensity score to construct weights, and the balancing approach, which estimates the inverse propensity weights essentially by the method of moments, finding weights that achieve balance in the sample. We review ideas from the causal inference, sample surveys, and semiparametric estimation literatures, with particular attention to the role of balance as a sufficient condition for robust inference. We focus on the inverse propensity weighting and augmented inverse propensity weighting estimators for the average treatment effect given strong ignorability and consider generalizations for a broader class of problems including policy evaluation and the estimation of individualized treatment effects. △ Less

Submitted 27 October, 2021; originally announced October 2021.

Comments: 42 pages, 0 figures

MSC Class: 62Gxx

Profile Matching for the Generalization and Personalization of Causal Inferences

Authors: Eric R. Cohn, Jose R. Zubizarreta

Abstract: We introduce profile matching, a multivariate matching method for randomized experiments and observational studies that finds the largest possible unweighted samples across multiple treatment groups that are balanced relative to a covariate profile. This covariate profile can represent a specific population or a target individual, facilitating the generalization and personalization of causal infer… ▽ More We introduce profile matching, a multivariate matching method for randomized experiments and observational studies that finds the largest possible unweighted samples across multiple treatment groups that are balanced relative to a covariate profile. This covariate profile can represent a specific population or a target individual, facilitating the generalization and personalization of causal inferences. For generalization, because the profile often amounts to summary statistics for a target population, profile matching does not always require accessing individual-level data, which may be unavailable for confidentiality reasons. For personalization, the profile comprises the characteristics of a single individual. Profile matching achieves covariate balance by construction, but unlike existing approaches to matching, it does not require specifying a matching ratio, as this is implicitly optimized for the data. The method can also be used for the selection of units for study follow-up, and it readily applies to multi-valued treatments with many treatment categories. We evaluate the performance of profile matching in a simulation study of the generalization of a randomized trial to a target population. We further illustrate this method in an exploratory observational study of the relationship between opioid use and mental health outcomes. We analyze these relationships for three covariate profiles representing: (i) sexual minorities, (ii) the Appalachian United States, and (iii) the characteristics of a hypothetical vulnerable patient. The method can be implemented via the new function profmatch in the designmatch package for R, for which we provide a step-by-step tutorial. △ Less

Submitted 6 July, 2022; v1 submitted 20 May, 2021; originally announced May 2021.

Randomized and Balanced Allocation of Units into Treatment Groups Using the Finite Selection Model for R

Authors: Ambarish Chattopadhyay, Carl N. Morris, Jose R. Zubizarreta

Abstract: The original Finite Selection Model (FSM) was developed in the 1970s to enhance the design of the RAND Health Insurance Experiment (HIE; Newhouse et al. 1993). At the time of its development by Carl Morris (Morris 1979), there were fundamental computational limitations to make the method widely available for practitioners. Today, as randomized experiments increasingly become more common, there is… ▽ More The original Finite Selection Model (FSM) was developed in the 1970s to enhance the design of the RAND Health Insurance Experiment (HIE; Newhouse et al. 1993). At the time of its development by Carl Morris (Morris 1979), there were fundamental computational limitations to make the method widely available for practitioners. Today, as randomized experiments increasingly become more common, there is a need for implementing experimental designs that are randomized, balanced, robust, and easily applicable to several treatment groups. To help address this problem, we revisit the original FSM under the potential outcome framework for causal inference and provide its first readily available software implementation. In this paper, we provide an introduction to the FSM and a step-by-step guide for its use in R. △ Less

Submitted 5 May, 2021; originally announced May 2021.

Targeted Quality Measurement of Health Care Providers

Authors: Jose R. Zubizarreta, Yige Li, Nancy L. Keating, Mary Beth Landrum

Abstract: Measuring quality of cancer care delivered by US health providers is challenging. Patients receiving oncology care greatly vary in disease presentation among other key characteristics. In this paper we discuss a framework for institutional quality measurement which addresses the heterogeneity of patient populations. For this, we follow recent statistical developments on health outcomes research an… ▽ More Measuring quality of cancer care delivered by US health providers is challenging. Patients receiving oncology care greatly vary in disease presentation among other key characteristics. In this paper we discuss a framework for institutional quality measurement which addresses the heterogeneity of patient populations. For this, we follow recent statistical developments on health outcomes research and conceptualize the task of quality measurement as a causal inference problem, hel** to target flexible covariate profiles that can represent specific populations of interest. To our knowledge, such covariate profiles have not been used in the quality measurement literature. We use different clinically relevant covariate profiles and evaluate methods for layered case-mix adjustments that combine weighting and regression modeling approaches in a sequential manner in order to reduce model extrapolation and allow for provider effect modification. We appraise these methods in an extensive simulation study and highlight the practical utility of weighting methods that warn the investigator when case-mix adjustments are infeasible without some form of extrapolation that goes beyond the support of the data. In a study of cancer-care outcomes, we assess the performance of oncology practices for different profiles that correspond to the types of patients who may receive cancer care. We describe how the methods examined may be particularly important for high-stakes quality measurement, such as public reporting or performance-based payments. These methods may also be applied to support the health care decisions of individual patients and provide a path to personalized quality measurement. △ Less

Submitted 27 October, 2021; v1 submitted 5 May, 2021; originally announced May 2021.

On the implied weights of linear regression for causal inference

Authors: Ambarish Chattopadhyay, Jose R. Zubizarreta

Abstract: A basic principle in the design of observational studies is to approximate the randomized experiment that would have been conducted under controlled circumstances. Now, linear regression models are commonly used to analyze observational data and estimate causal effects. How do linear regression adjustments in observational studies emulate key features of randomized experiments, such as covariate b… ▽ More A basic principle in the design of observational studies is to approximate the randomized experiment that would have been conducted under controlled circumstances. Now, linear regression models are commonly used to analyze observational data and estimate causal effects. How do linear regression adjustments in observational studies emulate key features of randomized experiments, such as covariate balance, self-weighted sampling, and study representativeness? In this paper, we provide answers to this and related questions by analyzing the implied (individual-level data) weights of linear regression methods. We derive new closed-form expressions of the weights and examine their properties in both finite and asymptotic regimes. We show that the implied weights of general regression problems can be equivalently obtained by solving a convex optimization problem. Among others, we study doubly and multiply robust properties of regression estimators from the perspective of their implied weights. This equivalence allows us to bridge ideas from the regression modeling and causal inference literatures. As a result, we propose novel regression diagnostics for causal inference that are part of the design stage of an observational study. As special cases, we analyze the implied weights in common settings such as multi-valued treatments and regression adjustment after matching. We implement the weights and diagnostics in the new lmw package for R. △ Less

Submitted 7 July, 2022; v1 submitted 13 April, 2021; originally announced April 2021.

Complex Discontinuity Designs Using Covariates: Impact of School Grade Retention on Later Life Outcomes in Chile

Authors: Juan D. Diaz, Jose R. Zubizarreta

Abstract: Regression discontinuity designs are extensively used for causal inference in observational studies. However, they are usually confined to settings with simple treatment rules, determined by a single running variable, with a single cutoff. Motivated by the problem of estimating the impact of grade retention on educational and juvenile crime outcomes in Chile, we propose a framework and methods for… ▽ More Regression discontinuity designs are extensively used for causal inference in observational studies. However, they are usually confined to settings with simple treatment rules, determined by a single running variable, with a single cutoff. Motivated by the problem of estimating the impact of grade retention on educational and juvenile crime outcomes in Chile, we propose a framework and methods for complex discontinuity designs that encompasses multiple treatment rules. In this framework, the observed covariates play a central role for identification, estimation, and generalization of causal effects. Identification is non-parametric and relies on a local strong ignorability assumption. Estimation proceeds as in any observational study under strong ignorability, yet in a neighborhood of the cutoffs of the running variables. We discuss estimation approaches based on matching and weighting, including complementary regression modeling adjustments. We present assumptions for generalization; that is, for identification and estimation of average treatment effects for target populations. We also describe two approaches to select the neighborhood for analysis. We find that grade retention in Chile has a negative impact on future grade retention, but is not associated with drop** out of school or committing a juvenile crime. △ Less

Submitted 9 February, 2022; v1 submitted 12 April, 2020; originally announced April 2020.

Large Sample Properties of Matching for Balance

Authors: Yixin Wang, José R. Zubizarreta

Abstract: Matching methods are widely used for causal inference in observational studies. Among them, nearest neighbor matching is arguably the most popular. However, nearest neighbor matching does not generally yield an average treatment effect estimator that is $\sqrt{n}$-consistent (Abadie and Imbens, 2006). Are matching methods not $\sqrt{n}$-consistent in general? In this paper, we study a recent class… ▽ More Matching methods are widely used for causal inference in observational studies. Among them, nearest neighbor matching is arguably the most popular. However, nearest neighbor matching does not generally yield an average treatment effect estimator that is $\sqrt{n}$-consistent (Abadie and Imbens, 2006). Are matching methods not $\sqrt{n}$-consistent in general? In this paper, we study a recent class of matching methods that use integer programming to directly target aggregate covariate balance as opposed to finding close neighbor matches. We show that under suitable conditions these methods can yield simple estimators that are $\sqrt{n}$-consistent and asymptotically optimal. △ Less

Submitted 11 September, 2021; v1 submitted 26 May, 2019; originally announced May 2019.

Comments: 32 pages

Minimax Linear Estimation of the Retargeted Mean

Authors: David A. Hirshberg, Arian Maleki, Jose R. Zubizarreta

Abstract: Evaluating treatments received by one population for application to a different target population of scientific interest is a central problem in causal inference from observational studies. We study the minimax linear estimator of the treatment-specific mean outcome on a target population and provide a theoretical basis for inference based on it. In particular, we provide a justification for the c… ▽ More Evaluating treatments received by one population for application to a different target population of scientific interest is a central problem in causal inference from observational studies. We study the minimax linear estimator of the treatment-specific mean outcome on a target population and provide a theoretical basis for inference based on it. In particular, we provide a justification for the common practice of ignoring bias when building confidence intervals with these linear estimators. Focusing on the case that the class of the unknown outcome function is the unit ball of a reproducing kernel Hilbert space, we show that the resulting linear estimator is asymptotically optimal under conditions only marginally stronger than those used with augmented estimators. We establish bounds attesting to the estimator's good finite sample properties. In an extensive simulation study, we observe promising performance of the estimator throughout a wide range of sample sizes, noise levels, and levels of overlap between the covariate distributions of the treated and target populations. △ Less

Submitted 26 February, 2021; v1 submitted 10 January, 2019; originally announced January 2019.

Comments: 25 pages, 4 figures

Building Representative Matched Samples with Multi-valued Treatments in Large Observational Studies

Authors: Magdalena Bennett, Juan Pablo Vielma, Jose R. Zubizarreta

Abstract: In this paper, we present a new way of matching in observational studies that overcomes three limitations of existing matching approaches. First, it directly balances covariates with multi-valued treatments without requiring the generalized propensity score. Second, it builds self-weighted matched samples that are representative of a target population by design. Third, it can handle large data set… ▽ More In this paper, we present a new way of matching in observational studies that overcomes three limitations of existing matching approaches. First, it directly balances covariates with multi-valued treatments without requiring the generalized propensity score. Second, it builds self-weighted matched samples that are representative of a target population by design. Third, it can handle large data sets, with hundreds of thousands of observations, in a couple of minutes. The key insights of this new approach to matching are balancing the treatment groups relative to a target population and positing a linear-sized mixed integer formulation of the matching problem. We formally show that this formulation is more effective than alternative quadratic-sized formulations, as its reduction in size does not affect its strength from the standpoint of its linear programming relaxation. We also show that this formulation can be used for matching with distributional covariate balance in polynomial time under certain assumptions on the covariates and that it can handle large data sets in practice even when the assumptions are not satisfied. This algorithmic characterization is key to handle large data sets. We illustrate this new approach to matching in both a simulation study and an observational study of the impact of an earthquake on educational attainment. After matching, the results can be visualized with simple and transparent graphical displays: while increasing levels of exposure to the earthquake have a negative impact on school attendance, there is no effect on college admission test scores. △ Less

Submitted 9 July, 2019; v1 submitted 15 October, 2018; originally announced October 2018.

Shape-constrained partial identification of a population mean under unknown probabilities of sample selection

Authors: Luke W. Miratrix, Stefan Wager, Jose R. Zubizarreta

Abstract: A prevailing challenge in the biomedical and social sciences is to estimate a population mean from a sample obtained with unknown selection probabilities. Using a well-known ratio estimator, Aronow and Lee (2013) proposed a method for partial identification of the mean by allowing the unknown selection probabilities to vary arbitrarily between two fixed extreme values. In this paper, we show how t… ▽ More A prevailing challenge in the biomedical and social sciences is to estimate a population mean from a sample obtained with unknown selection probabilities. Using a well-known ratio estimator, Aronow and Lee (2013) proposed a method for partial identification of the mean by allowing the unknown selection probabilities to vary arbitrarily between two fixed extreme values. In this paper, we show how to leverage auxiliary shape constraints on the population outcome distribution, such as symmetry or log-concavity, to obtain tighter bounds on the population mean. We use this method to estimate the performance of Aymara students---an ethnic minority in the north of Chile---in a national educational standardized test. We implement this method in the new statistical software package scbounds for R. △ Less

Submitted 22 June, 2017; originally announced June 2017.

Minimal Dispersion Approximately Balancing Weights: Asymptotic Properties and Practical Considerations

Authors: Yixin Wang, José R. Zubizarreta

Abstract: Weighting methods are widely used to adjust for covariates in observational studies, sample surveys, and regression settings. In this paper, we study a class of recently proposed weighting methods which find the weights of minimum dispersion that approximately balance the covariates. We call these weights "minimal weights" and study them under a common optimization framework. The key observation i… ▽ More Weighting methods are widely used to adjust for covariates in observational studies, sample surveys, and regression settings. In this paper, we study a class of recently proposed weighting methods which find the weights of minimum dispersion that approximately balance the covariates. We call these weights "minimal weights" and study them under a common optimization framework. The key observation is the connection between approximate covariate balance and shrinkage estimation of the propensity score. This connection leads to both theoretical and practical developments. From a theoretical standpoint, we characterize the asymptotic properties of minimal weights and show that, under standard smoothness conditions on the propensity score function, minimal weights are consistent estimates of the true inverse probability weights. Also, we show that the resulting weighting estimator is consistent, asymptotically normal, and semiparametrically efficient. From a practical standpoint, we present a finite sample oracle inequality that bounds the loss incurred by balancing more functions of the covariates than strictly needed. This inequality shows that minimal weights implicitly bound the number of active covariate balance constraints. We finally provide a tuning algorithm for choosing the degree of approximate balance in minimal weights. We conclude the paper with four empirical studies that suggest approximate balance is preferable to exact balance, especially when there is limited overlap in covariate distributions. In these studies, we show that the root mean squared error of the weighting estimator can be reduced by as much as a half with approximate balance. △ Less

Submitted 24 April, 2019; v1 submitted 2 May, 2017; originally announced May 2017.

Comments: 41 pages

Confidence intervals for means under constrained dependence

Authors: Peter M. Aronow, Forrest W. Crawford, José R. Zubizarreta

Abstract: We develop a general framework for conducting inference on the mean of dependent random variables given constraints on their dependency graph. We establish the consistency of an oracle variance estimator of the mean when the dependency graph is known, along with an associated central limit theorem. We derive an integer linear program for finding an upper bound for the estimated variance when the g… ▽ More We develop a general framework for conducting inference on the mean of dependent random variables given constraints on their dependency graph. We establish the consistency of an oracle variance estimator of the mean when the dependency graph is known, along with an associated central limit theorem. We derive an integer linear program for finding an upper bound for the estimated variance when the graph is unknown, but topological and degree-based constraints are available. We develop alternative bounds, including a closed-form bound, under an additional homoskedasticity assumption. We establish a basis for Wald-type confidence intervals for the mean that are guaranteed to have asymptotically conservative coverage. We apply the approach to inference from a social network link-tracing study and provide statistical software implementing the approach. △ Less

Submitted 31 January, 2016; originally announced February 2016.

Isolation in the construction of natural experiments

Authors: José R. Zubizarreta, Dylan S. Small, Paul R. Rosenbaum

Abstract: A natural experiment is a type of observational study in which treatment assignment, though not randomized by the investigator, is plausibly close to random. A process that assigns treatments in a highly nonrandom, inequitable manner may, in rare and brief moments, assign aspects of treatments at random or nearly so. Isolating those moments and aspects may extract a natural experiment from a setti… ▽ More A natural experiment is a type of observational study in which treatment assignment, though not randomized by the investigator, is plausibly close to random. A process that assigns treatments in a highly nonrandom, inequitable manner may, in rare and brief moments, assign aspects of treatments at random or nearly so. Isolating those moments and aspects may extract a natural experiment from a setting in which treatment assignment is otherwise quite biased, far from random. Isolation is a tool that focuses on those rare, brief instances, extracting a small natural experiment from otherwise useless data. We discuss the theory behind isolation and illustrate its use in a reanalysis of a well-known study of the effects of fertility on workforce participation. Whether a woman becomes pregnant at a certain moment in her life and whether she brings that pregnancy to term may reflect her aspirations for family, education and career, the degree of control she exerts over her fertility, and the quality of her relationship with the father; moreover, these aspirations and relationships are unlikely to be recorded with precision in surveys and censuses, and they may confound studies of workforce participation. However, given that a women is pregnant and will bring the pregnancy to term, whether she will have twins or a single child is, to a large extent, simply luck. Given that a woman is pregnant at a certain moment, the differential comparison of two types of pregnancies on workforce participation, twins or a single child, may be close to randomized, not biased by unmeasured aspirations. In this comparison, we find in our case study that mothers of twins had more children but only slightly reduced workforce participation, approximately 5% less time at work for an additional child. △ Less

Submitted 19 January, 2015; originally announced January 2015.

Comments: Published in at http://dx.doi.org/10.1214/14-AOAS770 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS770

Journal ref: Annals of Applied Statistics 2014, Vol. 8, No. 4, 2096-2121

Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System

Authors: Luke Keele, Jose R. Zubizarreta

Abstract: A distinctive feature of a clustered observational study is its multilevel or nested data structure arising from the assignment of treatment, in a non-random manner, to groups or clusters of units or individuals. Examples are ubiquitous in the health and social sciences including patients in hospitals, employees in firms, and students in schools. What is the optimal matching strategy in a clustere… ▽ More A distinctive feature of a clustered observational study is its multilevel or nested data structure arising from the assignment of treatment, in a non-random manner, to groups or clusters of units or individuals. Examples are ubiquitous in the health and social sciences including patients in hospitals, employees in firms, and students in schools. What is the optimal matching strategy in a clustered observational study? At first thought, one might start by matching clusters of individuals and then, within matched clusters, continue by matching individuals. But as we discuss in this paper, the optimal strategy is the opposite: in typical applications, where the intracluster correlation is not perfect, it is best to first match individuals and, once all possible combinations of matched individuals are known, then match clusters. In this paper we use dynamic and integer programming to implement this strategy and extend optimal matching methods to hierarchical and multilevel settings. Among other matched designs, our strategy can approximate a paired clustered randomized study by finding the largest sample of matched pairs of treated and control individuals within matched pairs of treated and control clusters that is balanced according to specifications given by the investigator. This strategy directly balances covariates both at the cluster and individual levels and does not require estimating the propensity score, although the propensity score can be balanced as an additional covariate. We illustrate our results with a case study of the comparative effectiveness of public versus private voucher schools in Chile, a question of intense policy debate in the country at the present. △ Less

Submitted 28 April, 2016; v1 submitted 30 September, 2014; originally announced September 2014.

Matching for balance, pairing for heterogeneity in an observational study of the effectiveness of for-profit and not-for-profit high schools in Chile

Authors: José R. Zubizarreta, Ricardo D. Paredes, Paul R. Rosenbaum

Abstract: Conventionally, the construction of a pair-matched sample selects treated and control units and pairs them in a single step with a view to balancing observed covariates $\mathbf{x}$ and reducing the heterogeneity or dispersion of treated-minus-control response differences, $Y$. In contrast, the method of cardinality matching developed here first selects the maximum number of units subject to covar… ▽ More Conventionally, the construction of a pair-matched sample selects treated and control units and pairs them in a single step with a view to balancing observed covariates $\mathbf{x}$ and reducing the heterogeneity or dispersion of treated-minus-control response differences, $Y$. In contrast, the method of cardinality matching developed here first selects the maximum number of units subject to covariate balance constraints and, with a balanced sample for $\mathbf{x}$ in hand, then separately pairs the units to minimize heterogeneity in $Y$. Reduced heterogeneity of pair differences in responses $Y$ is known to reduce sensitivity to unmeasured biases, so one might hope that cardinality matching would succeed at both tasks, balancing $\mathbf{x}$, stabilizing $Y$. We use cardinality matching in an observational study of the effectiveness of for-profit and not-for-profit private high schools in Chile - a controversial subject in Chile - focusing on students who were in government run primary schools in 2004 but then switched to private high schools. By pairing to minimize heterogeneity in a cardinality match that has balanced covariates, a meaningful reduction in sensitivity to unmeasured biases is obtained. △ Less

Submitted 14 April, 2014; originally announced April 2014.

Comments: Published in at http://dx.doi.org/10.1214/13-AOAS713 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS713

Journal ref: Annals of Applied Statistics 2014, Vol. 8, No. 1, 204-231

Stronger instruments via integer programming in an observational study of late preterm birth outcomes

Authors: José R. Zubizarreta, Dylan S. Small, Neera K. Goyal, Scott Lorch, Paul R. Rosenbaum

Abstract: In an optimal nonbipartite match, a single population is divided into matched pairs to minimize a total distance within matched pairs. Nonbipartite matching has been used to strengthen instrumental variables in observational studies of treatment effects, essentially by forming pairs that are similar in terms of covariates but very different in the strength of encouragement to accept the treatment.… ▽ More In an optimal nonbipartite match, a single population is divided into matched pairs to minimize a total distance within matched pairs. Nonbipartite matching has been used to strengthen instrumental variables in observational studies of treatment effects, essentially by forming pairs that are similar in terms of covariates but very different in the strength of encouragement to accept the treatment. Optimal nonbipartite matching is typically done using network optimization techniques that can be quick, running in polynomial time, but these techniques limit the tools available for matching. Instead, we use integer programming techniques, thereby obtaining a wealth of new tools not previously available for nonbipartite matching, including fine and near-fine balance for several nominal variables, forced near balance on means and optimal subsetting. We illustrate the methods in our on-going study of outcomes of late-preterm births in California, that is, births of 34 to 36 weeks of gestation. Would lengthening the time in the hospital for such births reduce the frequency of rapid readmissions? A straightforward comparison of babies who stay for a shorter or longer time would be severely biased, because the principal reason for a long stay is some serious health problem. We need an instrument, something inconsequential and haphazard that encourages a shorter or a longer stay in the hospital. It turns out that babies born at certain times of day tend to stay overnight once with a shorter length of stay, whereas babies born at other times of day tend to stay overnight twice with a longer length of stay, and there is nothing particularly special about a baby who is born at 11:00 pm. △ Less

Submitted 15 April, 2013; originally announced April 2013.

Comments: Published in at http://dx.doi.org/10.1214/12-AOAS582 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS582

Journal ref: Annals of Applied Statistics 2013, Vol. 7, No. 1, 25-50