Showing 1–3 of 3 results for author: Shem-Tov, Y

On Recoding Ordered Treatments as Binary Indicators

Authors: Evan K. Rose, Yotam Shem-Tov

Abstract: Researchers using instrumental variables to investigate ordered treatments often recode treatment into an indicator for any exposure. We investigate this estimand under the assumption that the instruments shift compliers from no treatment to some but not from some treatment to more. We show that when there are extensive margin compliers only (EMCO) this estimand captures a weighted average of trea… ▽ More Researchers using instrumental variables to investigate ordered treatments often recode treatment into an indicator for any exposure. We investigate this estimand under the assumption that the instruments shift compliers from no treatment to some but not from some treatment to more. We show that when there are extensive margin compliers only (EMCO) this estimand captures a weighted average of treatment effects that can be partially unbundled into each complier group's potential outcome means. We also establish an equivalence between EMCO and a two-factor selection model and apply our results to study treatment heterogeneity in the Oregon Health Insurance Experiment. △ Less

Submitted 1 March, 2024; v1 submitted 23 November, 2021; originally announced November 2021.

Inference on a New Class of Sample Average Treatment Effects

Authors: Jasjeet S. Sekhon, Yotam Shem-Tov

Abstract: We derive new variance formulas for inference on a general class of estimands of causal average treatment effects in a Randomized Control Trial (RCT). We generalize Robins (1988) and show that when the estimand of interest is the Sample Average Treatment Effect of the Treated (SATT or SATC for controls), a consistent variance estimator exists. Although these estimands are equal to the Sample Avera… ▽ More We derive new variance formulas for inference on a general class of estimands of causal average treatment effects in a Randomized Control Trial (RCT). We generalize Robins (1988) and show that when the estimand of interest is the Sample Average Treatment Effect of the Treated (SATT or SATC for controls), a consistent variance estimator exists. Although these estimands are equal to the Sample Average Treatment Effect (SATE) in expectation, potentially large differences in both accuracy and coverage can occur by the change of estimand, even asymptotically. Inference on the SATE, even using a conservative confidence interval, provides incorrect coverage of the SATT or SATC. We derive the variance and limiting distribution of a new and general class of estimands---any mixing between SATT and SATC---for which the SATE is a specific case. We demonstrate the applicability of the new theoretical results using Monte-Carlo simulations and an empirical application with hundreds of online experiments with an average sample size of approximately one hundred million observations per experiment. An R package, estCI, that implements all the proposed estimation procedures is available. △ Less

Submitted 18 October, 2017; v1 submitted 7 August, 2017; originally announced August 2017.

The Classification Permutation Test: A Nonparametric Test for Equality of Multivariate Distributions

Authors: Johann Gagnon-Bartsch, Yotam Shem-Tov

Abstract: The gold standard for identifying causal relationships is a randomized controlled experiment. In many applications in the social sciences and medicine, the researcher does not control the assignment mechanism and instead may rely upon natural experiments, regression discontinuity designs, RCTs with attrition, or matching methods as a substitute to experimental randomization. The standard testable… ▽ More The gold standard for identifying causal relationships is a randomized controlled experiment. In many applications in the social sciences and medicine, the researcher does not control the assignment mechanism and instead may rely upon natural experiments, regression discontinuity designs, RCTs with attrition, or matching methods as a substitute to experimental randomization. The standard testable implication of random assignment is covariate balance between the treated and control units. Covariate balance is therefore commonly used to validate the claim of "as-if" random assignment. We develop a new nonparametric test of covariate balance. Our Classification Permutation Test (CPT) is based on a combination of classification methods (e.g. logistic regression or random forests) with Fisherian permutation inference. The CPT is guaranteed to have correct coverage and is consistent under weak assumptions on the chosen classifier. To illustrate the gains of using the CPT, we revisit four real data examples: Lyall (2009); Green and Winik (2010); Eggers and Hainmueller (2009); and Rouse (1995). Monte Carlo power simulations are used to compare the CPT to two existing nonparametric tests of equality of multivariate distributions. △ Less

Submitted 19 November, 2016; originally announced November 2016.