Showing 1–23 of 23 results for author: Aronow, P M

Randomization-based confidence intervals for the local average treatment effect

Authors: P. M. Aronow, Haoge Chang, Patrick Lopatto

Abstract: We consider the problem of generating confidence intervals in randomized experiments with noncompliance. We show that a refinement of a randomization-based procedure proposed by Imbens and Rosenbaum (2005) has desirable properties. Namely, we show that using a studentized Anderson-Rubin-type statistic as a test statistic yields confidence intervals that are finite-sample exact under treatment effe… ▽ More We consider the problem of generating confidence intervals in randomized experiments with noncompliance. We show that a refinement of a randomization-based procedure proposed by Imbens and Rosenbaum (2005) has desirable properties. Namely, we show that using a studentized Anderson-Rubin-type statistic as a test statistic yields confidence intervals that are finite-sample exact under treatment effect homogeneity, and remain asymptotically valid for the Local Average Treatment Effect when the treatment effect is heterogeneous. We provide a uniform analysis of this procedure. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: 40 pages

Operationalizing Counterfactual Metrics: Incentives, Ranking, and Information Asymmetry

Authors: Serena Wang, Stephen Bates, P. M. Aronow, Michael I. Jordan

Abstract: From the social sciences to machine learning, it has been well documented that metrics to be optimized are not always aligned with social welfare. In healthcare, Dranove et al. (2003) showed that publishing surgery mortality metrics actually harmed the welfare of sicker patients by increasing provider selection behavior. We analyze the incentive misalignments that arise from such average treated o… ▽ More From the social sciences to machine learning, it has been well documented that metrics to be optimized are not always aligned with social welfare. In healthcare, Dranove et al. (2003) showed that publishing surgery mortality metrics actually harmed the welfare of sicker patients by increasing provider selection behavior. We analyze the incentive misalignments that arise from such average treated outcome metrics, and show that the incentives driving treatment decisions would align with maximizing total patient welfare if the metrics (i) accounted for counterfactual untreated outcomes and (ii) considered total welfare instead of averaging over treated patients. Operationalizing this, we show how counterfactual metrics can be modified to behave reasonably in patient-facing ranking systems. Extending to realistic settings when providers observe more about patients than the regulatory agencies do, we bound the decay in performance by the degree of information asymmetry between principal and agent. In doing so, our model connects principal-agent information asymmetry with unobserved heterogeneity in causal inference. △ Less

Submitted 29 November, 2023; v1 submitted 23 May, 2023; originally announced May 2023.

Fast computation of exact confidence intervals for randomized experiments with binary outcomes

Authors: P. M. Aronow, Haoge Chang, Patrick Lopatto

Abstract: Given a randomized experiment with binary outcomes, exact confidence intervals for the average causal effect of the treatment can be computed through a series of permutation tests. This approach requires minimal assumptions and is valid for all sample sizes, as it does not rely on large-sample approximations such as the central limit theorem. We show that these confidence intervals can be found in… ▽ More Given a randomized experiment with binary outcomes, exact confidence intervals for the average causal effect of the treatment can be computed through a series of permutation tests. This approach requires minimal assumptions and is valid for all sample sizes, as it does not rely on large-sample approximations such as the central limit theorem. We show that these confidence intervals can be found in $O(n \log n)$ permutation tests in the case of balanced designs, where the treatment and control groups have equal sizes, and $O(n^2)$ permutation tests in the general case. Prior to this work, the most efficient known constructions required $O(n^2)$ such tests in the balanced case [Li and Ding, 2016], and $O(n^4)$ tests in the general case [Rigdon and Hudgens, 2015]. Our results thus facilitate exact inference as a viable option for randomized experiments far larger than those accessible by previous methods. △ Less

Submitted 16 May, 2023; originally announced May 2023.

Comments: 37 pages

Exact Bias Correction for Linear Adjustment of Randomized Controlled Trials

Authors: Haoge Chang, Joel Middleton, P. M. Aronow

Abstract: In an influential critique of empirical practice, Freedman (2008) showed that the linear regression estimator was biased for the analysis of randomized controlled trials under the randomization model. Under Freedman's assumptions, we derive exact closed-form bias corrections for the linear regression estimator with and without treatment-by-covariate interactions. We show that the limiting distribu… ▽ More In an influential critique of empirical practice, Freedman (2008) showed that the linear regression estimator was biased for the analysis of randomized controlled trials under the randomization model. Under Freedman's assumptions, we derive exact closed-form bias corrections for the linear regression estimator with and without treatment-by-covariate interactions. We show that the limiting distribution of the bias corrected estimator is identical to the uncorrected estimator, implying that the asymptotic gains from adjustment can be attained without introducing any risk of bias. Taken together with results from Lin (2013), our results show that Freedman's theoretical arguments against the use of regression adjustment can be completely resolved with minor modifications to practice. △ Less

Submitted 25 October, 2021; v1 submitted 15 October, 2021; originally announced October 2021.

On the reliability of published findings using the regression discontinuity design in political science

Authors: Drew Stommes, P. M. Aronow, Fredrik Sävje

Abstract: The regression discontinuity (RD) design offers identification of causal effects under weak assumptions, earning it a position as a standard method in modern political science research. But identification does not necessarily imply that causal effects can be estimated accurately with limited data. In this paper, we highlight that estimation under the RD design involves serious statistical challeng… ▽ More The regression discontinuity (RD) design offers identification of causal effects under weak assumptions, earning it a position as a standard method in modern political science research. But identification does not necessarily imply that causal effects can be estimated accurately with limited data. In this paper, we highlight that estimation under the RD design involves serious statistical challenges and investigate how these challenges manifest themselves in the empirical literature in political science. We collect all RD-based findings published in top political science journals in the period 2009-2018. The distribution of published results exhibits pathological features; estimates tend to bunch just above the conventional level of statistical significance. A reanalysis of all studies with available data suggests that researcher discretion is not a major driver of these features. However, researchers tend to use inappropriate methods for inference, rendering standard errors artificially small. A retrospective power analysis reveals that most of these studies were underpowered to detect all but large effects. The issues we uncover, combined with well-documented selection pressures in academic publishing, cause concern that many published findings using the RD design may be exaggerated. △ Less

Submitted 12 March, 2023; v1 submitted 29 September, 2021; originally announced September 2021.

Dyadic Clustering in International Relations

Authors: Jacob Carlson, Trevor Incerti, P. M. Aronow

Abstract: Quantitative empirical inquiry in international relations often relies on dyadic data. Standard analytic techniques do not account for the fact that dyads are not generally independent of one another. That is, when dyads share a constituent member (e.g., a common country), they may be statistically dependent, or "clustered." Recent work has developed dyadic clustering robust standard errors (DCRSE… ▽ More Quantitative empirical inquiry in international relations often relies on dyadic data. Standard analytic techniques do not account for the fact that dyads are not generally independent of one another. That is, when dyads share a constituent member (e.g., a common country), they may be statistically dependent, or "clustered." Recent work has developed dyadic clustering robust standard errors (DCRSEs) that account for this dependence. Using these DCRSEs, we reanalyzed all empirical articles published in International Organization between January 2014 and January 2020 that feature dyadic data. We find that published standard errors for key explanatory variables are, on average, approximately half as large as DCRSEs, suggesting that dyadic clustering is leading researchers to severely underestimate uncertainty. However, most (67% of) statistically significant findings remain statistically significant when using DCRSEs. We conclude that accounting for dyadic clustering is both important and feasible, and offer software in R and Stata to facilitate use of DCRSEs in future research. △ Less

Submitted 21 July, 2023; v1 submitted 8 September, 2021; originally announced September 2021.

Nonparametric identification is not enough, but randomized controlled trials are

Authors: P. M. Aronow, James M. Robins, Theo Saarinen, Fredrik Sävje, Jasjeet Sekhon

Abstract: We argue that randomized controlled trials (RCTs) are special even among settings where average treatment effects are identified by a nonparametric unconfoundedness assumption. This claim follows from two results of Robins and Ritov (1997): (1) with at least one continuous covariate control, no estimator of the average treatment effect exists which is uniformly consistent without further assumptio… ▽ More We argue that randomized controlled trials (RCTs) are special even among settings where average treatment effects are identified by a nonparametric unconfoundedness assumption. This claim follows from two results of Robins and Ritov (1997): (1) with at least one continuous covariate control, no estimator of the average treatment effect exists which is uniformly consistent without further assumptions, (2) knowledge of the propensity score yields a uniformly consistent estimator and honest confidence intervals that shrink at parametric rates with increasing sample size, regardless of how complicated the propensity score function is. We emphasize the latter point, and note that successfully-conducted RCTs provide knowledge of the propensity score to the researcher. We discuss modern developments in covariate adjustment for RCTs, noting that statistical models and machine learning methods can be used to improve efficiency while preserving finite sample unbiasedness. We conclude that statistical inference has the potential to be fundamentally more difficult in observational settings than it is in RCTs, even when all confounders are measured. △ Less

Submitted 26 September, 2021; v1 submitted 25 August, 2021; originally announced August 2021.

Inference in Spatial Experiments with Interference using the SpatialEffect Package

Authors: Peter M. Aronow, Cyrus Samii, Jonathan Sullivan, Ye Wang

Abstract: This paper presents methods for analyzing spatial experiments when complex spillovers, displacement effects, and other types of "interference" are present. We present a robust, design-based approach to analyzing effects in such settings. The design-based approach derives inferential properties for causal effect estimators from known features of the experimental design, in a manner analogous to inf… ▽ More This paper presents methods for analyzing spatial experiments when complex spillovers, displacement effects, and other types of "interference" are present. We present a robust, design-based approach to analyzing effects in such settings. The design-based approach derives inferential properties for causal effect estimators from known features of the experimental design, in a manner analogous to inference in sample surveys. The methods presented here target a quantity of interest called the "average marginalized response," which is equal to the average effect of activating a treatment at an intervention point that is a given distance away, averaging ambient effects emanating from other intervention points. We provide a step-by-step tutorial based on the SpatialEffect package for R. We apply the methods to a randomized experiment on payments for community forest conservation in Uganda, showing how our methods reveal possibly substantial spatial spillovers that more conventional analyses cannot detect. △ Less

Submitted 29 June, 2021; originally announced June 2021.

Listwise Deletion in High Dimensions

Authors: J. Sophia Wang, Peter M. Aronow

Abstract: We consider the properties of listwise deletion when both $n$ and the number of variables grow large. We show that when (i) all data has some idiosyncratic missingness and (ii) the number of variables grows superlogarithmically in $n$, then, for large $n$, listwise deletion will drop all rows with probability 1. Using two canonical datasets from the study of comparative politics and international… ▽ More We consider the properties of listwise deletion when both $n$ and the number of variables grow large. We show that when (i) all data has some idiosyncratic missingness and (ii) the number of variables grows superlogarithmically in $n$, then, for large $n$, listwise deletion will drop all rows with probability 1. Using two canonical datasets from the study of comparative politics and international relations, we provide numerical illustration that these problems may emerge in real world settings. These results suggest, in practice, using listwise deletion may mean using few of the variables available to the researcher. △ Less

Submitted 18 July, 2021; v1 submitted 11 January, 2021; originally announced January 2021.

Design-Based Inference for Spatial Experiments under Unknown Interference

Authors: Ye Wang, Cyrus Samii, Haoge Chang, P. M. Aronow

Abstract: We consider design-based causal inference in settings where randomized treatments have effects that bleed out into space in complex ways that overlap and in violation of the standard "no interference" assumption for many causal inference methods. We define a spatial "average marginalized effect," which characterizes how, in expectation, units of observation that are a specified distance from an in… ▽ More We consider design-based causal inference in settings where randomized treatments have effects that bleed out into space in complex ways that overlap and in violation of the standard "no interference" assumption for many causal inference methods. We define a spatial "average marginalized effect," which characterizes how, in expectation, units of observation that are a specified distance from an intervention node are affected by treatment at that node, averaging over effects emanating from other intervention nodes. We establish conditions for non-parametric identification under unknown interference, asymptotic distributions of estimators, and recovery of structural effects. We propose methods for both sample-theoretic and permutation-based inference. We provide illustrations using randomized field experiments on forest conservation and health. △ Less

Submitted 9 March, 2023; v1 submitted 26 October, 2020; originally announced October 2020.

Review of The Book of Why: The New Science of Cause and Effect

Authors: Peter M. Aronow, Fredrik Sävje

Abstract: Book review published as: Aronow, Peter M. and Fredrik Sävje (2020), "The Book of Why: The New Science of Cause and Effect." Journal of the American Statistical Association, 115: 482-485. Book review published as: Aronow, Peter M. and Fredrik Sävje (2020), "The Book of Why: The New Science of Cause and Effect." Journal of the American Statistical Association, 115: 482-485. △ Less

Submitted 25 March, 2020; originally announced March 2020.

Journal ref: J. Amer. Statist. Assoc. (2020) 115: 482--485

Spillover Effects in Experimental Data

Authors: Peter M. Aronow, Dean Eckles, Cyrus Samii, Stephanie Zonszein

Abstract: We present current methods for estimating treatment effects and spillover effects under "interference", a term which covers a broad class of situations in which a unit's outcome depends not only on treatments received by that unit, but also on treatments received by other units. To the extent that units react to each other, interact, or otherwise transmit effects of treatments, valid inference req… ▽ More We present current methods for estimating treatment effects and spillover effects under "interference", a term which covers a broad class of situations in which a unit's outcome depends not only on treatments received by that unit, but also on treatments received by other units. To the extent that units react to each other, interact, or otherwise transmit effects of treatments, valid inference requires that we account for such interference, which is a departure from the traditional assumption that units' outcomes are affected only by their own treatment assignment. Interference and associated spillovers may be a nuisance or they may be of substantive interest to the researcher. In this chapter, we focus on interference in the context of randomized experiments. We review methods for when interference happens in a general network setting. We then consider the special case where interference is contained within a hierarchical structure. Finally, we discuss the relationship between interference and contagion. We use the interference R package and simulated data to illustrate key points. We consider efficient designs that allow for estimation of the treatment and spillover effects and discuss recent empirical studies that try to capture such effects. △ Less

Submitted 15 January, 2020; originally announced January 2020.

Comments: Forthcoming in Advances in Experimental Political Science, J. N. Druckman and D. P. Green, eds. Cambridge University Press

A note on breaking ties among sample medians

Authors: Peter M. Aronow, Donald K. K. Lee

Abstract: Given samples $x_1,\cdots,x_n$, it is well known that any sample median value (not necessarily unique) minimizes the absolute loss $\sum_{i=1}^n |q-x_i|$. Interestingly, we show that the minimizer of the loss $\sum_{i=1}^n|q-x_i|^{1+ε}$ exhibits a singular perturbation behaviour that provides a unique definition for the sample median as $ε\rightarrow 0$. This definition is the unique point among a… ▽ More Given samples $x_1,\cdots,x_n$, it is well known that any sample median value (not necessarily unique) minimizes the absolute loss $\sum_{i=1}^n |q-x_i|$. Interestingly, we show that the minimizer of the loss $\sum_{i=1}^n|q-x_i|^{1+ε}$ exhibits a singular perturbation behaviour that provides a unique definition for the sample median as $ε\rightarrow 0$. This definition is the unique point among all candidate median values that balances the $logarithmic$ moment of the empirical distribution. The result generalizes directly to breaking ties among sample quantiles when the quantile regression loss is modified in the same way. △ Less

Submitted 2 September, 2019; v1 submitted 9 July, 2018; originally announced July 2018.

Comments: 5 pages, no tables or figures

Average treatment effects in the presence of unknown interference

Authors: Fredrik Sävje, Peter M. Aronow, Michael G. Hudgens

Abstract: We investigate large-sample properties of treatment effect estimators under unknown interference in randomized experiments. The inferential target is a generalization of the average treatment effect estimand that marginalizes over potential spillover effects. We show that estimators commonly used to estimate treatment effects under no interference are consistent for the generalized estimand for se… ▽ More We investigate large-sample properties of treatment effect estimators under unknown interference in randomized experiments. The inferential target is a generalization of the average treatment effect estimand that marginalizes over potential spillover effects. We show that estimators commonly used to estimate treatment effects under no interference are consistent for the generalized estimand for several common experimental designs under limited but otherwise arbitrary and unknown interference. The rates of convergence depend on the rate at which the amount of interference grows and the degree to which it aligns with dependencies in treatment assignment. Importantly for practitioners, the results imply that if one erroneously assumes that units do not interfere in a setting with limited, or even moderate, interference, standard estimators are nevertheless likely to be close to an average treatment effect if the sample is sufficiently large. Conventional confidence statements may, however, not be accurate. △ Less

Submitted 23 October, 2019; v1 submitted 16 November, 2017; originally announced November 2017.

A Note on "How Robust Standard Errors Expose Methodological Problems They Do Not Fix, and What to Do About It"

Authors: Peter M. Aronow

Abstract: King and Roberts (2015, KR) claim that a disagreement between robust and classical standard errors exposes model misspecification. We emphasize that KR's claim only generally applies to parametric models: models that assume a restrictive form of the distribution of the outcome. Many common models in use in political science, including the linear model, are not necessarily parametric -- rather they… ▽ More King and Roberts (2015, KR) claim that a disagreement between robust and classical standard errors exposes model misspecification. We emphasize that KR's claim only generally applies to parametric models: models that assume a restrictive form of the distribution of the outcome. Many common models in use in political science, including the linear model, are not necessarily parametric -- rather they may be semiparametric. Common estimators of model parameters such as ordinary least squares have both robust (corresponding to a semiparametric model) and classical (corresponding to a more restrictive model) standard error estimates. Given a properly specified semiparametric model and mild regularity conditions, the classical standard errors are not generally consistent, but the robust standard errors are. To illustrate this point, we consider the case of the regression estimate of a semiparametric linear model with no model misspecification, and show that robust standard errors may nevertheless systematically differ from classical standard errors. We show that a disagreement between robust and classical standard errors is not generally suitable as a diagnostic for regression estimators, and that KR's reanalyses of Neumayer (2003) and Büthe and Milner (2008) are predicated on strong assumptions that the original authors did not invoke nor require. △ Less

Submitted 6 September, 2016; originally announced September 2016.

Confidence intervals for means under constrained dependence

Authors: Peter M. Aronow, Forrest W. Crawford, José R. Zubizarreta

Abstract: We develop a general framework for conducting inference on the mean of dependent random variables given constraints on their dependency graph. We establish the consistency of an oracle variance estimator of the mean when the dependency graph is known, along with an associated central limit theorem. We derive an integer linear program for finding an upper bound for the estimated variance when the g… ▽ More We develop a general framework for conducting inference on the mean of dependent random variables given constraints on their dependency graph. We establish the consistency of an oracle variance estimator of the mean when the dependency graph is known, along with an associated central limit theorem. We derive an integer linear program for finding an upper bound for the estimated variance when the graph is unknown, but topological and degree-based constraints are available. We develop alternative bounds, including a closed-form bound, under an additional homoskedasticity assumption. We establish a basis for Wald-type confidence intervals for the mean that are guaranteed to have asymptotically conservative coverage. We apply the approach to inference from a social network link-tracing study and provide statistical software implementing the approach. △ Less

Submitted 31 January, 2016; originally announced February 2016.

Local average causal effects and superefficiency

Authors: Peter M. Aronow

Abstract: Recent approaches in causal inference have proposed estimating average causal effects that are local to some subpopulation, often for reasons of efficiency. These inferential targets are sometimes data-adaptive, in that they are dependent on the empirical distribution of the data. In this short note, we show that if researchers are willing to adapt the inferential target on the basis of efficiency… ▽ More Recent approaches in causal inference have proposed estimating average causal effects that are local to some subpopulation, often for reasons of efficiency. These inferential targets are sometimes data-adaptive, in that they are dependent on the empirical distribution of the data. In this short note, we show that if researchers are willing to adapt the inferential target on the basis of efficiency, then extraordinary gains in precision can be obtained. Specifically, when causal effects are heterogeneous, any asymptotically normal and root-$n$ consistent estimator of the population average causal effect is superefficient for a data-adaptive local average causal effect. Our result illustrates the fundamental gain in statistical certainty afforded by indifference about the inferential target. △ Less

Submitted 4 February, 2016; v1 submitted 7 January, 2016; originally announced January 2016.

Identification of homophily and preferential recruitment in respondent-driven sampling

Authors: Forrest W. Crawford, Peter M. Aronow, Li Zeng, Jianghong Li

Abstract: Respondent-driven sampling (RDS) is a link-tracing procedure for surveying hidden or hard-to-reach populations in which subjects recruit other subjects via their social network. There is significant research interest in detecting clustering or dependence of epidemiological traits in networks, but researchers disagree about whether data from RDS studies can reveal it. Two distinct mechanisms accoun… ▽ More Respondent-driven sampling (RDS) is a link-tracing procedure for surveying hidden or hard-to-reach populations in which subjects recruit other subjects via their social network. There is significant research interest in detecting clustering or dependence of epidemiological traits in networks, but researchers disagree about whether data from RDS studies can reveal it. Two distinct mechanisms account for dependence in traits of recruiters and recruitees in an RDS study: homophily, the tendency for individuals to share social ties with others exhibiting similar characteristics, and preferential recruitment, in which recruiters do not recruit uniformly at random from their available alters. The different effects of network homophily and preferential recruitment in RDS studies have been a source of confusion in methodological research on RDS, and in empirical studies of the social context of health risk in hidden populations. In this paper, we give rigorous definitions of homophily and preferential recruitment and show that neither can be measured precisely in general RDS studies. We derive nonparametric identification regions for homophily and preferential recruitment and show that these parameters are not point identified unless the network takes a degenerate form. The results indicate that claims of homophily or recruitment bias measured from empirical RDS studies may not be credible. We apply our identification results to a study involving both a network census and RDS on a population of injection drug users in Hartford, CT. △ Less

Submitted 17 November, 2015; originally announced November 2015.

Nonparametric Identification for Respondent-Driven Sampling

Authors: Peter M. Aronow, Forrest W. Crawford

Abstract: Respondent-driven sampling is a survey method for hidden or hard-to-reach populations in which sampled individuals recruit others in the study population via their social links. The most popular estimator for for the population mean assumes that individual sampling probabilities are proportional to each subject's reported degree in a social network connecting members of the hidden population. Howe… ▽ More Respondent-driven sampling is a survey method for hidden or hard-to-reach populations in which sampled individuals recruit others in the study population via their social links. The most popular estimator for for the population mean assumes that individual sampling probabilities are proportional to each subject's reported degree in a social network connecting members of the hidden population. However, it remains unclear under what circumstances these estimators are valid, and what assumptions are formally required to identify population quantities. In this short note we detail nonparametric identification results for the population mean when the sampling probability is assumed to be a function of network degree known to scale. Importantly, we establish general conditions for the consistency of the popular Volz-Heckathorn (VH) estimator. Our results imply that the conditions for consistency of the VH estimator are far less stringent than those suggested by recent work on diagnostics for RDS. In particular, our results do not require random sampling or the existence of a network connecting the population. △ Less

Submitted 14 April, 2015; originally announced April 2015.

Sharp bounds on the variance in randomized experiments

Authors: Peter M. Aronow, Donald P. Green, Donald K. K. Lee

Abstract: We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773-785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465-472. Reprint of the original 1923 paper]. A practical implication of our results is that… ▽ More We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773-785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465-472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials. △ Less

Submitted 26 May, 2014; originally announced May 2014.

Comments: Published in at http://dx.doi.org/10.1214/13-AOS1200 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS1200

Journal ref: Annals of Statistics 2014, Vol. 42, No. 3, 850-871

Cluster-Robust Variance Estimation for Dyadic Data

Authors: Peter M. Aronow, Cyrus Samii, Valentina A. Assenova

Abstract: Dyadic data are common in the social sciences, although inference for such settings involves accounting for a complex clustering structure. Many analyses in the social sciences fail to account for the fact that multiple dyads share a member, and that errors are thus likely correlated across these dyads. We propose a nonparametric sandwich-type robust variance estimator for linear regression to acc… ▽ More Dyadic data are common in the social sciences, although inference for such settings involves accounting for a complex clustering structure. Many analyses in the social sciences fail to account for the fact that multiple dyads share a member, and that errors are thus likely correlated across these dyads. We propose a nonparametric sandwich-type robust variance estimator for linear regression to account for such clustering in dyadic data. We enumerate conditions for estimator consistency. We also extend our results to repeated and weighted observations, including directed dyads and longitudinal data, and provide an implementation for generalized linear models such as logistic regression. We examine empirical performance with simulations and applications to international relations and speed dating. △ Less

Submitted 22 July, 2015; v1 submitted 11 December, 2013; originally announced December 2013.

Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence

Authors: Peter M. Aronow, Alexander Coppock, Forrest W. Crawford, Donald P. Green

Abstract: Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments may be less prone to bias than direct questioning, list experiments are also more susceptible to sampling variability. We show that researchers do not h… ▽ More Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments may be less prone to bias than direct questioning, list experiments are also more susceptible to sampling variability. We show that researchers do not have to abandon direct questioning altogether in order to gain the advantages of list experimentation. We develop a nonparametric estimator of the prevalence of sensitive behaviors that combines list experimentation and direct questioning. We prove that this estimator is asymptotically more efficient than the standard difference-in-means estimator, and we provide a basis for inference using Wald-type confidence intervals. Additionally, leveraging information from the direct questioning, we derive two nonparametric placebo tests of the identifying assumptions for the list experiment. We demonstrate the effectiveness of our combined estimator and placebo tests with an original survey experiment. △ Less

Submitted 1 June, 2014; v1 submitted 4 December, 2013; originally announced December 2013.

Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment

Authors: Peter M. Aronow, Cyrus Samii

Abstract: This paper presents a randomization-based framework for estimating causal effects under interference between units, motivated by challenges that arise in analyzing experiments on social networks. The framework integrates three components: (i) an experimental design that defines the probability distribution of treatment assignments, (ii) a map** that relates experimental treatment assignments to… ▽ More This paper presents a randomization-based framework for estimating causal effects under interference between units, motivated by challenges that arise in analyzing experiments on social networks. The framework integrates three components: (i) an experimental design that defines the probability distribution of treatment assignments, (ii) a map** that relates experimental treatment assignments to exposures received by units in the experiment, and (iii) estimands that make use of the experiment to answer questions of substantive interest. We develop the case of estimating average unit-level causal effects from a randomized experiment with interference of arbitrary but known form. The resulting estimators are based on inverse probability weighting. We provide randomization-based variance estimators that account for the complex clustering that can occur when interference is present. We also establish consistency and asymptotic normality under local dependence assumptions. We discuss refinements including covariate-adjusted effect estimators and ratio estimation. We evaluate empirical performance in realistic settings with a naturalistic simulation using social network data from American schools. We then present results from a field experiment on the spread of anti-conflict norms and behavior among school students. △ Less

Submitted 20 June, 2018; v1 submitted 27 May, 2013; originally announced May 2013.

MSC Class: 62-07; 62G05; 62P25

Journal ref: Aronow, Peter M.; Samii, Cyrus. Estimating average causal effects under general interference, with application to a social network experiment. Ann. Appl. Stat. 11 (2017), no. 4, 1912--1947