-
Assumption Smuggling in Intermediate Outcome Tests of Causal Mechanisms
Authors:
Matthew Blackwell,
Ruofan Ma,
Aleksei Opacic
Abstract:
Political scientists are increasingly attuned to the promises and pitfalls of establishing causal effects. But the vital question for many is not if a causal effect exists but why and how it exists. Even so, many researchers avoid causal mediation analyses due to the assumptions required, instead opting to explore causal mechanisms through what we call intermediate outcome tests. These tests use t…
▽ More
Political scientists are increasingly attuned to the promises and pitfalls of establishing causal effects. But the vital question for many is not if a causal effect exists but why and how it exists. Even so, many researchers avoid causal mediation analyses due to the assumptions required, instead opting to explore causal mechanisms through what we call intermediate outcome tests. These tests use the same research design used to estimate the effect of treatment on the outcome to estimate the effect of the treatment on one or more mediators, with authors often concluding that evidence of the latter is evidence of a causal mechanism. We show in this paper that, without further assumptions, this can neither establish nor rule out the existence of a causal mechanism. Instead, such conclusions about the indirect effect of treatment rely on implicit and usually very strong assumptions that are often unmet. Thus, such causal mechanism tests, though very common in political science, should not be viewed as a free lunch but rather should be used judiciously, and researchers should explicitly state and defend the requisite assumptions.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Priming bias versus post-treatment bias in experimental designs
Authors:
Matthew Blackwell,
Jacob R. Brown,
Sophie Hill,
Kosuke Imai,
Teppei Yamamoto
Abstract:
Conditioning on variables affected by treatment can induce post-treatment bias when estimating causal effects. Although this suggests that researchers should measure potential moderators before administering the treatment in an experiment, doing so may also bias causal effect estimation if the covariate measurement primes respondents to react differently to the treatment. This paper formally analy…
▽ More
Conditioning on variables affected by treatment can induce post-treatment bias when estimating causal effects. Although this suggests that researchers should measure potential moderators before administering the treatment in an experiment, doing so may also bias causal effect estimation if the covariate measurement primes respondents to react differently to the treatment. This paper formally analyzes this trade-off between post-treatment and priming biases in three experimental designs that vary when moderators are measured: pre-treatment, post-treatment, or a randomized choice between the two. We derive nonparametric bounds for interactions between the treatment and the moderator under each design and show how to use substantive assumptions to narrow these bounds. These bounds allow researchers to assess the sensitivity of their empirical findings to either source of bias. We then apply the proposed methodology to a survey experiment on electoral messaging.
△ Less
Submitted 28 June, 2024; v1 submitted 1 June, 2023;
originally announced June 2023.
-
Adjusting for Unmeasured Confounding in Marginal Structural Models with Propensity-Score Fixed Effects
Authors:
Matthew Blackwell,
Soichiro Yamauchi
Abstract:
Marginal structural models are a popular tool for investigating the effects of time-varying treatments, but they require an assumption of no unobserved confounders between the treatment and outcome. With observational data, this assumption may be difficult to maintain, and in studies with panel data, many researchers use fixed effects models to purge the data of time-constant unmeasured confoundin…
▽ More
Marginal structural models are a popular tool for investigating the effects of time-varying treatments, but they require an assumption of no unobserved confounders between the treatment and outcome. With observational data, this assumption may be difficult to maintain, and in studies with panel data, many researchers use fixed effects models to purge the data of time-constant unmeasured confounding. Unfortunately, traditional linear fixed effects models are not suitable for estimating the effects of time-varying treatments, since they can only estimate lagged effects under implausible assumptions. To resolve this tension, we a propose a novel inverse probability of treatment weighting estimator with propensity-score fixed effects to adjust for time-constant unmeasured confounding in marginal structural models of fixed-length treatment histories. We show that these estimators are consistent and asymptotically normal when the number of units and time periods grow at a similar rate. Unlike traditional fixed effect models, this approach works even when the outcome is only measured at a single point in time as is common in marginal structural models. We apply these methods to estimating the effect of negative advertising on the electoral success of candidates for statewide offices in the United States.
△ Less
Submitted 9 June, 2021; v1 submitted 7 May, 2021;
originally announced May 2021.
