-
Inference procedures in sequential trial emulation with survival outcomes: comparing confidence intervals based on the sandwich variance estimator, bootstrap and jackknife
Authors:
Juliette M. Limozin,
Shaun R. Seaman,
Li Su
Abstract:
Sequential trial emulation (STE) is an approach to estimating causal treatment effects by emulating a sequence of target trials from observational data. In STE, inverse probability weighting is commonly utilised to address time-varying confounding and/or dependent censoring. Then structural models for potential outcomes are applied to the weighted data to estimate treatment effects. For inference,…
▽ More
Sequential trial emulation (STE) is an approach to estimating causal treatment effects by emulating a sequence of target trials from observational data. In STE, inverse probability weighting is commonly utilised to address time-varying confounding and/or dependent censoring. Then structural models for potential outcomes are applied to the weighted data to estimate treatment effects. For inference, the simple sandwich variance estimator is popular but conservative, while nonparametric bootstrap is computationally expensive, and a more efficient alternative, linearised estimating function (LEF) bootstrap, has not been adapted to STE. We evaluated the performance of various methods for constructing confidence intervals (CIs) of marginal risk differences in STE with survival outcomes by comparing the coverage of CIs based on nonparametric/LEF bootstrap, jackknife, and the sandwich variance estimator through simulations. LEF bootstrap CIs demonstrated the best coverage with small/moderate sample sizes, low event rates and low treatment prevalence, which were the motivating scenarios for STE. They were less affected by treatment group imbalance and faster to compute than nonparametric bootstrap CIs. With large sample sizes and medium/high event rates, the sandwich-variance-estimator-based CIs had the best coverage and were the fastest to compute. These findings offer guidance in constructing CIs in causal survival analysis using STE.
△ Less
Submitted 12 July, 2024; v1 submitted 11 July, 2024;
originally announced July 2024.
-
TrialEmulation: An R Package to Emulate Target Trials for Causal Analysis of Observational Time-to-event Data
Authors:
Li Su,
Roonak Rezvani,
Shaun R. Seaman,
Colin Starr,
Isaac Gravestock
Abstract:
Randomised controlled trials (RCTs) are regarded as the gold standard for estimating causal treatment effects on health outcomes. However, RCTs are not always feasible, because of time, budget or ethical constraints. Observational data such as those from electronic health records (EHRs) offer an alternative way to estimate the causal effects of treatments. Recently, the `target trial emulation' fr…
▽ More
Randomised controlled trials (RCTs) are regarded as the gold standard for estimating causal treatment effects on health outcomes. However, RCTs are not always feasible, because of time, budget or ethical constraints. Observational data such as those from electronic health records (EHRs) offer an alternative way to estimate the causal effects of treatments. Recently, the `target trial emulation' framework was proposed by Hernan and Robins (2016) to provide a formal structure for estimating causal treatment effects from observational data. To promote more widespread implementation of target trial emulation in practice, we develop the R package TrialEmulation to emulate a sequence of target trials using observational time-to-event data, where individuals who start to receive treatment and those who have not been on the treatment at the baseline of the emulated trials are compared in terms of their risks of an outcome event. Specifically, TrialEmulation provides (1) data preparation for emulating a sequence of target trials, (2) calculation of the inverse probability of treatment and censoring weights to handle treatment switching and dependent censoring, (3) fitting of marginal structural models for the time-to-event outcome given baseline covariates, (4) estimation and inference of marginal intention to treat and per-protocol effects of the treatment in terms of marginal risk differences between treated and untreated for a user-specified target trial population. In particular, TrialEmulation can accommodate large data sets (e.g., from EHRs) within memory constraints of R by processing data in chunks and applying case-control sampling. We demonstrate the functionality of TrialEmulation using a simulated data set that mimics typical observational time-to-event data in practice.
△ Less
Submitted 19 February, 2024;
originally announced February 2024.
-
Simulating data from marginal structural models for a survival time outcome
Authors:
Shaun R Seaman,
Ruth H Keogh
Abstract:
Marginal structural models (MSMs) are often used to estimate causal effects of treatments on survival time outcomes from observational data when time-dependent confounding may be present. They can be fitted using, e.g., inverse probability of treatment weighting (IPTW). It is important to evaluate the performance of statistical methods in different scenarios, and simulation studies are a key tool…
▽ More
Marginal structural models (MSMs) are often used to estimate causal effects of treatments on survival time outcomes from observational data when time-dependent confounding may be present. They can be fitted using, e.g., inverse probability of treatment weighting (IPTW). It is important to evaluate the performance of statistical methods in different scenarios, and simulation studies are a key tool for such evaluations. In such simulation studies, it is common to generate data in such a way that the model of interest is correctly specified, but this is not always straightforward when the model of interest is for potential outcomes, as is an MSM. Methods have been proposed for simulating from MSMs for a survival outcome, but these methods impose restrictions on the data-generating mechanism. Here we propose a method that overcomes these restrictions. The MSM can be a marginal structural logistic model for a discrete survival time or a Cox or additive hazards MSM for a continuous survival time. The hazard of the potential survival time can be conditional on baseline covariates, and the treatment variable can be discrete or continuous. We illustrate the use of the proposed simulation algorithm by carrying out a brief simulation study. This study compares the coverage of confidence intervals calculated in two different ways for causal effect estimates obtained by fitting an MSM via IPTW.
△ Less
Submitted 23 December, 2023; v1 submitted 10 September, 2023;
originally announced September 2023.
-
Relationship between Collider Bias and Interactions on the Log-Additive Scale
Authors:
Apostolos Gkatzionis,
Shaun R. Seaman,
Rachael A. Hughes,
Kate Tilling
Abstract:
Collider bias occurs when conditioning on a common effect (collider) of two variables $X, Y$. In this manuscript, we quantify the collider bias in the estimated association between exposure $X$ and outcome $Y$ induced by selecting on one value of a binary collider $S$ of the exposure and the outcome. In the case of logistic regression, it is known that the magnitude of the collider bias in the exp…
▽ More
Collider bias occurs when conditioning on a common effect (collider) of two variables $X, Y$. In this manuscript, we quantify the collider bias in the estimated association between exposure $X$ and outcome $Y$ induced by selecting on one value of a binary collider $S$ of the exposure and the outcome. In the case of logistic regression, it is known that the magnitude of the collider bias in the exposure-outcome regression coefficient is proportional to the strength of interaction $δ_3$ between $X$ and $Y$ in a log-additive model for the collider: $\mathbb{P} (S = 1 | X, Y) = \exp \left\{ δ_0 + δ_1 X + δ_2 Y + δ_3 X Y \right\}$. We show that this result also holds under a linear or Poisson regression model for the exposure-outcome association. We then illustrate by simulation that even if a log-additive model with interactions is not the true model for the collider, the interaction term in such a model is still informative about the magnitude of collider bias. Finally, we discuss the implications of these findings for methods that attempt to adjust for collider bias, such as inverse probability weighting which is often implemented without including interactions between variables in the weighting model.
△ Less
Submitted 7 August, 2023; v1 submitted 1 August, 2023;
originally announced August 2023.
-
Causal inference in survival analysis using longitudinal observational data: Sequential trials and marginal structural models
Authors:
Ruth H. Keogh,
Jon Michael Gran,
Shaun R. Seaman,
Gwyneth Davies,
Stijn Vansteelandt
Abstract:
Longitudinal observational patient data can be used to investigate the causal effects of time-varying treatments on time-to-event outcomes. Several methods have been developed for controlling for the time-dependent confounding that typically occurs. The most commonly used is inverse probability weighted estimation of marginal structural models (MSM-IPTW). An alternative, the sequential trials appr…
▽ More
Longitudinal observational patient data can be used to investigate the causal effects of time-varying treatments on time-to-event outcomes. Several methods have been developed for controlling for the time-dependent confounding that typically occurs. The most commonly used is inverse probability weighted estimation of marginal structural models (MSM-IPTW). An alternative, the sequential trials approach, is increasingly popular, in particular in combination with the target trial emulation framework. This approach involves creating a sequence of `trials' from new time origins, restricting to individuals as yet untreated and meeting other eligibility criteria, and comparing treatment initiators and non-initiators. Individuals are censored when they deviate from their treatment status at the start of each `trial' (initiator/non-initiator) and this is addressed using inverse probability of censoring weights. The analysis is based on data combined across trials. We show that the sequential trials approach can estimate the parameter of a particular MSM, and compare it to a MSM-IPTW with respect to the estimands being identified, the assumptions needed and how data are used differently. We show how both approaches can estimate the same marginal risk differences. The two approaches are compared using a simulation study. The sequential trials approach, which tends to involve less extreme weights than MSM-IPTW, results in greater efficiency for estimating the marginal risk difference at most follow-up times, but this can, in certain scenarios, be reversed at late time points. We apply the methods to longitudinal observational data from the UK Cystic Fibrosis Registry to estimate the effect of dornase alfa on survival.
△ Less
Submitted 6 October, 2021;
originally announced October 2021.
-
Evaluating the impact of local tracing partnerships on the performance of contact tracing for COVID-19 in England
Authors:
Pantelis Samartsidis,
Shaun R. Seaman,
Abbie Harrison,
Angelos Alexopoulos,
Gareth J. Hughes,
Christopher Rawlinson,
Charlotte Anderson,
Andre Charlett,
Isabel Oliver,
Daniela De Angelis
Abstract:
Assessing the impact of an intervention using time-series observational data on multiple units and outcomes is a frequent problem in many fields of scientific research. In this paper, we present a novel method to estimate intervention effects in such a setting by generalising existing approaches based on the factor analysis model and develo** a Bayesian algorithm for inference. Our method is one…
▽ More
Assessing the impact of an intervention using time-series observational data on multiple units and outcomes is a frequent problem in many fields of scientific research. In this paper, we present a novel method to estimate intervention effects in such a setting by generalising existing approaches based on the factor analysis model and develo** a Bayesian algorithm for inference. Our method is one of the few that can simultaneously: deal with outcomes of mixed type (continuous, binomial, count); increase efficiency in the estimates of the causal effects by jointly modelling multiple outcomes affected by the intervention; easily provide uncertainty quantification for all causal estimands of interest. We use the proposed approach to evaluate the impact that local tracing partnerships (LTP) had on the effectiveness of England's Test and Trace (TT) programme for COVID-19. Our analyses suggest that, overall, LTPs had a small positive impact on TT. However, there is considerable heterogeneity in the estimates of the causal effects over units and time.
△ Less
Submitted 5 October, 2021;
originally announced October 2021.
-
Hospitalisation risk for COVID-19 patients infected with SARS-CoV-2 variant B.1.1.7: cohort analysis
Authors:
Tommy Nyberg,
Katherine A. Twohig,
Ross J. Harris,
Shaun R. Seaman,
Joe Flannagan,
Hester Allen,
Andre Charlett,
Daniela De Angelis,
Gavin Dabrera,
Anne M. Presanis
Abstract:
Objective: To evaluate the relationship between coronavirus disease 2019 (COVID-19) diagnosis with SARS-CoV-2 variant B.1.1.7 (also known as Variant of Concern 202012/01) and the risk of hospitalisation compared to diagnosis with wildtype SARS-CoV-2 variants.
Design: Retrospective cohort, analysed using stratified Cox regression.
Setting: Community-based SARS-CoV-2 testing in England, individu…
▽ More
Objective: To evaluate the relationship between coronavirus disease 2019 (COVID-19) diagnosis with SARS-CoV-2 variant B.1.1.7 (also known as Variant of Concern 202012/01) and the risk of hospitalisation compared to diagnosis with wildtype SARS-CoV-2 variants.
Design: Retrospective cohort, analysed using stratified Cox regression.
Setting: Community-based SARS-CoV-2 testing in England, individually linked with hospitalisation data.
Participants: 839,278 laboratory-confirmed COVID-19 patients, of whom 36,233 had been hospitalised within 14 days, tested between 23rd November 2020 and 31st January 2021 and analysed at a laboratory with an available TaqPath assay that enables assessment of S-gene target failure (SGTF). SGTF is a proxy test for the B.1.1.7 variant. Patient data were stratified by age, sex, ethnicity, deprivation, region of residence, and date of positive test.
Main outcome measures: Hospitalisation between 1 and 14 days after the first positive SARS-CoV-2 test.
Results: 27,710 of 592,409 SGTF patients (4.7%) and 8,523 of 246,869 non-SGTF patients (3.5%) had been hospitalised within 1-14 days. The stratum-adjusted hazard ratio (HR) of hospitalisation was 1.52 (95% confidence interval [CI] 1.47 to 1.57) for COVID-19 patients infected with SGTF variants, compared to those infected with non-SGTF variants. The effect was modified by age (P<0.001), with HRs of 0.93-1.21 for SGTF compared to non-SGTF patients below age 20 years, 1.29 in those aged 20-29, and 1.45-1.65 in age groups 30 years or older.
Conclusions: The results suggest that the risk of hospitalisation is higher for individuals infected with the B.1.1.7 variant compared to wildtype SARS-CoV-2, likely reflecting a more severe disease. The higher severity may be specific to adults above the age of 30.
△ Less
Submitted 29 May, 2021; v1 submitted 12 April, 2021;
originally announced April 2021.
-
Simulating longitudinal data from marginal structural models using the additive hazard model
Authors:
Ruth H. Keogh,
Shaun R. Seaman,
Jon Michael Gran,
Stijn Vansteelandt
Abstract:
Observational longitudinal data on treatments and covariates are increasingly used to investigate treatment effects, but are often subject to time-dependent confounding. Marginal structural models (MSMs), estimated using inverse probability of treatment weighting or the g-formula, are popular for handling this problem. With increasing development of advanced causal inference methods, it is importa…
▽ More
Observational longitudinal data on treatments and covariates are increasingly used to investigate treatment effects, but are often subject to time-dependent confounding. Marginal structural models (MSMs), estimated using inverse probability of treatment weighting or the g-formula, are popular for handling this problem. With increasing development of advanced causal inference methods, it is important to be able to assess their performance in different scenarios to guide their application. Simulation studies are a key tool for this, but their use to evaluate causal inference methods has been limited. This paper focuses on the use of simulations for evaluations involving MSMs in studies with a time-to-event outcome. In a simulation, it is important to be able to generate the data in such a way that the correct form of any models to be fitted to those data is known. However, this is not straightforward in the longitudinal setting because it is natural for data to be generated in a sequential conditional manner, whereas MSMs involve fitting marginal rather than conditional hazard models. We provide general results that enable the form of the correctly-specified MSM to be derived based on a conditional data generating procedure, and show how the results can be applied when the conditional hazard model is an Aalen additive hazard or Cox model. Using conditional additive hazard models is advantageous because they imply additive MSMs that can be fitted using standard software. We describe and illustrate a simulation algorithm. Our results will help researchers to effectively evaluate causal inference methods via simulation.
△ Less
Submitted 10 February, 2020;
originally announced February 2020.
-
Assessing the causal effect of binary interventions from observational panel data with few treated units
Authors:
Pantelis Samartsidis,
Shaun R. Seaman,
Anne M. Presanis,
Matthew Hickman,
Daniela De Angelis
Abstract:
Researchers are often challenged with assessing the impact of an intervention on an outcome of interest in situations where the intervention is non-randomised, the intervention is only applied to one or few units, the intervention is binary, and outcome measurements are available at multiple time points. In this paper, we review existing methods for causal inference in these situations. We detail…
▽ More
Researchers are often challenged with assessing the impact of an intervention on an outcome of interest in situations where the intervention is non-randomised, the intervention is only applied to one or few units, the intervention is binary, and outcome measurements are available at multiple time points. In this paper, we review existing methods for causal inference in these situations. We detail the assumptions underlying each method, emphasize connections between the different approaches and provide guidelines regarding their practical implementation. Several open problems are identified thus highlighting the need for future research.
△ Less
Submitted 19 December, 2019; v1 submitted 20 April, 2018;
originally announced April 2018.
-
Propensity score analysis with partially observed confounders: how should multiple imputation be used?
Authors:
Clemence Leyrat,
Shaun R. Seaman,
Ian R. White,
Ian Douglas,
Liam Smeeth,
Joseph Kim,
Matthieu Resche-Rigon,
James R. Carpenter,
Elizabeth J. Williamson
Abstract:
Inverse probability of treatment weighting (IPTW) is a popular propensity score (PS)-based approach to estimate causal effects in observational studies at risk of confounding bias. A major issue when estimating the PS is the presence of partially observed covariates. Multiple imputation (MI) is a natural approach to handle missing data on covariates, but its use in the PS context raises three impo…
▽ More
Inverse probability of treatment weighting (IPTW) is a popular propensity score (PS)-based approach to estimate causal effects in observational studies at risk of confounding bias. A major issue when estimating the PS is the presence of partially observed covariates. Multiple imputation (MI) is a natural approach to handle missing data on covariates, but its use in the PS context raises three important questions: (i) should we apply Rubin's rules to the IPTW treatment effect estimates or to the PS estimates themselves? (ii) does the outcome have to be included in the imputation model? (iii) how should we estimate the variance of the IPTW estimator after MI? We performed a simulation study focusing on the effect of a binary treatment on a binary outcome with three confounders (two of them partially observed). We used MI with chained equations to create complete datasets and compared three ways of combining the results: combining treatment effect estimates (MIte); combining the PS across the imputed datasets (MIps); or combining the PS parameters and estimating the PS of the average covariates across the imputed datasets (MIpar). We also compared the performance of these methods to complete case (CC) analysis and the missingness pattern (MP) approach, a method which uses a different PS model for each pattern of missingness. We also studied empirically the consistency of these 3 MI estimators. Under a missing at random (MAR) mechanism, CC and MP analyses were biased in most cases when estimating the marginal treatment effect, whereas MI approaches had good performance in reducing bias as long as the outcome was included in the imputation model. However, only MIte was unbiased in all the studied scenarios and Rubin's rules provided good variance estimates for MIte.
△ Less
Submitted 19 August, 2016;
originally announced August 2016.
-
Multiple imputation of covariates by fully conditional specification: accommodating the substantive model
Authors:
Jonathan W. Bartlett,
Shaun R. Seaman,
Ian R. White,
James R. Carpenter
Abstract:
Missing covariate data commonly occur in epidemiological and clinical research, and are often dealt with using multiple imputation (MI). Imputation of partially observed covariates is complicated if the substantive model is non-linear (e.g. Cox proportional hazards model), or contains non-linear (e.g. squared) or interaction terms, and standard software implementations of MI may impute covariates…
▽ More
Missing covariate data commonly occur in epidemiological and clinical research, and are often dealt with using multiple imputation (MI). Imputation of partially observed covariates is complicated if the substantive model is non-linear (e.g. Cox proportional hazards model), or contains non-linear (e.g. squared) or interaction terms, and standard software implementations of MI may impute covariates from models that are incompatible with such substantive models. We show how imputation by fully conditional specification, a popular approach for performing MI, can be modified so that covariates are imputed from models which are compatible with the substantive model. We investigate through simulation the performance of this proposal, and compare it to existing approaches. Simulation results suggest our proposal gives consistent estimates for a range of common substantive models, including models which contain non-linear covariate effects or interactions, provided data are missing at random and the assumed imputation models are correctly specified and mutually compatible.
△ Less
Submitted 17 January, 2013; v1 submitted 25 October, 2012;
originally announced October 2012.
