-
Finite-sample adjustments for comparing clustered adaptive interventions using data from a clustered SMART
Authors:
Wenchu Pan,
Daniel Almirall,
Amy M. Kilbourne,
Andrew Quanbeck,
Lu Wang
Abstract:
Adaptive interventions, aka dynamic treatment regimens, are sequences of pre-specified decision rules that guide the provision of treatment for an individual given information about their baseline and evolving needs, including in response to prior intervention. Clustered adaptive interventions (cAIs) extend this idea by guiding the provision of intervention at the level of clusters (e.g., clinics)…
▽ More
Adaptive interventions, aka dynamic treatment regimens, are sequences of pre-specified decision rules that guide the provision of treatment for an individual given information about their baseline and evolving needs, including in response to prior intervention. Clustered adaptive interventions (cAIs) extend this idea by guiding the provision of intervention at the level of clusters (e.g., clinics), but with the goal of improving outcomes at the level of individuals within the cluster (e.g., clinicians or patients within clinics). A clustered, sequential multiple-assignment randomized trials (cSMARTs) is a multistage, multilevel randomized trial design used to construct high-quality cAIs. In a cSMART, clusters are randomized at multiple intervention decision points; at each decision point, the randomization probability can depend on response to prior data. A challenge in cluster-randomized trials, including cSMARTs, is the deleterious effect of small samples of clusters on statistical inference, particularly via estimation of standard errors. \par This manuscript develops finite-sample adjustment (FSA) methods for making improved statistical inference about the causal effects of cAIs in a cSMART. The paper develops FSA methods that (i) scale variance estimators using a degree-of-freedom adjustment, (ii) reference a t distribution (instead of a normal), and (iii) employ a ``bias corrected" variance estimator. Method (iii) requires extensions that are unique to the analysis of cSMARTs. Extensive simulation experiments are used to test the performance of the methods. The methods are illustrated using the Adaptive School-based Implementation of CBT (ASIC) study, a cSMART designed to construct a cAI for improving the delivery of cognitive behavioral therapy (CBT) by school mental health professionals within high schools in Michigan.
△ Less
Submitted 30 April, 2024;
originally announced May 2024.
-
Approaches to Statistical Efficiency when comparing the embedded adaptive interventions in a SMART
Authors:
Timothy Lycurgus,
Amy Kilbourne,
Daniel Almirall
Abstract:
Sequential, multiple assignment randomized trials (SMARTs), which assist in the optimization of adaptive interventions, are growing in popularity in education and behavioral sciences. This is unsurprising, as adaptive interventions reflect the sequential, tailored nature of learning in a classroom or school. Nonetheless, as is true elsewhere in education research, observed effect sizes in educatio…
▽ More
Sequential, multiple assignment randomized trials (SMARTs), which assist in the optimization of adaptive interventions, are growing in popularity in education and behavioral sciences. This is unsurprising, as adaptive interventions reflect the sequential, tailored nature of learning in a classroom or school. Nonetheless, as is true elsewhere in education research, observed effect sizes in education-based SMARTs are frequently small. As a consequence, statistical efficiency is of paramount importance in their analysis. The contributions of this manuscript are two-fold. First, we provide an overview of adaptive interventions and SMART designs for researchers in education science. Second, we propose four techniques that have the potential to improve statistical efficiency in the analysis of SMARTs. We demonstrate the benefits of these techniques in SMART settings both through the analysis of a SMART designed to optimize an adaptive intervention for increasing cognitive behavioral therapy delivery in school settings and through a comprehensive simulation study. Each of the proposed techniques is easily implementable, either with over-the-counter statistical software or through R code provided in an online supplement.
△ Less
Submitted 22 September, 2023; v1 submitted 19 April, 2023;
originally announced April 2023.
-
SMART Binary: Sample Size Calculation for Comparing Adaptive Interventions in SMART studies with Longitudinal Binary Outcomes
Authors:
John J. Dziak,
Daniel Almirall,
Walter Dempsey,
Catherine Stanger,
Inbal Nahum-Shani
Abstract:
Sequential Multiple-Assignment Randomized Trials (SMARTs) play an increasingly important role in psychological and behavioral health research. This experimental approach enables researchers to answer scientific questions about how to sequence and match interventions to the unique, changing needs of individuals. A variety of sample size planning resources for SMART studies have been developed in re…
▽ More
Sequential Multiple-Assignment Randomized Trials (SMARTs) play an increasingly important role in psychological and behavioral health research. This experimental approach enables researchers to answer scientific questions about how to sequence and match interventions to the unique, changing needs of individuals. A variety of sample size planning resources for SMART studies have been developed in recent years; these enable researchers to plan SMARTs for addressing different types of scientific questions. However, relatively limited attention has been given to planning SMARTs with binary (dichotomous) outcomes, which often require higher sample sizes relative to continuous outcomes. Existing resources for estimating sample size requirements for SMARTs with binary outcomes do not consider the potential to improve power by including a baseline measurement and/or multiple repeated outcome measurements. The current paper addresses this issue by providing sample size simulation code and approximate formulas for two-wave repeated measures binary outcomes (i.e., two measurement times for the outcome variable, before and after receiving the intervention). The simulation results agree well with the formulas. We also discuss how to use simulations to calculate power for studies with more than two outcome measurement occasions. The results show that having at least one repeated measurement of the outcome can substantially improve power under certain conditions.
△ Less
Submitted 19 June, 2023; v1 submitted 11 October, 2021;
originally announced October 2021.
-
Linear Mixed Models for Comparing Dynamic Treatment Regimens on a Longitudinal Outcome in Sequentially Randomized Trials
Authors:
Brook Luers,
Min Qian,
Inbal Nahum-Shani,
Connie Kasari,
Daniel Almirall
Abstract:
A dynamic treatment regimen (DTR) is a pre-specified sequence of decision rules which maps baseline or time-varying measurements on an individual to a recommended intervention or set of interventions. Sequential multiple assignment randomized trials (SMARTs) represent an important data collection tool for informing the construction of effective DTRs. A common primary aim in a SMART is the marginal…
▽ More
A dynamic treatment regimen (DTR) is a pre-specified sequence of decision rules which maps baseline or time-varying measurements on an individual to a recommended intervention or set of interventions. Sequential multiple assignment randomized trials (SMARTs) represent an important data collection tool for informing the construction of effective DTRs. A common primary aim in a SMART is the marginal mean comparison between two or more of the DTRs embedded in the trial. This manuscript develops a mixed effects modeling and estimation approach for these primary aim comparisons based on a continuous, longitudinal outcome. The method is illustrated using data from a SMART in autism research.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
Estimating Time-Varying Causal Excursion Effect in Mobile Health with Binary Outcomes
Authors:
Tianchen Qian,
Hyesun Yoo,
Predrag Klasnja,
Daniel Almirall,
Susan A. Murphy
Abstract:
Advances in wearables and digital technology now make it possible to deliver behavioral mobile health interventions to individuals in their everyday life. The micro-randomized trial (MRT) is increasingly used to provide data to inform the construction of these interventions. In an MRT, each individual is repeatedly randomized among multiple intervention options, often hundreds or even thousands of…
▽ More
Advances in wearables and digital technology now make it possible to deliver behavioral mobile health interventions to individuals in their everyday life. The micro-randomized trial (MRT) is increasingly used to provide data to inform the construction of these interventions. In an MRT, each individual is repeatedly randomized among multiple intervention options, often hundreds or even thousands of times, over the course of the trial. This work is motivated by multiple MRTs that have been conducted, or are currently in the field, in which the primary outcome is a longitudinal binary outcome. The primary aim of such MRTs is to examine whether a particular time-varying intervention has an effect on the longitudinal binary outcome, often marginally over all but a small subset of the individual's data. We propose the definition of causal excursion effect that can be used in such primary aim analysis for MRTs with binary outcomes. Under rather restrictive assumptions one can, based on existing literature, derive a semiparametric, locally efficient estimator of the causal effect. We, starting from this estimator, develop an estimator that can be used as the basis of a primary aim analysis under more plausible assumptions. Simulation studies are conducted to compare the estimators. We illustrate the developed methods using data from the MRT, BariFit. In BariFit, the goal is to support weight maintenance for individuals who received bariatric surgery.
△ Less
Submitted 29 July, 2020; v1 submitted 2 June, 2019;
originally announced June 2019.
-
Sample size considerations for comparing dynamic treatment regimens in a sequential multiple-assignment randomized trial with a continuous longitudinal outcome
Authors:
Nicholas J. Seewald,
Kelley M. Kidwell,
Inbal Nahum-Shani,
Tianshuang Wu,
James R. McKay,
Daniel Almirall
Abstract:
Clinicians and researchers alike are increasingly interested in how best to personalize interventions. A dynamic treatment regimen (DTR) is a sequence of pre-specified decision rules which can be used to guide the delivery of a sequence of treatments or interventions that are tailored to the changing needs of the individual. The sequential multiple-assignment randomized trial (SMART) is a research…
▽ More
Clinicians and researchers alike are increasingly interested in how best to personalize interventions. A dynamic treatment regimen (DTR) is a sequence of pre-specified decision rules which can be used to guide the delivery of a sequence of treatments or interventions that are tailored to the changing needs of the individual. The sequential multiple-assignment randomized trial (SMART) is a research tool which allows for the construction of effective DTRs. We derive easy-to-use formulae for computing the total sample size for three common two-stage SMART designs in which the primary aim is to compare mean end-of-study outcomes for two embedded DTRs which recommend different first-stage treatments. The formulae are derived in the context of a regression model which leverages information from a longitudinal outcome collected over the entire study. We show that the sample size formula for a SMART can be written as the product of the sample size formula for a standard two-arm randomized trial, a deflation factor that accounts for the increased statistical efficiency resulting from a longitudinal analysis, and an inflation factor that accounts for the design of a SMART. The SMART design inflation factor is typically a function of the anticipated probability of response to first-stage treatment. We review modeling and estimation for DTR effect analyses using a longitudinal outcome from a SMART, as well as the estimation of standard errors. We also present estimators for the covariance matrix for a variety of common working correlation structures. Methods are motivated using the ENGAGE study, a SMART aimed at develo** a DTR for increasing motivation to attend treatments among alcohol- and cocaine-dependent patients.
△ Less
Submitted 5 January, 2020; v1 submitted 30 October, 2018;
originally announced October 2018.
-
Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: Regression estimation and sample size considerations
Authors:
Timothy NeCamp,
Amy Kilbourne,
Daniel Almirall
Abstract:
Cluster-level dynamic treatment regimens can be used to guide sequential, intervention or treatment decision-making at the cluster level in order to improve outcomes at the individual or patient-level. In a cluster-level DTR, the intervention or treatment is potentially adapted and re-adapted over time based on changes in the cluster that could be impacted by prior intervention, including based on…
▽ More
Cluster-level dynamic treatment regimens can be used to guide sequential, intervention or treatment decision-making at the cluster level in order to improve outcomes at the individual or patient-level. In a cluster-level DTR, the intervention or treatment is potentially adapted and re-adapted over time based on changes in the cluster that could be impacted by prior intervention, including based on aggregate measures of the individuals or patients that comprise it. Cluster-randomized sequential multiple assignment randomized trials (SMARTs) can be used to answer multiple open questions preventing scientists from develo** high-quality cluster-level DTRs. In a cluster-randomized SMART, sequential randomizations occur at the cluster level and outcomes are at the individual level. This manuscript makes two contributions to the design and analysis of cluster-randomized SMARTs: First, a weighted least squares regression approach is proposed for comparing the mean of a patient-level outcome between the cluster-level DTRs embedded in a SMART. The regression approach facilitates the use of baseline covariates which is often critical in the analysis of cluster-level trials. Second, sample size calculators are derived for two common cluster-randomized SMART designs for use when the primary aim is a between-DTR comparison of the mean of a continuous patient-level outcome. The methods are motivated by the Adaptive Implementation of Effective Programs Trial, which is, to our knowledge, the first-ever cluster-randomized SMART in psychiatry.
△ Less
Submitted 14 July, 2016;
originally announced July 2016.
-
Assessing Time-Varying Causal Effect Moderation in Mobile Health
Authors:
Audrey Boruvka,
Daniel Almirall,
Katie Witkiewitz,
Susan A. Murphy
Abstract:
In mobile health interventions aimed at behavior change and maintenance, treatments are provided in real time to manage current or impending high risk situations or promote healthy behaviors in near real time. Currently there is great scientific interest in develo** data analysis approaches to guide the development of mobile interventions. In particular data from mobile health studies might be u…
▽ More
In mobile health interventions aimed at behavior change and maintenance, treatments are provided in real time to manage current or impending high risk situations or promote healthy behaviors in near real time. Currently there is great scientific interest in develo** data analysis approaches to guide the development of mobile interventions. In particular data from mobile health studies might be used to examine effect moderators-i.e., individual characteristics, time-varying context or past treatment response that moderate the effect of current treatment on a subsequent response. This paper introduces a formal definition for moderated effects in terms of potential outcomes, a definition that is particularly suited to mobile interventions, where treatment occasions are numerous, individuals are not always available for treatment, and potential moderators might be influenced by past treatment. Methods for estimating moderated effects are developed and compared. The proposed approach is illustrated using BASICS-Mobile, a smartphone-based intervention designed to curb heavy drinking and smoking among college students.
△ Less
Submitted 16 August, 2016; v1 submitted 2 January, 2016;
originally announced January 2016.
