-
Optimal Bayesian stepped-wedge cluster randomised trial designs for binary outcome data
Authors:
Laura Etfer,
James M. S. Wason,
Michael J. Grayling
Abstract:
Under a generalised estimating equation analysis approach, approximate design theory is used to determine Bayesian D-optimal designs. For two examples, considering simple exchangeable and exponential decay correlation structures, we compare the efficiency of identified optimal designs to balanced stepped-wedge designs and corresponding stepped-wedge designs determined by optimising using a normal…
▽ More
Under a generalised estimating equation analysis approach, approximate design theory is used to determine Bayesian D-optimal designs. For two examples, considering simple exchangeable and exponential decay correlation structures, we compare the efficiency of identified optimal designs to balanced stepped-wedge designs and corresponding stepped-wedge designs determined by optimising using a normal approximation approach. The dependence of the Bayesian D-optimal designs on the assumed correlation structure is explored; for the considered settings, smaller decay in the correlation between outcomes across time periods, along with larger values of the intra-cluster correlation, leads to designs closer to a balanced design being optimal. Unlike for normal data, it is shown that the optimal design need not be centro-symmetric in the binary outcome case. The efficiency of the Bayesian D-optimal design relative to a balanced design can be large, but situations are demonstrated in which the advantages are small. Similarly, the optimal design from a normal approximation approach is often not much less efficient than the Bayesian D-optimal design. Bayesian D-optimal designs can be readily identified for stepped-wedge cluster randomised trials with binary outcome data. In certain circumstances, principally ones with strong time period effects, they will indicate that a design unlikely to have been identified by previous methods may be substantially more efficient. However, they require a larger number of assumptions than existing optimal designs, and in many situations existing theory under a normal approximation will provide an easier means of identifying an efficient design for binary outcome data.
△ Less
Submitted 15 February, 2024;
originally announced February 2024.
-
Bayesian sample size determination using robust commensurate priors with interpretable discrepancy weights
Authors:
Lou E. Whitehead,
James M. S. Wason,
Oliver Sailer,
Haiyan Zheng
Abstract:
Randomized controlled clinical trials provide the gold standard for evidence generation in relation to the efficacy of a new treatment in medical research. Relevant information from previous studies may be desirable to incorporate in the design and analysis of a new trial, with the Bayesian paradigm providing a coherent framework to formally incorporate prior knowledge. Many established methods in…
▽ More
Randomized controlled clinical trials provide the gold standard for evidence generation in relation to the efficacy of a new treatment in medical research. Relevant information from previous studies may be desirable to incorporate in the design and analysis of a new trial, with the Bayesian paradigm providing a coherent framework to formally incorporate prior knowledge. Many established methods involve the use of a discounting factor, sometimes related to a measure of `similarity' between historical and the new trials. However, it is often the case that the sample size is highly nonlinear in those discounting factors. This hinders communication with subject-matter experts to elicit sensible values for borrowing strength at the trial design stage. Focusing on a commensurate predictive prior method that can incorporate historical data from multiple sources, we highlight a particular issue of nonmonotonicity and explain why this causes issues with interpretability of the discounting factors (hereafter referred to as `weights'). We propose a solution for this, from which an analytical sample size formula is derived. We then propose a linearization technique such that the sample size changes uniformly over the weights. Our approach leads to interpretable weights that represent the probability that historical data are (ir)relevant to the new trial, and could therefore facilitate easier elicitation of expert opinion on their values.
Keywords: Bayesian sample size determination; Commensurate priors; Historical borrowing; Prior aggregation; Uniform shrinkage.
△ Less
Submitted 28 May, 2024; v1 submitted 19 January, 2024;
originally announced January 2024.
-
Evaluating the impact of outcome delay on the efficiency of two-arm group-sequential trials
Authors:
Aritra Mukherjee,
Michael J. Grayling,
James M. S. Wason
Abstract:
Adaptive designs(AD) are a broad class of trial designs that allow preplanned modifications based on patient data providing improved efficiency and flexibility. However, a delay in observing the primary outcome variable can harm this added efficiency. In this paper, we aim to ascertain the size of such outcome delay that results in the realised efficiency gains of ADs becoming negligible compared…
▽ More
Adaptive designs(AD) are a broad class of trial designs that allow preplanned modifications based on patient data providing improved efficiency and flexibility. However, a delay in observing the primary outcome variable can harm this added efficiency. In this paper, we aim to ascertain the size of such outcome delay that results in the realised efficiency gains of ADs becoming negligible compared to classical fixed sample RCTs.
We measure the impact of delay by develo** formulae for the no. of overruns in 2 arm GSDs with normal data, assuming different recruitment models. The efficiency of a GSD is usually measured in terms of the expected sample size (ESS), with GSDs generally reducing the ESS compared to a standard RCT. Our formulae measures the efficiency gain from a GSD in terms of ESS reduction that is lost due to delay. We assess whether careful choice of design (e.g., altering the spacing of the IAs) can help recover the benefits of GSDs in presence of delay. We also analyse the efficiency of GSDs with respect to time to complete the trial.
Comparing the expected efficiency gains, with and without consideration of delay, it is evident GSDs suffer considerable losses due to delay. Even a small delay can have a significant impact on the trial's efficiency. In contrast, even in the presence of substantial delay, a GSD will have a smaller expected time to trial completion in comparison to a simple RCT. Although the no. of stages have little influence on the efficiency losses, the timing of IAs can impact the efficiency of a GSDs with delay. Particularly, for unequally spaced IAs, pushing IAs towards latter end of the trial can be harmful for the design with delay.
△ Less
Submitted 7 June, 2023;
originally announced June 2023.
-
Utilising high-dimensional data in randomised clinical trials: a review of methods and practice
Authors:
Svetlana Cherlin,
Theophile Bigirumurame,
Michael J Grayling,
Jérémie Nsengimana,
Luke Ouma,
Aida Santaolalla,
Fang Wan,
S Faye Williamson,
James M S Wason
Abstract:
Introduction: Even in effectively conducted randomised trials, the probability of a successful study remains relatively low. With recent advances in the next-generation sequencing technologies, there is a rapidly growing number of high-dimensional data, including genetic, molecular and phenotypic information, that have improved our understanding of driver genes, drug targets, and drug mechanisms o…
▽ More
Introduction: Even in effectively conducted randomised trials, the probability of a successful study remains relatively low. With recent advances in the next-generation sequencing technologies, there is a rapidly growing number of high-dimensional data, including genetic, molecular and phenotypic information, that have improved our understanding of driver genes, drug targets, and drug mechanisms of action. The leveraging of high-dimensional data holds promise for increased success of clinical trials. Methods: We provide an overview of methods for utilising high-dimensional data in clinical trials. We also investigate the use of these methods in practice through a review of recently published randomised clinical trials that utilise high-dimensional genetic data. The review includes articles that were published between 2019 and 2021, identified through the PubMed database. Results: Out of 174 screened articles, 100 (57.5%) were randomised clinical trials that collected high-dimensional data. The most common clinical area was oncology (30%), followed by chronic diseases (28%), nutrition and ageing (18%) and cardiovascular diseases (7%). The most common types of data analysed were gene expression data (70%), followed by DNA data (21%). The most common method of analysis (36.3%) was univariable analysis. Articles that described multivariable analyses used standard statistical methods. Most of the clinical trials had two arms. Discussion: New methodological approaches are required for more efficient analysis of the increasing amount of high-dimensional data collected in randomised clinical trials. We highlight the limitations and barriers to the current use of high-dimensional data in trials, and suggest potential avenues for improvement and future work.
△ Less
Submitted 5 February, 2024; v1 submitted 17 May, 2023;
originally announced May 2023.
-
Online multiple hypothesis testing
Authors:
David S. Robertson,
James M. S. Wason,
Aaditya Ramdas
Abstract:
Modern data analysis frequently involves large-scale hypothesis testing, which naturally gives rise to the problem of maintaining control of a suitable type I error rate, such as the false discovery rate (FDR). In many biomedical and technological applications, an additional complexity is that hypotheses are tested in an online manner, one-by-one over time. However, traditional procedures that con…
▽ More
Modern data analysis frequently involves large-scale hypothesis testing, which naturally gives rise to the problem of maintaining control of a suitable type I error rate, such as the false discovery rate (FDR). In many biomedical and technological applications, an additional complexity is that hypotheses are tested in an online manner, one-by-one over time. However, traditional procedures that control the FDR, such as the Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. To address these challenges, a new field of methodology has developed over the past 15 years showing how to control error rates for online multiple hypothesis testing. In this framework, hypotheses arrive in a stream, and at each time point the analyst decides whether to reject the current hypothesis based both on the evidence against it, and on the previous rejection decisions. In this paper, we present a comprehensive exposition of the literature on online error rate control, with a review of key theory as well as a focus on applied examples. We also provide simulation results comparing different online testing algorithms and an up-to-date overview of the many methodological extensions that have been proposed.
△ Less
Submitted 24 July, 2023; v1 submitted 24 August, 2022;
originally announced August 2022.
-
Bayesian sample size determination in basket trials borrowing information between subsets
Authors:
Haiyan Zheng,
Michael J. Grayling,
Pavel Mozgunov,
Thomas Jaki,
James M. S. Wason
Abstract:
Basket trials are increasingly used for the simultaneous evaluation of a new treatment in various patient subgroups under one overarching protocol. We propose a Bayesian approach to sample size determination in basket trials that permit borrowing of information between commensurate subsets. Specifically, we consider a randomised basket trial design where patients are randomly assigned to the new t…
▽ More
Basket trials are increasingly used for the simultaneous evaluation of a new treatment in various patient subgroups under one overarching protocol. We propose a Bayesian approach to sample size determination in basket trials that permit borrowing of information between commensurate subsets. Specifically, we consider a randomised basket trial design where patients are randomly assigned to the new treatment or a control within each trial subset (`subtrial' for short). Closed-form sample size formulae are derived to ensure each subtrial has a specified chance of correctly deciding whether the new treatment is superior to or not better than the control by some clinically relevant difference. Given pre-specified levels of pairwise (in)commensurability, the subtrial sample sizes are solved simultaneously. The proposed Bayesian approach resembles the frequentist formulation of the problem in yielding comparable sample sizes for circumstances of no borrowing. When borrowing is enabled between commensurate subtrials, a considerably smaller trial sample size is required compared to the widely implemented approach of no borrowing. We illustrate the use of our sample size formulae with two examples based on real basket trials. A comprehensive simulation study further shows that the proposed methodology can maintain the true positive and false positive rates at desired levels.
△ Less
Submitted 24 May, 2022;
originally announced May 2022.
-
Online error control for platform trials
Authors:
David S. Robertson,
James M. S. Wason,
Franz König,
Martin Posch,
Thomas Jaki
Abstract:
Platform trials evaluate multiple experimental treatments under a single master protocol, where new treatment arms are added to the trial over time. Given the multiple treatment comparisons, there is the potential for inflation of the overall type I error rate, which is complicated by the fact that the hypotheses are tested at different times and are not all necessarily pre-specified. Online error…
▽ More
Platform trials evaluate multiple experimental treatments under a single master protocol, where new treatment arms are added to the trial over time. Given the multiple treatment comparisons, there is the potential for inflation of the overall type I error rate, which is complicated by the fact that the hypotheses are tested at different times and are not all necessarily pre-specified. Online error control methodology provides a possible solution to the problem of multiplicity for platform trials where a relatively large number of hypotheses are expected to be tested over time. In the online testing framework, hypotheses are tested in a sequential manner, where at each time-step an analyst decides whether to reject the current null hypothesis without knowledge of future tests but based solely on past decisions. Methodology has recently been developed for online control of the false discovery rate as well as the familywise error rate (FWER). In this paper, we describe how to apply online error control to the platform trial setting, present extensive simulation results, and give some recommendations for the use of this new methodology in practice. We show that the algorithms for online error rate control can have a substantially lower FWER than uncorrected testing, while still achieving noticeable gains in power when compared with the use of a Bonferroni procedure. We also illustrate how online error control would have impacted a currently ongoing platform trial.
△ Less
Submitted 8 February, 2022;
originally announced February 2022.
-
Cross-validated risk scores adaptive enrichment (CADEN) design
Authors:
Svetlana Cherlin,
James M S Wason
Abstract:
We propose a Cross-validated ADaptive ENrichment design (CADEN) in which a trial population is enriched with a subpopulation of patients who are predicted to benefit from the treatment more than an average patient (the sensitive group). This subpopulation is found using a risk score constructed from the baseline (potentially high-dimensional) information about patients. The design incorporates an…
▽ More
We propose a Cross-validated ADaptive ENrichment design (CADEN) in which a trial population is enriched with a subpopulation of patients who are predicted to benefit from the treatment more than an average patient (the sensitive group). This subpopulation is found using a risk score constructed from the baseline (potentially high-dimensional) information about patients. The design incorporates an early stop** rule for futility. Simulation studies are used to assess the properties of CADEN against the original (non-enrichment) cross-validated risk scores (CVRS) design that constructs a risk score at the end of the trial. We show that when there exists a sensitive group of patients, CADEN achieves a higher power and a reduction in the expected sample size, in comparison to the CVRS design. We illustrate the application of the design in two real clinical trials. We conclude that the new design offers improved statistical efficiency in comparison to the existing non-enrichment method, as well as increased benefit to patients. The method has been implemented in an R package caden.
△ Less
Submitted 5 February, 2024; v1 submitted 3 November, 2021;
originally announced November 2021.
-
Bayesian sample size determination using commensurate priors to leverage pre-experimental data
Authors:
Haiyan Zheng,
Thomas Jaki,
James M. S. Wason
Abstract:
This paper develops Bayesian sample size formulae for experiments comparing two groups. We assume the experimental data will be analysed in the Bayesian framework, where pre-experimental information from multiple sources can be represented into robust priors. In particular, such robust priors account for preliminary belief about the pairwise commensurability between parameters that underpin the hi…
▽ More
This paper develops Bayesian sample size formulae for experiments comparing two groups. We assume the experimental data will be analysed in the Bayesian framework, where pre-experimental information from multiple sources can be represented into robust priors. In particular, such robust priors account for preliminary belief about the pairwise commensurability between parameters that underpin the historical and new experiments, to permit flexible borrowing of information. Averaged over the probability space of the new experimental data, appropriate sample sizes are found according to criteria that control certain aspects of the posterior distribution, such as the coverage probability or length of a defined density region. Our Bayesian methodology can be applied to circumstances where the common variance in the new experiment is known or unknown. Exact solutions are available based on most of the criteria considered for Bayesian sample size determination, while a search procedure is described in cases for which there are no closed-form expressions. We illustrate the application of our Bayesian sample size formulae in the setting of designing a clinical trial. Hypothetical data examples, motivated by a rare-disease trial with elicitation of expert prior opinion, and a comprehensive performance evaluation of the proposed methodology are presented.
△ Less
Submitted 2 November, 2020;
originally announced November 2020.
-
Conditional Power and Friends: The Why and How of (Un)planned, Unblinded Sample Size Recalculations in Confirmatory Trials
Authors:
Kevin Kunzmann,
Michael J. Grayling,
Kim M. Lee,
David S. Robertson,
Kaspar Rufibach,
James M. S. Wason
Abstract:
Adapting the final sample size of a trial to the evidence accruing during the trial is a natural way to address planning uncertainty. Designs with adaptive sample size need to account for their optional stop** to guarantee strict type-I error-rate control. A variety of different methods to maintain type-I error-rate control after unplanned changes of the initial sample size have been proposed in…
▽ More
Adapting the final sample size of a trial to the evidence accruing during the trial is a natural way to address planning uncertainty. Designs with adaptive sample size need to account for their optional stop** to guarantee strict type-I error-rate control. A variety of different methods to maintain type-I error-rate control after unplanned changes of the initial sample size have been proposed in the literature. This makes interim analyses for the purpose of sample size recalculation feasible in a regulatory context. Since the sample size is usually determined via an argument based on the power of the trial, an interim analysis raises the question of how the final sample size should be determined conditional on the accrued information. Conditional power is a concept often put forward in this context. Since it depends on the unknown effect size, we take a strict estimation perspective and compare assumed conditional power, observed conditional power, and predictive power with respect to their properties as estimators of the unknown conditional power. We then demonstrate that pre-planning an interim analysis using methodology for unplanned interim analyses is ineffective and naturally leads to the concept of optimal two-stage designs. We conclude that unplanned design adaptations should only be conducted as reaction to trial-external new evidence, operational needs to violate the originally chosen design, or post hoc changes in the objective criterion. Finally, we show that commonly discussed sample size recalculation rules can lead to paradoxical outcomes and propose two alternative ways of reacting to newly emerging trial-external evidence.
△ Less
Submitted 13 October, 2020;
originally announced October 2020.
-
Develo** a predictive signature for two trial endpoints using the cross-validated risk scores method
Authors:
Svetlana Cherlin,
James M. S. Wason
Abstract:
The existing cross-validated risk scores (CVRS) design has been proposed for develo** and testing the efficacy of a treatment in a high-efficacy patient group (the sensitive group) using high-dimensional data (such as genetic data). The design is based on computing a risk score for each patient and dividing them into clusters using a non-parametric clustering procedure. In some settings it is de…
▽ More
The existing cross-validated risk scores (CVRS) design has been proposed for develo** and testing the efficacy of a treatment in a high-efficacy patient group (the sensitive group) using high-dimensional data (such as genetic data). The design is based on computing a risk score for each patient and dividing them into clusters using a non-parametric clustering procedure. In some settings it is desirable to consider the trade-off between two outcomes, such as efficacy and toxicity, or cost and effectiveness. With this motivation, we extend the CVRS design (CVRS2) to consider two outcomes. The design employs bivariate risk scores that are divided into clusters. We assess the properties of the CVRS2 using simulated data and illustrate its application on a randomised psychiatry trial. We show that CVRS2 is able to reliably identify the sensitive group (the group for which the new treatment provides benefit on both outcomes) in the simulated data. We apply the CVRS2 design to a psychology clinical trial that had offender status and substance use status as two outcomes and collected a large number of baseline covariates. The CVRS2 design yields a significant treatment effect for both outcomes, while the CVRS approach identified a significant effect for the offender status only after pre-filtering the covariates.
△ Less
Submitted 5 February, 2024; v1 submitted 3 July, 2020;
originally announced July 2020.
-
A review of Bayesian perspectives on sample size derivation for confirmatory trials
Authors:
Kevin Kunzmann,
Michael J. Grayling,
Kim May Lee,
David S. Robertson,
Kaspar Rufibach,
James M. S. Wason
Abstract:
Sample size derivation is a crucial element of the planning phase of any confirmatory trial. A sample size is typically derived based on constraints on the maximal acceptable type I error rate and a minimal desired power. Here, power depends on the unknown true effect size. In practice, power is typically calculated either for the smallest relevant effect size or a likely point alternative. The fo…
▽ More
Sample size derivation is a crucial element of the planning phase of any confirmatory trial. A sample size is typically derived based on constraints on the maximal acceptable type I error rate and a minimal desired power. Here, power depends on the unknown true effect size. In practice, power is typically calculated either for the smallest relevant effect size or a likely point alternative. The former might be problematic if the minimal relevant effect is close to the null, thus requiring an excessively large sample size. The latter is dubious since it does not account for the a priori uncertainty about the likely alternative effect size. A Bayesian perspective on the sample size derivation for a frequentist trial naturally emerges as a way of reconciling arguments about the relative a priori plausibility of alternative effect sizes with ideas based on the relevance of effect sizes. Many suggestions as to how such `hybrid' approaches could be implemented in practice have been put forward in the literature. However, key quantities such as assurance, probability of success, or expected power are often defined in subtly different ways in the literature. Starting from the traditional and entirely frequentist approach to sample size derivation, we derive consistent definitions for the most commonly used `hybrid' quantities and highlight connections, before discussing and demonstrating their use in the context of sample size derivation for clinical trials.
△ Less
Submitted 28 June, 2020;
originally announced June 2020.
-
Two-Stage Penalized Regression Screening to Detect Biomarker-Treatment Interactions in Randomized Clinical Trials
Authors:
Jixiong Wang,
Ashish Patel,
James M. S. Wason,
Paul J. Newcombe
Abstract:
High-dimensional biomarkers such as genomics are increasingly being measured in randomized clinical trials. Consequently, there is a growing interest in develo** methods that improve the power to detect biomarker-treatment interactions. We adapt recently proposed two-stage interaction detecting procedures in the setting of randomized clinical trials. We also propose a new stage 1 multivariate sc…
▽ More
High-dimensional biomarkers such as genomics are increasingly being measured in randomized clinical trials. Consequently, there is a growing interest in develo** methods that improve the power to detect biomarker-treatment interactions. We adapt recently proposed two-stage interaction detecting procedures in the setting of randomized clinical trials. We also propose a new stage 1 multivariate screening strategy using ridge regression to account for correlations among biomarkers. For this multivariate screening, we prove the asymptotic between-stage independence, required for family-wise error rate control, under biomarker-treatment independence. Simulation results show that in various scenarios, the ridge regression screening procedure can provide substantially greater power than the traditional one-biomarker-at-a-time screening procedure in highly correlated data. We also exemplify our approach in two real clinical trial data applications.
△ Less
Submitted 28 April, 2021; v1 submitted 24 April, 2020;
originally announced April 2020.
-
Graphical approaches for the control of generalised error rates
Authors:
David S. Robertson,
James M. S. Wason,
Frank Bretz
Abstract:
When simultaneously testing multiple hypotheses, the usual approach in the context of confirmatory clinical trials is to control the familywise error rate (FWER), which bounds the probability of making at least one false rejection. In many trial settings, these hypotheses will additionally have a hierarchical structure that reflects the relative importance and links between different clinical obje…
▽ More
When simultaneously testing multiple hypotheses, the usual approach in the context of confirmatory clinical trials is to control the familywise error rate (FWER), which bounds the probability of making at least one false rejection. In many trial settings, these hypotheses will additionally have a hierarchical structure that reflects the relative importance and links between different clinical objectives. The graphical approach of Bretz et al. (2009) is a flexible and easily communicable way of controlling the FWER while respecting complex trial objectives and multiple structured hypotheses. However, the FWER can be a very stringent criterion that leads to procedures with low power, and may not be appropriate in exploratory trial settings. This motivates controlling generalised error rates, particularly when the number of hypotheses tested is no longer small. We consider the generalised familywise error rate (k-FWER), which is the probability of making k or more false rejections, as well as the tail probability of the false discovery proportion (FDP), which is the probability that the proportion of false rejections is greater than some threshold. We also consider asymptotic control of the false discovery rate (FDR), which is the expectation of the FDP. In this paper, we show how to control these generalised error rates when using the graphical approach and its extensions. We demonstrate the utility of the resulting graphical procedures on three clinical trial case studies.
△ Less
Submitted 2 June, 2020; v1 submitted 3 April, 2020;
originally announced April 2020.
-
Sample Size Estimation using a Latent Variable Model for Mixed Outcome Co-Primary, Multiple Primary and Composite Endpoints
Authors:
Martina McMenamin,
Jessica K. Barrett,
Anna Berglind,
James M. S. Wason
Abstract:
Mixed outcome endpoints that combine multiple continuous and discrete components to form co-primary, multiple primary or composite endpoints are often employed as primary outcome measures in clinical trials. There are many advantages to joint modelling the individual outcomes using a latent variable framework, however in order to make use of the model in practice we require techniques for sample s…
▽ More
Mixed outcome endpoints that combine multiple continuous and discrete components to form co-primary, multiple primary or composite endpoints are often employed as primary outcome measures in clinical trials. There are many advantages to joint modelling the individual outcomes using a latent variable framework, however in order to make use of the model in practice we require techniques for sample size estimation. In this paper we show how the latent variable model can be applied to the three types of joint endpoints and propose appropriate hypotheses, power and sample size estimation methods for each. We illustrate the techniques using a numerical example based on the four dimensional endpoint in the MUSE trial and find that the sample size required for the co-primary endpoint is larger than that required for the individual endpoint with the smallest effect size. Conversely, the sample size required for the multiple primary endpoint is reduced from that required for the individual outcome with the largest effect size. We show that the analytical technique agrees with the empirical power from simulation studies. We further illustrate the reduction in required sample size that may be achieved in trials of mixed outcome composite endpoints through a simulation study and find that the sample size primarily depends on the components driving response and the correlation structure and much less so on the treatment effect structure in the individual endpoints.
△ Less
Submitted 11 December, 2019;
originally announced December 2019.
-
Bayesian design and analysis of external pilot trials for complex interventions
Authors:
Duncan T. Wilson,
James M. S. Wason,
Julia Brown,
Amanda J. Farrin,
Rebecca E. A. Walwyn
Abstract:
External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre-specified thresholds known as progression criteria, although the statistical prop…
▽ More
External pilot trials of complex interventions are used to help determine if and how a confirmatory trial should be undertaken, providing estimates of parameters such as recruitment, retention and adherence rates. The decision to progress to the confirmatory trial is typically made by comparing these estimates to pre-specified thresholds known as progression criteria, although the statistical properties of such decision rules are rarely assessed. Such assessment is complicated by several methodological challenges, including the simultaneous evaluation of multiple endpoints, complex multi-level models, small sample sizes, and uncertainty in nuisance parameters. In response to these challenges, we describe a Bayesian approach to the design and analysis of external pilot trials. We show how progression decisions can be made by minimising the expected value of a loss function, defined over the whole parameter space to allow for preferences and trade-offs between multiple parameters to be articulated and used in the decision making process. The assessment of preferences is kept feasible by using a piecewise constant parameterisation of the loss function, the parameters of which are chosen at the design stage to lead to desirable operating characteristics. We describe a flexible, yet computationally intensive, nested Monte Carlo algorithm for estimating operating characteristics. The method is used to revisit the design of an external pilot trial of a complex intervention designed to increase the physical activity of care home residents.
△ Less
Submitted 20 May, 2020; v1 submitted 16 August, 2019;
originally announced August 2019.
-
Borrowing of information across patient subgroups in a basket trial based on distributional discrepancy
Authors:
Haiyan Zheng,
James M. S. Wason
Abstract:
Basket trials have emerged as a new class of efficient approaches in oncology to evaluate a new treatment in several patient subgroups simultaneously. In this paper, we extend the key ideas to disease areas outside of oncology, develo** a robust Bayesian methodology for randomised, placebo-controlled basket trials with a continuous endpoint to enable borrowing of information across subtrials wit…
▽ More
Basket trials have emerged as a new class of efficient approaches in oncology to evaluate a new treatment in several patient subgroups simultaneously. In this paper, we extend the key ideas to disease areas outside of oncology, develo** a robust Bayesian methodology for randomised, placebo-controlled basket trials with a continuous endpoint to enable borrowing of information across subtrials with similar treatment effects. After adjusting for covariates, information from a complementary subtrial can be represented into a commensurate prior for the parameter that underpins the subtrial under consideration. We propose using distributional discrepancy to characterise the commensurability between subtrials for appropriate borrowing of information through a spike-and-slab prior, which is placed on the prior precision factor. When the basket trial has at least three subtrials, commensurate priors for point-to-point borrowing are combined into a marginal predictive prior, according to the weights transformed from the pairwise discrepancy measures. In this way, only information from subtrial(s) with the most commensurate treatment effect is leveraged. The marginal predictive prior is updated to a robust posterior by the contemporary subtrial data to inform decision making. Operating characteristics of the proposed methodology are evaluated through simulations motivated by a real basket trial in chronic diseases. The proposed methodology has advantages compared to other selected Bayesian analysis models, for (i) identifying the most commensurate source of information, and (ii) gauging the degree of borrowing from specific subtrials. Numerical results also suggest that our methodology can improve the precision of estimates and, potentially, the statistical power for hypothesis testing.
△ Less
Submitted 8 May, 2020; v1 submitted 14 August, 2019;
originally announced August 2019.
-
Employing latent variable models to improve efficiency in composite endpoint analysis
Authors:
Martina McMenamin,
Jessica K. Barrett,
Anna Berglind,
James M. S. Wason
Abstract:
Composite endpoints that combine multiple outcomes on different scales are common in clinical trials, particularly in chronic conditions. In many of these cases, patients will have to cross a predefined responder threshold in each of the outcomes to be classed as a responder overall. One instance of this occurs in systemic lupus erythematosus (SLE), where the responder endpoint combines two contin…
▽ More
Composite endpoints that combine multiple outcomes on different scales are common in clinical trials, particularly in chronic conditions. In many of these cases, patients will have to cross a predefined responder threshold in each of the outcomes to be classed as a responder overall. One instance of this occurs in systemic lupus erythematosus (SLE), where the responder endpoint combines two continuous, one ordinal and one binary measure. The overall binary responder endpoint is typically analysed using logistic regression, resulting in a substantial loss of information. We propose a latent variable model for the SLE endpoint, which assumes that the discrete outcomes are manifestations of latent continuous measures and can proceed to jointly model the components of the composite. We perform a simulation study and find the method to offer large efficiency gains over the standard analysis. We find that the magnitude of the precision gains are highly dependent on which components are driving response. Bias is introduced when joint normality assumptions are not satisfied, which we correct for using a bootstrap procedure. The method is applied to the Phase IIb MUSE trial in patients with moderate to severe SLE. We show that it estimates the treatment effect 2.5 times more precisely, offering a 60% reduction in required sample size.
△ Less
Submitted 19 February, 2019;
originally announced February 2019.
-
Online control of the false discovery rate in biomedical research
Authors:
David S. Robertson,
James M. S. Wason
Abstract:
Modern biomedical research frequently involves testing multiple related hypotheses, while maintaining control over a suitable error rate. In many applications the false discovery rate (FDR), which is the expected proportion of false positives among the rejected hypotheses, has become the standard error criterion. Procedures that control the FDR, such as the well-known Benjamini-Hochberg procedure,…
▽ More
Modern biomedical research frequently involves testing multiple related hypotheses, while maintaining control over a suitable error rate. In many applications the false discovery rate (FDR), which is the expected proportion of false positives among the rejected hypotheses, has become the standard error criterion. Procedures that control the FDR, such as the well-known Benjamini-Hochberg procedure, assume that all p-values are available to be tested at a single time point. However, this ignores the sequential nature of many biomedical experiments, where a sequence of hypotheses is tested without having access to future p-values or even the number of hypotheses. Recently, the first procedures that control the FDR in this online manner have been proposed by Javanmard and Montanari (Ann. Stat. 2018), and built upon by Ramdas et al. (NIPS 2017, ICML 2018). In this paper, we compare and contrast these proposed procedures, with a particular focus on the setting where the p-values are dependent. We also propose a simple modification of the procedures for when there is an upper bound on the number of hypotheses to be tested. Using comprehensive simulation scenarios and case studies, we provide recommendations for which procedures to use in practice for online FDR control.
△ Less
Submitted 26 September, 2018; v1 submitted 19 September, 2018;
originally announced September 2018.
-
Familywise error control in multi-armed response-adaptive trials
Authors:
David S. Robertson,
James M. S. Wason
Abstract:
Response-adaptive designs allow the randomization probabilities to change during the course of a trial based on cumulated response data, so that a greater proportion of patients can be allocated to the better performing treatments. A major concern over the use of response-adaptive designs in practice, particularly from a regulatory viewpoint, is controlling the type I error rate. In particular, we…
▽ More
Response-adaptive designs allow the randomization probabilities to change during the course of a trial based on cumulated response data, so that a greater proportion of patients can be allocated to the better performing treatments. A major concern over the use of response-adaptive designs in practice, particularly from a regulatory viewpoint, is controlling the type I error rate. In particular, we show that the naive z-test can have an inflated type I error rate even after applying a Bonferroni correction. Simulation studies have often been used to demonstrate error control, but do not provide a guarantee. In this paper, we present adaptive testing procedures for normally distributed outcomes that ensure strong familywise error control, by iteratively applying the conditional invariance principle. Our approach can be used for fully sequential and block randomized trials, and for a large class of adaptive randomization rules found in the literature. We show there is a high price to pay in terms of power to guarantee familywise error control for randomization schemes with extreme allocation probabilities. However, for proposed Bayesian adaptive randomization schemes in the literature, our adaptive tests maintain or increase the power of the trial compared to the z-test. We illustrate our method using a three-armed trial in primary hypercholesterolemia.
△ Less
Submitted 14 March, 2018;
originally announced March 2018.
