Showing 1–50 of 66 results for author: Friede, T

A Review of EMA Public Assessment Reports where Non-Proportional Hazards were Identified

Authors: Florian Klinglmueller, Norbert Benda, Tim Friede, Tobias Fellinger, Harald Heinzl, Andrew Hooker, Franz Koenig, Tim Mathes, Martin Posch, Florian Stampfer, Susanne Urach

Abstract: While well-established methods for time-to-event data are available when the proportional hazards assumption holds, there is no consensus on the best approach under non-proportional hazards. A wide range of parametric and non-parametric methods for testing and estimation in this scenario have been proposed. In this review we identified EMA marketing authorization procedures where non-proportional… ▽ More While well-established methods for time-to-event data are available when the proportional hazards assumption holds, there is no consensus on the best approach under non-proportional hazards. A wide range of parametric and non-parametric methods for testing and estimation in this scenario have been proposed. In this review we identified EMA marketing authorization procedures where non-proportional hazards were raised as a potential issue in the risk-benefit assessment and extract relevant information on trial design and results reported in the corresponding European Assessment Reports (EPARs) available in the database at paediatricdata.eu. We identified 16 Marketing authorization procedures, reporting results on a total of 18 trials. Most procedures covered the authorization of treatments from the oncology domain. For the majority of trials NPH issues were related to a suspected delayed treatment effect, or different treatment effects in known subgroups. Issues related to censoring, or treatment switching were also identified. For most of the trials the primary analysis was performed using conventional methods assuming proportional hazards, even if NPH was anticipated. Differential treatment effects were addressed using stratification and delayed treatment effect considered for sample size planning. Even though, not considered in the primary analysis, some procedures reported extensive sensitivity analyses and model diagnostics evaluating the proportional hazards assumption. For a few procedures methods addressing NPH (e.g.~weighted log-rank tests) were used in the primary analysis. We extracted estimates of the median survival, hazard ratios, and time of survival curve separation. In addition, we digitized the KM curves to reconstruct close to individual patient level data. Extracted outcomes served as the basis for a simulation study of methods for time to event analysis under NPH. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 16 pages, 5 figures

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): summary of findings and a roadmap for the future of safety analyses in clinical trials

Authors: Kaspar Rufibach, Jan Beyersmann, Tim Friede, Claudia Schmoor, Regina Stegherr

Abstract: The SAVVY project aims to improve the analyses of adverse events (AEs) in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). This paper summarizes key features and conclusions from the various SAVVY papers. Through theoretical investigations using simulations and in an empirical study including randomized clinical t… ▽ More The SAVVY project aims to improve the analyses of adverse events (AEs) in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). This paper summarizes key features and conclusions from the various SAVVY papers. Through theoretical investigations using simulations and in an empirical study including randomized clinical trials from several sponsor organisations, biases from ignoring varying follow-up times or CEs are investigated. The bias of commonly used estimators of the absolute and relative AE risk is quantified. Furthermore, we provide a cursory assessment of how pertinent guidelines for the analysis of safety data deal with the features of varying follow-up time and CEs. SAVVY finds that for both, avoiding bias and categorization of evidence with respect to treatment effect on AE risk into categories, the choice of the estimator is key and more important than features of the underlying data such as percentage of censoring, CEs, amount of follow-up, or value of the gold-standard. The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. SAVVY recommends using the Aalen-Johansen estimator (AJE) with an appropriate definition of CEs whenever the risk for AEs is to be quantified. There is an urgent need to improve the guidelines of reporting AEs so that incidence proportions or one minus Kaplan-Meier estimators are finally replaced by the AJE with appropriate definition of CEs. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Comments: 17 pages, 1 Figure, 4 Tables. arXiv admin note: text overlap with arXiv:2008.07881

A two-step approach for analyzing time to event data under non-proportional hazards

Authors: Jonas Brugger, Tim Friede, Florian Klinglmüller, Martin Posch, Robin Ristl, Franz König

Abstract: The log-rank test and the Cox proportional hazards model are commonly used to compare time-to-event data in clinical trials, as they are most powerful under proportional hazards. But there is a loss of power if this assumption is violated, which is the case for some new oncology drugs like immunotherapies. We consider a two-stage test procedure, in which the weighting of the log-rank test statisti… ▽ More The log-rank test and the Cox proportional hazards model are commonly used to compare time-to-event data in clinical trials, as they are most powerful under proportional hazards. But there is a loss of power if this assumption is violated, which is the case for some new oncology drugs like immunotherapies. We consider a two-stage test procedure, in which the weighting of the log-rank test statistic depends on a pre-test of the proportional hazards assumption. I.e., depending on the pre-test either the log-rank or an alternative test is used to compare the survival probabilities. We show that if naively implemented this can lead to a substantial inflation of the type-I error rate. To address this, we embed the two-stage test in a permutation test framework to keep the nominal level alpha. We compare the operating characteristics of the two-stage test with the log-rank test and other tests by clinical trial simulations. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Investigating the heterogeneity of "study twins"

Authors: Christian Röver, Tim Friede

Abstract: Meta-analyses are commonly performed based on random-effects models, while in certain cases one might also argue in favour of a common-effect model. One such case may be given by the example of two "study twins" that are performed according to a common (or at least very similar) protocol. Here we investigate the particular case of meta-analysis of a pair of studies, e.g. summarizing the results of… ▽ More Meta-analyses are commonly performed based on random-effects models, while in certain cases one might also argue in favour of a common-effect model. One such case may be given by the example of two "study twins" that are performed according to a common (or at least very similar) protocol. Here we investigate the particular case of meta-analysis of a pair of studies, e.g. summarizing the results of two confirmatory clinical trials in phase III of a clinical development programme. Thereby, we focus on the question of to what extent homogeneity or heterogeneity may be discernible, and include an empirical investigation of published ("twin") pairs of studies. A pair of estimates from two studies only provides very little evidence on homogeneity or heterogeneity of effects, and ad-hoc decision criteria may often be misleading. △ Less

Submitted 6 May, 2024; v1 submitted 15 December, 2023; originally announced December 2023.

Comments: 18 pages, 5 figures

Subgroup identification using individual participant data from multiple trials on low back pain

Authors: Cynthia Huber, Tim Friede

Abstract: Model-based recursive partitioning (MOB) and its extension, metaMOB, are potent tools for identifying subgroups with differential treatment effects. In the metaMOB approach random effects are used to model heterogeneity of the treatment effects when pooling data from various trials. In situations where interventions offer only small overall benefits and require extensive, costly trials with a larg… ▽ More Model-based recursive partitioning (MOB) and its extension, metaMOB, are potent tools for identifying subgroups with differential treatment effects. In the metaMOB approach random effects are used to model heterogeneity of the treatment effects when pooling data from various trials. In situations where interventions offer only small overall benefits and require extensive, costly trials with a large participant enrollment, leveraging individual-participant data (IPD) from multiple trials can help identify individuals who are most likely to benefit from the intervention. We explore the application of MOB and metaMOB in the context of non specific low back pain treatment, using synthesized data based on a subset of the individual participant data meta-analysis by Patel et al. Our study underscores the need to explore heterogeneity in intercepts and treatment effects to identify subgroups with differential treatment effects in IPD meta-analyses. △ Less

Submitted 2 November, 2023; originally announced November 2023.

A neutral comparison of statistical methods for time-to-event analyses under non-proportional hazards

Authors: Florian Klinglmüller, Tobias Fellinger, Franz König, Tim Friede, Andrew C. Hooker, Harald Heinzl, Martina Mittlböck, Jonas Brugger, Maximilian Bardo, Cynthia Huber, Norbert Benda, Martin Posch, Robin Ristl

Abstract: While well-established methods for time-to-event data are available when the proportional hazards assumption holds, there is no consensus on the best inferential approach under non-proportional hazards (NPH). However, a wide range of parametric and non-parametric methods for testing and estimation in this scenario have been proposed. To provide recommendations on the statistical analysis of clinic… ▽ More While well-established methods for time-to-event data are available when the proportional hazards assumption holds, there is no consensus on the best inferential approach under non-proportional hazards (NPH). However, a wide range of parametric and non-parametric methods for testing and estimation in this scenario have been proposed. To provide recommendations on the statistical analysis of clinical trials where non proportional hazards are expected, we conducted a comprehensive simulation study under different scenarios of non-proportional hazards, including delayed onset of treatment effect, crossing hazard curves, subgroups with different treatment effect and changing hazards after disease progression. We assessed type I error rate control, power and confidence interval coverage, where applicable, for a wide range of methods including weighted log-rank tests, the MaxCombo test, summary measures such as the restricted mean survival time (RMST), average hazard ratios, and milestone survival probabilities as well as accelerated failure time regression models. We found a trade-off between interpretability and power when choosing an analysis strategy under NPH scenarios. While analysis methods based on weighted logrank tests typically were favorable in terms of power, they do not provide an easily interpretable treatment effect estimate. Also, depending on the weight function, they test a narrow null hypothesis of equal hazard functions and rejection of this null hypothesis may not allow for a direct conclusion of treatment benefit in terms of the survival function. In contrast, non-parametric procedures based on well interpretable measures as the RMST difference had lower power in most scenarios. Model based methods based on specific survival distributions had larger power, however often gave biased estimates and lower than nominal confidence interval coverage. △ Less

Submitted 9 October, 2023; originally announced October 2023.

How trace plots help interpret meta-analysis results

Authors: Christian Röver, David Rindskopf, Tim Friede

Abstract: The trace plot is seldom used in meta-analysis, yet it is a very informative plot. In this article we define and illustrate what the trace plot is, and discuss why it is important. The Bayesian version of the plot combines the posterior density of tau, the between-study standard deviation, and the shrunken estimates of the study effects as a function of tau. With a small or moderate number of stud… ▽ More The trace plot is seldom used in meta-analysis, yet it is a very informative plot. In this article we define and illustrate what the trace plot is, and discuss why it is important. The Bayesian version of the plot combines the posterior density of tau, the between-study standard deviation, and the shrunken estimates of the study effects as a function of tau. With a small or moderate number of studies, tau is not estimated with much precision, and parameter estimates and shrunken study effect estimates can vary widely depending on the correct value of tau. The trace plot allows visualization of the sensitivity to tau along with a plot that shows which values of tau are plausible and which are implausible. A comparable frequentist or empirical Bayes version provides similar results. The concepts are illustrated using examples in meta-analysis and meta-regression; implementaton in R is facilitated in a Bayesian or frequentist framework using the bayesmeta and metafor packages, respectively. △ Less

Submitted 30 November, 2023; v1 submitted 29 June, 2023; originally announced June 2023.

Comments: 14 pages, 18 Figures

Methods for non-proportional hazards in clinical trials: A systematic review

Authors: Maximilian Bardo, Cynthia Huber, Norbert Benda, Jonas Brugger, Tobias Fellinger, Vaidotas Galaune, Judith Heinz, Harald Heinzl, Andrew C. Hooker, Florian Klinglmüller, Franz König, Tim Mathes, Martina Mittlböck, Martin Posch, Robin Ristl, Tim Friede

Abstract: For the analysis of time-to-event data, frequently used methods such as the log-rank test or the Cox proportional hazards model are based on the proportional hazards assumption, which is often debatable. Although a wide range of parametric and non-parametric methods for non-proportional hazards (NPH) has been proposed, there is no consensus on the best approaches. To close this gap, we conducted a… ▽ More For the analysis of time-to-event data, frequently used methods such as the log-rank test or the Cox proportional hazards model are based on the proportional hazards assumption, which is often debatable. Although a wide range of parametric and non-parametric methods for non-proportional hazards (NPH) has been proposed, there is no consensus on the best approaches. To close this gap, we conducted a systematic literature search to identify statistical methods and software appropriate under NPH. Our literature search identified 907 abstracts, out of which we included 211 articles, mostly methodological ones. Review articles and applications were less frequently identified. The articles discuss effect measures, effect estimation and regression approaches, hypothesis tests, and sample size calculation approaches, which are often tailored to specific NPH situations. Using a unified notation, we provide an overview of methods available. Furthermore, we derive some guidance from the identified articles. We summarized the contents from the literature review in a concise way in the main text and provide more detailed explanations in the supplement. △ Less

Submitted 29 January, 2024; v1 submitted 29 June, 2023; originally announced June 2023.

MSC Class: 62Nxx; 62Pxx

The impact of neglected confounding and interactions in mixed-effects meta-regression

Authors: Eric S. Knop, Markus Pauly, Tim Friede, Thilo Welz

Abstract: Analysts seldom include interaction terms in meta-regression model, what can introduce bias if an interaction is present. We illustrate this in the current paper by re-analyzing an example from research on acute heart failure, where neglecting an interaction might have led to erroneous inference and conclusions. Moreover, we perform a brief simulation study based on this example highlighting the e… ▽ More Analysts seldom include interaction terms in meta-regression model, what can introduce bias if an interaction is present. We illustrate this in the current paper by re-analyzing an example from research on acute heart failure, where neglecting an interaction might have led to erroneous inference and conclusions. Moreover, we perform a brief simulation study based on this example highlighting the effects caused by omitting or unnecessarily including interaction terms. Based on our results, we recommend to always include interaction terms in mixed-effects meta-regression models, when such interactions are plausible. △ Less

Submitted 9 January, 2023; originally announced January 2023.

Comments: 10 pages, 2 figures. arXiv admin note: text overlap with arXiv:2201.05491

Accounting for Time Dependency in Meta-Analyses of Concordance Probability Estimates

Authors: Matthias Schmid, Tim Friede, Nadja Klein, Leonie Weinhold

Abstract: Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small-sized validation studies. It is therefore necessary to synthesize the results of these studies usin… ▽ More Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small-sized validation studies. It is therefore necessary to synthesize the results of these studies using techniques for meta-analysis. Here we consider strategies for meta-analyzing the concordance probability for time-to-event data ("C-index"), which has become a popular tool to evaluate the discriminatory power of prediction models with a right-censored outcome. We show that standard meta-analysis of the C-index may lead to biased results, as the magnitude of the concordance probability depends on the length of the time interval used for evaluation (defined e.g. by the follow-up time, which might differ considerably between studies). To address this issue, we propose a set of methods for random-effects meta-regression that incorporate time directly as covariate in the model equation. In addition to analyzing nonlinear time trends via fractional polynomial, spline, and exponential decay models, we provide recommendations on suitable transformations of the C-index before meta-regression. Our results suggest that the C-index is best meta-analyzed using fractional polynomial meta-regression with logit-transformed C-index values. Classical random-effects meta-analysis (not considering time as covariate) is demonstrated to be a suitable alternative when follow-up times are small. Our findings have implications for the reporting of C-index values in future studies, which should include information on the length of the time interval underlying the calculations. △ Less

Submitted 3 December, 2022; originally announced December 2022.

Model-based recursive partitioning for discrete event times

Authors: Cynthia Huber, Matthias Schmid, Tim Friede

Abstract: Model-based recursive partitioning (MOB) is a semi-parametric statistical approach allowing the identification of subgroups that can be combined with a broad range of outcome measures including continuous time-to-event outcomes. When time is measured on a discrete scale, methods and models need to account for this discreetness as otherwise subgroups might be spurious and effects biased. The test u… ▽ More Model-based recursive partitioning (MOB) is a semi-parametric statistical approach allowing the identification of subgroups that can be combined with a broad range of outcome measures including continuous time-to-event outcomes. When time is measured on a discrete scale, methods and models need to account for this discreetness as otherwise subgroups might be spurious and effects biased. The test underlying the splitting criterion of MOB, the M-fluctuation test, assumes independent observations. However, for fitting discrete time-to-event models the data matrix has to be modified resulting in an augmented data matrix violating the independence assumption. We propose MOB for discrete Survival data (MOB-dS) which controls the type I error rate of the test used for data splitting and therefore the rate of identifying subgroups although none is present. MOB-ds uses a permutation approach accounting for dependencies in the augmented time-to-event data to obtain the distribution under the null hypothesis of no subgroups being present. Through simulations we investigate the type I error rate of the new MOB-dS and the standard MOB for different patterns of survival curves and event rates. We find that the type I error rates of the test is well controlled for MOB-dS, but observe some considerable inflations of the error rate for MOB. To illustrate the proposed methods, MOB-dS is applied to data on unemployment duration. △ Less

Submitted 14 September, 2022; originally announced September 2022.

Using the bayesmeta R package for Bayesian random-effects meta-regression

Authors: Christian Röver, Tim Friede

Abstract: BACKGROUND: Random-effects meta-analysis within a hierarchical normal modeling framework is commonly implemented in a wide range of evidence synthesis applications. More general problems may even be tackled when considering meta-regression approaches that in addition allow for the inclusion of study-level covariables. METHODS: We describe the Bayesian meta-regression implementation provided in the… ▽ More BACKGROUND: Random-effects meta-analysis within a hierarchical normal modeling framework is commonly implemented in a wide range of evidence synthesis applications. More general problems may even be tackled when considering meta-regression approaches that in addition allow for the inclusion of study-level covariables. METHODS: We describe the Bayesian meta-regression implementation provided in the bayesmeta R package including the choice of priors, and we illustrate its practical use. RESULTS: A wide range of example applications are given, such as binary and continuous covariables, subgroup analysis, indirect comparisons, and model selection. Example R code is provided. CONCLUSIONS: The bayesmeta package provides a flexible implementation. Due to the avoidance of MCMC methods, computations are fast and reproducible, facilitating quick sensitivity checks or large-scale simulation studies. △ Less

Submitted 25 November, 2022; v1 submitted 13 September, 2022; originally announced September 2022.

Comments: 17 pages, 8 figures

Journal ref: Computer Methods and Programs in Biomedicine, 229:107303, 2023

On the role of benchmarking data sets and simulations in method comparison studies

Authors: Sarah Friedrich, Tim Friede

Abstract: Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral but favour a novel method. Apart from the choice of design and a proper reporting of the findings, there are different approaches concerning the underlying data… ▽ More Method comparisons are essential to provide recommendations and guidance for applied researchers, who often have to choose from a plethora of available approaches. While many comparisons exist in the literature, these are often not neutral but favour a novel method. Apart from the choice of design and a proper reporting of the findings, there are different approaches concerning the underlying data for such method comparison studies. Most manuscripts on statistical methodology rely on simulation studies and provide a single real-world data set as an example to motivate and illustrate the methodology investigated. In the context of supervised learning, in contrast, methods are often evaluated using so-called benchmarking data sets, i.e. real-world data that serve as gold standard in the community. Simulation studies, on the other hand, are much less common in this context. The aim of this paper is to investigate differences and similarities between these approaches, to discuss their advantages and disadvantages and ultimately to develop new approaches to the evaluation of methods picking the best of both worlds. To this aim, we borrow ideas from different contexts such as mixed methods research and Clinical Scenario Evaluation. △ Less

Submitted 6 December, 2022; v1 submitted 2 August, 2022; originally announced August 2022.

Co** with Information Loss and the Use of Auxiliary Sources of Data: A Report from the NISS Ingram Olkin Forum Series on Unplanned Clinical Trial Disruptions

Authors: Silvia Calderazzo, Sergey Tarima, Carissa Reid, Nancy Flournoy, Tim Friede, Nancy Geller, James L Rosenberger, Nigel Stallard, Moreno Ursino, Marc Vandemeulebroecke, Kelly Van Lancker, Sarah Zohar

Abstract: Clinical trials disruption has always represented a non negligible part of the ending of interventional studies. While the SARS-CoV-2 (COVID-19) pandemic has led to an impressive and unprecedented initiation of clinical research, it has also led to considerable disruption of clinical trials in other disease areas, with around 80% of non-COVID-19 trials stopped or interrupted during the pandemic. I… ▽ More Clinical trials disruption has always represented a non negligible part of the ending of interventional studies. While the SARS-CoV-2 (COVID-19) pandemic has led to an impressive and unprecedented initiation of clinical research, it has also led to considerable disruption of clinical trials in other disease areas, with around 80% of non-COVID-19 trials stopped or interrupted during the pandemic. In many cases the disrupted trials will not have the planned statistical power necessary to yield interpretable results. This paper describes methods to compensate for the information loss arising from trial disruptions by incorporating additional information available from auxiliary data sources. The methods described include the use of auxiliary data on baseline and early outcome data available from the trial itself and frequentist and Bayesian approaches for the incorporation of information from external data sources. The methods are illustrated by application to the analysis of artificial data based on the Primary care pediatrics Learning Activity Nutrition (PLAN) study, a clinical trial assessing a diet and exercise intervention for overweight children, that was affected by the COVID-19 pandemic. We show how all of the methods proposed lead to an increase in precision relative to use of complete case data only. △ Less

Submitted 22 June, 2022; originally announced June 2022.

Double arcsine transform not appropriate for meta-analysis

Authors: Christian Röver, Tim Friede

Abstract: The variance-stabilizing Freeman-Tukey double arcsine transform was originally proposed for inference on single proportions. Subsequently, its use has been suggested in the context of meta-analysis of proportions. While some erratic behaviour has been observed previously, here we point out and illustrate general issues of monotonicity and invertibility that make this transform unsuitable for meta-… ▽ More The variance-stabilizing Freeman-Tukey double arcsine transform was originally proposed for inference on single proportions. Subsequently, its use has been suggested in the context of meta-analysis of proportions. While some erratic behaviour has been observed previously, here we point out and illustrate general issues of monotonicity and invertibility that make this transform unsuitable for meta-analysis purposes. △ Less

Submitted 8 July, 2022; v1 submitted 9 March, 2022; originally announced March 2022.

Comments: 4 pages, 2 figures, 1 table

Journal ref: Research Synthesis Methods, 13(5):645-648, 2022

Summarizing empirical information on between-study heterogeneity for Bayesian random-effects meta-analysis

Authors: Christian Röver, Sibylle Sturtz, Jona Lilienthal, Ralf Bender, Tim Friede

Abstract: In Bayesian meta-analysis, the specification of prior probabilities for the between-study heterogeneity is commonly required, and is of particular benefit in situations where only few studies are included. Among the considerations in the set-up of such prior distributions, the consultation of available empirical data on a set of relevant past analyses sometimes plays a role. How exactly to summari… ▽ More In Bayesian meta-analysis, the specification of prior probabilities for the between-study heterogeneity is commonly required, and is of particular benefit in situations where only few studies are included. Among the considerations in the set-up of such prior distributions, the consultation of available empirical data on a set of relevant past analyses sometimes plays a role. How exactly to summarize historical data sensibly is not immediately obvious; in particular, the investigation of an empirical collection of heterogeneity estimates will not target the actual problem and will usually only be of limited use. The commonly used normal-normal hierarchical model for random-effects meta-analysis is extended to infer a heterogeneity prior. Using an example data set, we demonstrate how to fit a distribution to empirically observed heterogeneity data from a set of meta-analyses. Considerations also include the choice of a parametric distribution family. Here, we focus on simple and readily applicable approaches to then translate these into (prior) probability distributions. △ Less

Submitted 17 November, 2022; v1 submitted 25 February, 2022; originally announced February 2022.

Comments: 18 pages, 5 tables, 4 figures

Journal ref: Statistics in Medicine, 42(14):2439-2454, 2023

MetaStan: An R package for Bayesian (model-based) meta-analysis using Stan

Authors: Burak Kürsad Günhan, Christian Röver, Tim Friede

Abstract: Meta-analysis methods are used to combine evidence from multiple studies. Meta-regression as well as model-based meta-analysis are extensions of standard pairwise meta-analysis in which information about study-level covariates and (arm-level) dosing amount or exposure may be taken into account. A Bayesian approach to inference is very attractive in this context, especially when a meta-analysis is… ▽ More Meta-analysis methods are used to combine evidence from multiple studies. Meta-regression as well as model-based meta-analysis are extensions of standard pairwise meta-analysis in which information about study-level covariates and (arm-level) dosing amount or exposure may be taken into account. A Bayesian approach to inference is very attractive in this context, especially when a meta-analysis is based on few studies only or rare events. In this article, we present the R package MetaStan which implements a wide range of pairwise and model-based meta-analysis models. A generalised linear mixed model (GLMM) framework is used to describe the pairwise meta-analysis, meta-regression and model-based meta-analysis models. Within the GLMM framework, the likelihood and link functions are adapted to reflect the nature of the data. For example, a binomial likelihood with a logit link is used to perform a meta-analysis based on datasets with dichotomous endpoints. Bayesian computations are conducted using Stan via the rstan interface. Stan uses a Hamiltonian Monte Carlo sampler which belongs to the family of Markov chain Monte Carlo methods. Stan implementations are done by using suitable parametrizations to ease computations. The user-friendly R package MetaStan, available on CRAN, supports a wide range of pairwise and model-based meta-analysis models. MetaStan provides fitting functions for pairwise meta-analysis with the option of including covariates and model-based meta-analysis. The supported outcome types are continuous, binary, and count. Forest plots for the pairwise meta-analysis and dose-response plots for the model-based meta-analysis can be obtained from the package. The use of MetaStan is demonstrated through clinical examples. △ Less

Submitted 1 February, 2022; originally announced February 2022.

Comments: 18 pages, 2 figures, 4 tables

Robust Confidence Intervals for Meta-Regression with Interaction Effects

Authors: Thilo Welz, Eric S. Knop, Tim Friede, Markus Pauly

Abstract: Meta-analysis is an important statistical technique for synthesizing the results of multiple studies regarding the same or closely related research question. So-called meta-regression extends meta-analysis models by accounting for studylevel covariates. Mixed-effects meta-regression models provide a powerful tool for evidence synthesis, by appropriately accounting for betweem-study heterogeneity.… ▽ More Meta-analysis is an important statistical technique for synthesizing the results of multiple studies regarding the same or closely related research question. So-called meta-regression extends meta-analysis models by accounting for studylevel covariates. Mixed-effects meta-regression models provide a powerful tool for evidence synthesis, by appropriately accounting for betweem-study heterogeneity. In fact, modelling the study effect in terms of random effects and moderators not only allows to examine the impact of the moderators, but often leads to more accurate estimates of the involved parameters. Nevertheless, due to the often small number of studies on a specific research topic, interactions are often neglected in meta-regression. In this work, we consider the research questions (i) how moderator interactions influence inference in mixed-effects meta-regression models and (ii) whether some inference methods are more reliable than others. Here, we review robust methods for confidence intervals in meta-regression models including interaction effects. These methods are based on the application of robust sandwich estimators for estimating the variance-covariance matrix of the vector of model coefficients. Furthermore, we compare different versions of these robust estimators in an extensive simulation study. We thereby investigate coverage and length of seven different confidence intervals under varying conditions. We conclude with some practical recommendations. △ Less

Submitted 21 February, 2023; v1 submitted 14 January, 2022; originally announced January 2022.

Comments: main paper: 23 pages, 6 figures; supplement: 148 pages, 131 figures

A straightforward meta-analysis approach for oncology phase I dose-finding studies

Authors: Christian Röver, Moreno Ursino, Tim Friede, Sarah Zohar

Abstract: Phase I early-phase clinical studies aim at investigating the safety and the underlying dose-toxicity relationship of a drug or combination. While little may still be known about the compound's properties, it is crucial to consider quantitative information available from any studies that may have been conducted previously on the same drug. A meta-analytic approach has the advantages of being able… ▽ More Phase I early-phase clinical studies aim at investigating the safety and the underlying dose-toxicity relationship of a drug or combination. While little may still be known about the compound's properties, it is crucial to consider quantitative information available from any studies that may have been conducted previously on the same drug. A meta-analytic approach has the advantages of being able to properly account for between-study heterogeneity, and it may be readily extended to prediction or shrinkage applications. Here we propose a simple and robust two-stage approach for the estimation of maximum tolerated dose(s) (MTDs) utilizing penalized logistic regression and Bayesian random-effects meta-analysis methodology. Implementation is facilitated using standard R packages. The properties of the proposed methods are investigated in Monte-Carlo simulations. The investigations are motivated and illustrated by two examples from oncology. △ Less

Submitted 4 February, 2022; v1 submitted 12 October, 2021; originally announced October 2021.

Comments: 30 pages, 11 figures, 8 tables

Journal ref: Statistics in Medicine, 41(20):3915-3940, 2022

Adjusting for publication bias in meta-analysis via inverse probability weighting using clinical trial registries

Authors: Ao Huang, Kosuke Morikawa, Tim Friede, Satoshi Hattori

Abstract: Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models and they have some advantages over the widely used bias-correction method of the trim-and-fill method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates an… ▽ More Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models and they have some advantages over the widely used bias-correction method of the trim-and-fill method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates and reasonable confidence intervals or require a complicated sensitivity analysis process. In this paper, we develop a simple publication bias adjustment method utilizing information on conducted but still unpublished trials from clinical trial registries. We introduce an estimating equation for parameter estimation in the selection function by regarding the publication bias issue as a missing data problem under missing not at random. With the estimated selection function, we introduce the inverse probability weighting (IPW) method to estimate the overall mean across studies. Furthermore, the IPW versions of heterogeneity measures such as the between-study variance and the I2 measure are proposed. We propose methods to construct asymptotic confidence intervals and suggest intervals based on parametric bootstrap** as an alternative. Through numerical experiments, we observed that the estimators successfully eliminate biases and the confidence intervals had empirical coverage probabilities close to the nominal level. On the other hand, the asymptotic confidence interval is much wider in some scenarios than the bootstrap confidence interval. Therefore, the latter is recommended for practical use. △ Less

Submitted 26 September, 2021; originally announced September 2021.

Comments: 50 pages, 3 figures

On the role of data, statistics and decisions in a pandemic

Authors: Beate Jahn, Sarah Friedrich, Joachim Behnke, Joachim Engel, Ursula Garczarek, Ralf Münnich, Markus Pauly, Adalbert Wilhelm, Olaf Wolkenhauer, Markus Zwick, Uwe Siebert, Tim Friede

Abstract: A pandemic poses particular challenges to decision-making because of the need to continuously adapt decisions to rapidly changing evidence and available data. For example, which countermeasures are appropriate at a particular stage of the pandemic? How can the severity of the pandemic be measured? What is the effect of vaccination in the population and which groups should be vaccinated first? The… ▽ More A pandemic poses particular challenges to decision-making because of the need to continuously adapt decisions to rapidly changing evidence and available data. For example, which countermeasures are appropriate at a particular stage of the pandemic? How can the severity of the pandemic be measured? What is the effect of vaccination in the population and which groups should be vaccinated first? The process of decision-making starts with data collection and modeling and continues to the dissemination of results and the subsequent decisions taken. The goal of this paper is to give an overview of this process and to provide recommendations for the different steps from a statistical perspective. In particular, we discuss a range of modeling techniques including mathematical, statistical and decision-analytic models along with their applications in the COVID-19 context. With this overview, we aim to foster the understanding of the goals of these modeling approaches and the specific data requirements that are essential for the interpretation of results and for successful interdisciplinary collaborations. A special focus is on the role played by data in these different models, and we incorporate into the discussion the importance of statistical literacy, and of effective dissemination and communication of findings. △ Less

Submitted 8 March, 2022; v1 submitted 6 August, 2021; originally announced August 2021.

Improving the Power of Economic Experiments Using Adaptive Designs

Authors: Sebastian Jobjörnsson, Henning Schaak, Oliver Mußhoff, Tim Friede

Abstract: An important issue for many economic experiments is how the experimenter can ensure sufficient power for rejecting one or more hypotheses. Here, we apply methods developed mainly within the area of clinical trials for testing multiple hypotheses simultaneously in adaptive, two-stage designs. Our main goal is to illustrate how this approach can be used to improve the power of economic experiments.… ▽ More An important issue for many economic experiments is how the experimenter can ensure sufficient power for rejecting one or more hypotheses. Here, we apply methods developed mainly within the area of clinical trials for testing multiple hypotheses simultaneously in adaptive, two-stage designs. Our main goal is to illustrate how this approach can be used to improve the power of economic experiments. Having briefly introduced the relevant theory, we perform a simulation study supported by the open source R package asd in order to evaluate the power of some different designs. The simulations show that the power to reject at least one hypothesis can be improved while still ensuring strong control of the overall Type I error probability, and without increasing the total sample size and thus the costs of the study. The derived designs are further illustrated by applying them to two different real-world data sets from experimental economics. △ Less

Submitted 5 August, 2021; originally announced August 2021.

Comments: 28 pages, 4 figures

Improving Adaptive Seamless Designs through Bayesian optimization

Authors: Jakob Richter, Tim Friede, Jörg Rahnenführer

Abstract: We propose to use Bayesian optimization (BO) to improve the efficiency of the design selection process in clinical trials. BO is a method to optimize expensive black-box functions, by using a regression as a surrogate to guide the search. In clinical trials, planning test procedures and sample sizes is a crucial task. A common goal is to maximize the test power, given a set of treatments, correspo… ▽ More We propose to use Bayesian optimization (BO) to improve the efficiency of the design selection process in clinical trials. BO is a method to optimize expensive black-box functions, by using a regression as a surrogate to guide the search. In clinical trials, planning test procedures and sample sizes is a crucial task. A common goal is to maximize the test power, given a set of treatments, corresponding effect sizes, and a total number of samples. From a wide range of possible designs we aim to select the best one in a short time to allow quick decisions. The standard approach to simulate the power for each single design can become too time-consuming. When the number of possible designs becomes very large, either large computational resources are required or an exhaustive exploration of all possible designs takes too long. Here, we propose to use BO to quickly find a clinical trial design with high power from a large number of candidate designs. We demonstrate the effectiveness of our approach by optimizing the power of adaptive seamless designs for different sets of treatment effect sizes. Comparing BO with an exhaustive evaluation of all candidate designs shows that BO finds competitive designs in a fraction of the time. △ Less

Submitted 19 May, 2021; originally announced May 2021.

Comments: Submitted to: Biometrical Journal

Design aspects of COVID-19 treatment trials: Improving probability and time of favourable events

Authors: Jan Beyersmann, Tim Friede, Claudia Schmoor

Abstract: As a reaction to the pandemic of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a multitude of clinical trials for the treatment of SARS-CoV-2 or the resulting corona disease (COVID-19) are globally at various stages from planning to completion. Although some attempts were made to standardize study designs, this was hindered by the ferocity of the pandemic and the need to set up… ▽ More As a reaction to the pandemic of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a multitude of clinical trials for the treatment of SARS-CoV-2 or the resulting corona disease (COVID-19) are globally at various stages from planning to completion. Although some attempts were made to standardize study designs, this was hindered by the ferocity of the pandemic and the need to set up trials quickly. We take the view that a successful treatment of COVID-19 patients (i) increases the probability of a recovery or improvement within a certain time interval, say 28 days; (ii) aims to expedite favourable events within this time frame; and (iii) does not increase mortality over this time period. On this background we discuss the choice of endpoint and its analysis. Furthermore, we consider consequences of this choice for other design aspects including sample size and power and provide some guidance on the application of adaptive designs in this particular context. △ Less

Submitted 27 November, 2020; originally announced December 2020.

Blinded sample size re-calculation in multiple composite population designs with normal data and baseline adjustments

Authors: Roland Gerard Gera, Tim Friede

Abstract: The increasing interest in subpopulation analysis has led to the development of various new trial designs and analysis methods in the fields of personalized medicine and targeted therapies. In this paper, subpopulations are defined in terms of an accumulation of disjoint population subsets and will therefore be called composite populations. The proposed trial design is applicable to any set of com… ▽ More The increasing interest in subpopulation analysis has led to the development of various new trial designs and analysis methods in the fields of personalized medicine and targeted therapies. In this paper, subpopulations are defined in terms of an accumulation of disjoint population subsets and will therefore be called composite populations. The proposed trial design is applicable to any set of composite populations, considering normally distributed endpoints and random baseline covariates. Treatment effects for composite populations are tested by combining $p$-values, calculated on the subset levels, using the inverse normal combination function to generate test statistics for those composite populations. The family-wise type I error rate for simultaneous testing is controlled in the strong sense by the application of the closed testing procedure. Critical values for intersection hypothesis tests are derived using multivariate normal distributions, reflecting the joint distribution of composite population test statistics under the null hypothesis. For sample size calculation and sample size re-calculation multivariate normal distributions are derived which describe the joint distribution of composite population test statistics under an assumed alternative hypothesis. Simulations demonstrate the strong control of the family-wise type I error rate in fixed designs and re-calculation designs with blinded sample size re-calculation. The target power after sample size re-calculation is typically met or close to being met. △ Less

Submitted 30 November, 2020; originally announced November 2020.

Subgroup identification in individual patient data meta-analysis using model-based recursive partitioning

Authors: Cynthia Huber, Norbert Benda, Tim Friede

Abstract: Model-based recursive partitioning (MOB) can be used to identify subgroups with differing treatment effects. The detection rate of treatment-by-covariate interactions and the accuracy of identified subgroups using MOB depend strongly on the sample size. Using data from multiple randomized controlled clinical trials can overcome the problem of too small samples. However, naively pooling data from m… ▽ More Model-based recursive partitioning (MOB) can be used to identify subgroups with differing treatment effects. The detection rate of treatment-by-covariate interactions and the accuracy of identified subgroups using MOB depend strongly on the sample size. Using data from multiple randomized controlled clinical trials can overcome the problem of too small samples. However, naively pooling data from multiple trials may result in the identification of spurious subgroups as differences in study design, subject selection and other sources of between-trial heterogeneity are ignored. In order to account for between-trial heterogeneity in individual participant data (IPD) meta-analysis random-effect models are frequently used. Commonly, heterogeneity in the treatment effect is modelled using random effects whereas heterogeneity in the baseline risks is modelled by either fixed effects or random effects. In this article, we propose metaMOB, a procedure using the generalized mixed-effects model tree (GLMM tree) algorithm for subgroup identification in IPD meta-analysis. Although the application of metaMOB is potentially wider, e.g. randomized experiments with participants in social sciences or preclinical experiments in life sciences, we focus on randomized controlled clinical trials. In a simulation study, metaMOB outperformed GLMM trees assuming a random intercept only and model-based recursive partitioning (MOB), whose algorithm is the basis for GLMM trees, with respect to the false discovery rates, accuracy of identified subgroups and accuracy of estimated treatment effect. The most robust and therefore most promising method is metaMOB with fixed effects for modelling the between-trial heterogeneity in the baseline risks. △ Less

Submitted 22 September, 2020; originally announced September 2020.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) -- estimation of adverse event risks

Authors: Regina Stegherr, Claudia Schmoor, Jan Beyersmann, Kaspar Rufibach, Valentine Jehl, Andreas Brückner, Lewin Eisele, Thomas Künzel, Katrin Kupas, Frank Langer, Friedhelm Leverkus, Anja Loos, Christiane Norenberg, Florian Voss, Tim Friede

Abstract: The SAVVY project aims to improve the analyses of adverse event (AE) data in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). Although statistical methodologies have advanced, in AE analyses often the incidence proportion, the incidence density, or a non-parametric Kaplan-Meier estimator (KME) are used, which eith… ▽ More The SAVVY project aims to improve the analyses of adverse event (AE) data in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). Although statistical methodologies have advanced, in AE analyses often the incidence proportion, the incidence density, or a non-parametric Kaplan-Meier estimator (KME) are used, which either ignore censoring or CEs. In an empirical study including randomized clinical trials from several sponsor organisations, these potential sources of bias are investigated. The main aim is to compare the estimators that are typically used in AE analysis to the Aalen-Johansen estimator (AJE) as the gold-standard. Here, one-sample findings are reported, while a companion paper considers consequences when comparing treatment groups. Estimators are compared with descriptive statistics, graphical displays and with a random effects meta-analysis. The influence of different factors on the size of the bias is investigated in a meta-regression. Comparisons are conducted at the maximum follow-up time and at earlier evaluation time points. CEs definition does not only include death before AE but also end of follow-up for AEs due to events possibly related to the disease course or the treatment. Ten sponsor organisations provided 17 trials including 186 types of AEs. The one minus KME was on average about 1.2-fold larger than the AJE. Leading forces influencing bias were the amount of censoring and of CEs. As a consequence, the average bias using the incidence proportion was less than 5%. Assuming constant hazards using incidence densities was hardly an issue provided that CEs were accounted for. There is a need to improve the guidelines of reporting risks of AEs so that the KME and the incidence proportion are replaced by the AJE with an appropriate definition of CEs. △ Less

Submitted 18 August, 2020; originally announced August 2020.

Journal ref: Trials, 2021, 22(1), 420

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) -- comparison of adverse event risks in randomized controlled trials

Authors: Kaspar Rufibach, Regina Stegherr, Claudia Schmoor, Valentine Jehl, Arthur Allignol, Annette Boeckenhoff, Cornelia Dunger-Baldauf, Lewin Eisele, Thomas Künzel, Katrin Kupas, Friedhelm Leverkus, Matthias Trampisch, Yumin Zhao, Tim Friede, Jan Beyersmann

Abstract: Analyses of adverse events (AEs) are an important aspect of the evaluation of experimental therapies. The SAVVY (Survival analysis for AdVerse events with Varying follow-up times) project aims to improve the analyses of AE data in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times, censoring, and competing events (CE). In an empirical study in… ▽ More Analyses of adverse events (AEs) are an important aspect of the evaluation of experimental therapies. The SAVVY (Survival analysis for AdVerse events with Varying follow-up times) project aims to improve the analyses of AE data in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times, censoring, and competing events (CE). In an empirical study including seventeen randomized clinical trials the effect of varying follow-up times, censoring, and competing events on comparisons of two treatment arms with respect to AE risks is investigated. The comparisons of relative risks (RR) of standard probability-based estimators to the gold-standard Aalen-Johansen estimator or hazard-based estimators to an estimated hazard ratio (HR) from Cox regression are done descriptively, with graphical displays, and using a random effects meta-analysis on AE level. The influence of different factors on the size of the bias is investigated in a meta-regression. We find that for both, avoiding bias and categorization of evidence with respect to treatment effect on AE risk into categories, the choice of the estimator is key and more important than features of the underlying data such as percentage of censoring, CEs, amount of follow-up, or value of the gold-standard RR. There is an urgent need to improve the guidelines of reporting AEs so that incidence proportions are finally replaced by the Aalen-Johansen estimator - rather than by Kaplan-Meier - with appropriate definition of CEs. For RRs based on hazards, the HR based on Cox regression has better properties than the ratio of incidence densities. △ Less

Submitted 20 September, 2022; v1 submitted 18 August, 2020; originally announced August 2020.

Causal inference methods for small non-randomized studies: Methods and an application in COVID-19

Authors: Sarah Friedrich, Tim Friede

Abstract: The usual development cycles are too slow for the development of vaccines, diagnostics and treatments in pandemics such as the ongoing SARS-CoV-2 pandemic. Given the pressure in such a situation, there is a risk that findings of early clinical trials are overinterpreted despite their limitations in terms of size and design. Motivated by a non-randomized open-label study investigating the efficacy… ▽ More The usual development cycles are too slow for the development of vaccines, diagnostics and treatments in pandemics such as the ongoing SARS-CoV-2 pandemic. Given the pressure in such a situation, there is a risk that findings of early clinical trials are overinterpreted despite their limitations in terms of size and design. Motivated by a non-randomized open-label study investigating the efficacy of hydroxychloroquine in patients with COVID-19, we describe in a unified fashion various alternative approaches to the analysis of non-randomized studies. A widely used tool to reduce the impact of treatment-selection bias are so-called propensity score (PS) methods. Conditioning on the propensity score allows one to replicate the design of a randomized controlled trial, conditional on observed covariates. Extensions include the g-computation approach, which is less frequently applied, in particular in clinical studies. Moreover, doubly robust estimators provide additional advantages. Here, we investigate the properties of propensity score based methods including three variations of doubly robust estimators in small sample settings, typical for early trials, in a simulation study. R code for the simulations is provided. △ Less

Submitted 9 October, 2020; v1 submitted 31 July, 2020; originally announced July 2020.

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

Authors: Christian Röver, Ralf Bender, Sofia Dias, Christopher H. Schmid, Heinz Schmidli, Sibylle Sturtz, Sebastian Weber, Tim Friede

Abstract: The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributio… ▽ More The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While non-informative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification. △ Less

Submitted 13 January, 2021; v1 submitted 16 July, 2020; originally announced July 2020.

Comments: 42 pages, 10 figures, 20 tables

Journal ref: Research Synthesis Methods, 12(4):448-474, 2021

Using clinical trial registries to inform Copas selection model for publication bias in meta-analysis

Authors: Ao Huang, Sho Komukai, Tim Friede, Satoshi Hattori

Abstract: Prospective registration of study protocols in clinical trial registries is a useful way to minimize the risk of publication bias in meta-analysis, and several clinical trial registries are available nowadays. However, they are mainly used as a tool for searching studies and information submitted to the registries has not been utilized as efficiently as it could. In addressing publication bias in… ▽ More Prospective registration of study protocols in clinical trial registries is a useful way to minimize the risk of publication bias in meta-analysis, and several clinical trial registries are available nowadays. However, they are mainly used as a tool for searching studies and information submitted to the registries has not been utilized as efficiently as it could. In addressing publication bias in meta-analyses, sensitivity analysis with the Copas selection model is a more objective alternative to widely-used graphical methods such as the funnel-plot and the trim-and-fill method. Despite its ability to quantify the potential impact of publication bias, a drawback of the model is that some parameters not to be specified. This may result in some difficulty in interpreting the results of the sensitivity analysis. In this paper, we propose an alternative inference procedure for the Copas selection model by utilizing information from clinical trial registries. Our method provides a simple and accurate way to estimate all unknown parameters in the Copas selection model. A simulation study revealed that our proposed method resulted in smaller biases and more accurate confidence intervals than existing methods. Furthermore, two published meta-analyses had been re-analysed to demonstrate how to implement the proposed method in practice. △ Less

Submitted 29 May, 2020; originally announced May 2020.

Clinical trials impacted by the COVID-19 pandemic: Adaptive designs to the rescue?

Authors: Cornelia Ursula Kunz, Silke Jörgens, Frank Bretz, Nigel Stallard, Kelly Van Lancker, Dong Xi, Sarah Zohar, Christoph Gerlinger, Tim Friede

Abstract: Very recently the new pathogen severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified and the coronavirus disease 2019 (COVID-19) declared a pandemic by the World Health Organization. The pandemic has a number of consequences for the ongoing clinical trials in non-COVID-19 conditions. Motivated by four currently ongoing clinical trials in a variety of disease areas we illustra… ▽ More Very recently the new pathogen severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was identified and the coronavirus disease 2019 (COVID-19) declared a pandemic by the World Health Organization. The pandemic has a number of consequences for the ongoing clinical trials in non-COVID-19 conditions. Motivated by four currently ongoing clinical trials in a variety of disease areas we illustrate the challenges faced by the pandemic and sketch out possible solutions including adaptive designs. Guidance is provided on (i) where blinded adaptations can help; (ii) how to achieve type I error rate control, if required; (iii) how to deal with potential treatment effect heterogeneity; (iv) how to utilize early readouts; and (v) how to utilize Bayesian techniques. In more detail approaches to resizing a trial affected by the pandemic are developed including considerations to stop a trial early, the use of group-sequential designs or sample size adjustment. All methods considered are implemented in a freely available R shiny app. Furthermore, regulatory and operational issues including the role of data monitoring committees are discussed. △ Less

Submitted 28 May, 2020; originally announced May 2020.

Shrinkage estimation for dose-response modeling in phase II trials with multiple schedules

Authors: Burak Kürsad Günhan, Paul Meyvisch, Tim Friede

Abstract: Recently, phase II trials with multiple schedules (frequency of administrations) have become more popular, for instance in the development of treatments for atopic dermatitis. If the relationship of the dose and response is described by a parametric model, a simplistic approach is to pool doses from different schedules. However, this approach ignores the potential heterogeneity in dose-response cu… ▽ More Recently, phase II trials with multiple schedules (frequency of administrations) have become more popular, for instance in the development of treatments for atopic dermatitis. If the relationship of the dose and response is described by a parametric model, a simplistic approach is to pool doses from different schedules. However, this approach ignores the potential heterogeneity in dose-response curves between schedules. A more reasonable approach is the partial pooling, i.e. certain parameters of the dose-response curves are shared, while others are allowed to vary. Rather than using schedule-specific fixed-effects, we propose a Bayesian hierarchical model with random-effects to model the between-schedule heterogeneity with regard to certain parameters. Schedule-specific dose-response relationships can then be estimated using shrinkage estimation. Considering Emax models, the proposed method displayed desirable performance in terms of the mean absolute error and the coverage probabilities for the dose-response curve compared to the complete pooling. Furthermore, it outperformed the partial pooling with schedule-specific fixed-effects by producing lower mean absolute error and shorter credible intervals. The methods are illustrated using simulations and a phase II trial example in atopic dermatitis. A publicly available R package, \texttt{ModStan}, is developed to automate the implementation of the proposed method (\href{https://github.com/gunhanb/ModStan}{https://github.com/gunhanb/ModStan}). △ Less

Submitted 8 May, 2020; originally announced May 2020.

Journal ref: Statistics in Biopharmaceutical Research, 2020

Bounds for the weight of external data in shrinkage estimation

Authors: Christian Röver, Tim Friede

Abstract: Shrinkage estimation in a meta-analysis framework may be used to facilitate dynamical borrowing of information. This framework might be used to analyze a new study in the light of previous data, which might differ in their design (e.g., a randomized controlled trial (RCT) and a clinical registry). We show how the common study weights arise in effect and shrinkage estimation, and how these may be g… ▽ More Shrinkage estimation in a meta-analysis framework may be used to facilitate dynamical borrowing of information. This framework might be used to analyze a new study in the light of previous data, which might differ in their design (e.g., a randomized controlled trial (RCT) and a clinical registry). We show how the common study weights arise in effect and shrinkage estimation, and how these may be generalized to the case of Bayesian meta-analysis. Next we develop simple ways to compute bounds on the weights, so that the contribution of the external evidence may be assessed a priori. These considerations are illustrated and discussed using numerical examples, including applications in the treatment of Creutzfeldt-Jakob disease and in fetal monitoring to prevent the occurrence of metabolic acidosis. The target study's contribution to the resulting estimate is shown to be bounded below. Therefore, concerns of evidence being easily overwhelmed by external data are largely unwarranted. △ Less

Submitted 18 November, 2020; v1 submitted 6 April, 2020; originally announced April 2020.

Comments: 14 pages, 1 figure, 5 tables

Journal ref: Biometrical Journal, 65(5):1131-1143, 2021

A confidence interval robust to publication bias for random-effects meta-analysis of few studies

Authors: M. Henmi, S. Hattori, T. Friede

Abstract: Systematic reviews aim to summarize all the available evidence relevant to a particular research question. If appropriate, the data from identified studies are quantitatively combined in a meta-analysis. Often only few studies regarding a particular research question exist. In these settings the estimation of the between-study heterogeneity is challenging. Furthermore, the assessment of publicatio… ▽ More Systematic reviews aim to summarize all the available evidence relevant to a particular research question. If appropriate, the data from identified studies are quantitatively combined in a meta-analysis. Often only few studies regarding a particular research question exist. In these settings the estimation of the between-study heterogeneity is challenging. Furthermore, the assessment of publication bias is difficult as standard methods such as visual inspection or formal hypothesis tests in funnel plots do not provide adequate guidance. Previously, Henmi and Copas (Statistics in Medicine 2010, 29: 2969--2983) proposed a confidence interval for the overall effect in random-effects meta-analysis that is robust to publication bias to some extent. As is evident from their simulations, the confidence intervals have improved coverage compared with standard methods. To our knowledge, the properties of their method has never been assessed for meta-analyses including fewer than five studies. In this manuscript, we propose a variation of the method by Henmi and Copas employing an improved estimator of the between-study heterogeneity, in particular when dealing with few studies only. In a simulation study, the proposed method is compared to several competitors. Overall, we found that our method outperforms the others in terms of coverage probabilities. In particular, an improvement compared with the proposal by Henmi and Copas is demonstrated. The work is motivated and illustrated by a systematic review and meta-analysis in paediatric immunosuppression following liver transplantations. △ Less

Submitted 11 July, 2020; v1 submitted 18 February, 2020; originally announced February 2020.

A Bayesian time-to-event pharmacokinetic model for phase I dose-escalation trials with multiple schedules

Authors: Burak Kürsad Günhan, Sebastian Weber, Tim Friede

Abstract: Phase I dose-escalation trials must be guided by a safety model in order to avoid exposing patients to unacceptably high risk of toxicities. Traditionally, these trials are based on one type of schedule. In more recent practice, however, there is often a need to consider more than one schedule, which means that in addition to the dose itself, the schedule needs to be varied in the trial. Hence, th… ▽ More Phase I dose-escalation trials must be guided by a safety model in order to avoid exposing patients to unacceptably high risk of toxicities. Traditionally, these trials are based on one type of schedule. In more recent practice, however, there is often a need to consider more than one schedule, which means that in addition to the dose itself, the schedule needs to be varied in the trial. Hence, the aim is finding an acceptable dose-schedule combination. However, most established methods for dose-escalation trials are designed to escalate the dose only and ad-hoc choices must be made to adapt these to the more complicated setting of finding an acceptable dose-schedule combination. In this paper, we introduce a Bayesian time-to-event model which takes explicitly the dose amount and schedule into account through the use of pharmacokinetic principles. The model uses a time-varying exposure measure to account for the risk of a dose-limiting toxicity over time. The dose-schedule decisions are informed by an escalation with overdose control criterion. The model is formulated using interpretable parameters which facilitates the specification of priors. In a simulation study, we compared the proposed method with an existing method. The simulation study demonstrates that the proposed method yields similar or better results compared to an existing method in terms of recommending acceptable dose-schedule combinations, yet reduces the number of patients enrolled in most of scenarios. The \texttt{R} and \texttt{Stan} code to implement the proposed method is publicly available from Github (\url{https://github.com/gunhanb/TITEPK_code}). △ Less

Submitted 14 February, 2020; originally announced February 2020.

Comments: arXiv admin note: text overlap with arXiv:1811.09433

Journal ref: Statistics in Medicine, 2020

Estimating and comparing adverse event probabilities in the presence of varying follow-up times and competing events

Authors: Regina Stegherr, Claudia Schmoor, Michael Lübbert, Tim Friede, Jan Beyersmann

Abstract: Safety analyses in terms of adverse events (AEs) are an important aspect of benefit-risk assessments of therapies. Compared to efficacy analyses AE analyses are often rather simplistic. The probability of an AE of a specific type is typically estimated by the incidence proportion, sometimes the incidence density or the Kaplan-Meier estimator are proposed. But these analyses either do not account f… ▽ More Safety analyses in terms of adverse events (AEs) are an important aspect of benefit-risk assessments of therapies. Compared to efficacy analyses AE analyses are often rather simplistic. The probability of an AE of a specific type is typically estimated by the incidence proportion, sometimes the incidence density or the Kaplan-Meier estimator are proposed. But these analyses either do not account for censoring, rely on a too restrictive parametric model, or ignore competing events. With the non-parametric Aalen-Johansen estimator as the gold-standard, these potential sources of bias are investigated in a data example from oncology and in simulations, both in the one-sample and in the two-sample case. As the estimators may have large variances at the end of follow-up, the estimators are not only compared at the maximal event time but also at two quantiles of the observed times. To date, consequences for safety comparisons have hardly been investigated in the literature. The impact of using different estimators for group comparisons is unclear, as, for example, the ratio of two both underestimating or overestimating estimators may or may not be comparable to the ratio of the gold-standard estimator. Therefore, the ratio of the AE probabilities is also calculated based on different approaches. By simulations investigating constant and non-constant hazards, different censoring mechanisms and event frequencies, we show that ignoring competing events is more of a problem than falsely assuming constant hazards by use of the incidence density and that the choice of the AE probability estimator is crucial for group comparisons. △ Less

Submitted 16 January, 2020; originally announced January 2020.

Comments: 27 pages, 5 figures

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): Rationale and statistical concept of a meta-analytic study

Authors: Regina Stegherr, Jan Beyersmann, Valentine Jehl, Kaspar Rufibach, Friedhelm Leverkus, Claudia Schmoor, Tim Friede

Abstract: The assessment of safety is an important aspect of the evaluation of new therapies in clinical trials, with analyses of adverse events being an essential part of this. Standard methods for the analysis of adverse events such as the incidence proportion, i.e. the number of patients with a specific adverse event out of all patients in the treatment groups, do not account for both varying follow-up t… ▽ More The assessment of safety is an important aspect of the evaluation of new therapies in clinical trials, with analyses of adverse events being an essential part of this. Standard methods for the analysis of adverse events such as the incidence proportion, i.e. the number of patients with a specific adverse event out of all patients in the treatment groups, do not account for both varying follow-up times and competing risks. Alternative approaches such as the Aalen-Johansen estimator of the cumulative incidence function have been suggested. Theoretical arguments and numerical evaluations support the application of these more advanced methodology, but as yet there is to our knowledge only insufficient empirical evidence whether these methods would lead to different conclusions in safety evaluations. The Survival analysis for AdVerse events with VarYing follow-up times (SAVVY) project strives to close this gap in evidence by conducting a meta-analytical study to assess the impact of the methodology on the conclusion of the safety assessment empirically. Here we present the rationale and statistical concept of the empirical study conducted as part of the SAVVY project. The statistical methods are presented in unified notation and examples of their implementation in R and SAS are provided. △ Less

Submitted 30 November, 2019; originally announced December 2019.

Journal ref: Biom. J., 2021, 63, 650-670

Present criteria for prophylactic ICD implantation: Insights from the EU-CERT-ICD (Comparative Effectiveness Research to Assess the Use of Primary ProphylacTic Implantable Cardioverter Defibrillators in EUrope) project

Authors: M. Zabel, S. Schlögl, A. Lubinski, J. H. Svendsen, A. Bauer, E. Arbelo, S. Brusich, D. Conen, I. Cygankiewicz, M. Dommasch, P. Flevari, J. Galuszka, J. Hansen, G. Hasenfuß, R. Hatala, H. V. Huikuri, T. Kenttä, T. Kuczejko, H. Haarmann, M. Harden, S. Iovev, S. Kääb, G. Kaliska, A. Katsimardos, J. D. Kasprzak , et al. (23 additional authors not shown)

Abstract: BACKGROUND. The clinical effectiveness of primary prevention implantable cardioverter defibrillator (ICD) therapy is under debate. It is urgently needed to better identify patients who benefit from prophylactic ICD therapy. The EUropean Comparative Effectiveness Research to Assess the Use of Primary ProphylacTic Implantable Cardioverter Defibrillators (EU-CERT-ICD) completed in 2019 will assess th… ▽ More BACKGROUND. The clinical effectiveness of primary prevention implantable cardioverter defibrillator (ICD) therapy is under debate. It is urgently needed to better identify patients who benefit from prophylactic ICD therapy. The EUropean Comparative Effectiveness Research to Assess the Use of Primary ProphylacTic Implantable Cardioverter Defibrillators (EU-CERT-ICD) completed in 2019 will assess this issue. SUMMARY. The EU-CERT-ICD is a prospective investigator-initiated non-randomized, controlled, multicenter observational cohort study done in 44 centers across 15 European countries. A total of 2327 patients with heart failure due to ischemic heart disease or dilated cardiomyopathy indicated for primary prophylactic ICD implantation were recruited between 2014 and 2018 (>1500 patients at first ICD implantation, >750 patients non-randomized non-ICD control group). The primary endpoint was all-cause mortality, first appropriate shock was co-primary endpoint. At baseline, all patients underwent 12-lead ECG and Holter-ECG analysis using multiple advanced methods for risk stratification as well as documentation of clinical characteristics and laboratory values. The EU-CERT-ICD data will provide much needed information on the survival benefit of preventive ICD therapy and expand on previous prospective risk stratification studies which showed very good applicability of clinical parameters and advanced risk stratifiers in order to define patient subgroups with above or below average ICD benefit. CONCLUSION. The EU-CERT-ICD study will provide new and current data about effectiveness of primary prophylactic ICD implantation. The study also aims for improved risk stratification and patient selection using clinical risk markers in general, and advanced ECG risk markers in particular. △ Less

Submitted 12 September, 2019; originally announced September 2019.

Comments: 22 pages, 3 figures

Journal ref: Journal of Electrocardiology, 2019

Random-effects meta-analysis of phase I dose-finding studies using stochastic process priors

Authors: Moreno Ursino, Christian Röver, Sarah Zohar, Tim Friede

Abstract: Phase I dose-finding studies aim at identifying the maximal tolerated dose (MTD). It is not uncommon that several dose-finding studies are conducted, although often with some variation in the administration mode or dose panel. For instance, sorafenib (BAY 43-900) was used as monotherapy in at least 29 phase I trials according to a recent search in clinicaltrials.gov. Since the toxicity may not be… ▽ More Phase I dose-finding studies aim at identifying the maximal tolerated dose (MTD). It is not uncommon that several dose-finding studies are conducted, although often with some variation in the administration mode or dose panel. For instance, sorafenib (BAY 43-900) was used as monotherapy in at least 29 phase I trials according to a recent search in clinicaltrials.gov. Since the toxicity may not be directly related to the specific indication, synthesizing the information from several studies might be worthwhile. However, this is rarely done in practice and only a fixed-effect meta-analysis framework was proposed to date. We developed a Bayesian random-effects meta-analysis methodology to pool several phase I trials and suggest the MTD. A curve free hierarchical model on the logistic scale with random effects, accounting for between-trial heterogeneity, is used to model the probability of toxicity across the investigated doses. An Ornstein-Uhlenbeck Gaussian process is adopted for the random effects structure. Prior distributions for the curve free model are based on a latent Gamma process. An extensive simulation study showed good performance of the proposed method also under model deviations. Sharing information between phase I studies can improve the precision of MTD selection, at least when the number of trials is reasonably large. △ Less

Submitted 1 August, 2019; originally announced August 2019.

Comments: 23 pages, 6 figures, 7 tables

Journal ref: The Annals of Applied Statistics, 15(1):174-193, 2021

Decline of COPD exacerbations in clinical trials over two decades -- a systematic review and meta-regression

Authors: Stefan Andreas, Christian Röver, Judith Heinz, Sebastian Straube, Henrik Watz, Tim Friede

Abstract: BACKGROUND: An important goal of chronic obstructive pulmonary disease (COPD) treatment is to reduce the frequency of exacerbations. Some observations suggest a decline in exacerbation rates in clinical trials over time. A more systematic understanding would help to improve the design and interpretation of COPD trials. METHODS: We performed a systematic review and meta-regression of the placebo… ▽ More BACKGROUND: An important goal of chronic obstructive pulmonary disease (COPD) treatment is to reduce the frequency of exacerbations. Some observations suggest a decline in exacerbation rates in clinical trials over time. A more systematic understanding would help to improve the design and interpretation of COPD trials. METHODS: We performed a systematic review and meta-regression of the placebo groups in published randomized controlled trials reporting exacerbations as an outcome. A Bayesian negative binomial model was developed to accommodate results that are reported in different formats; results are reported with credible intervals (CI) and posterior tail probabilities ($p_B$). RESULTS: Of 1114 studies identified by our search, 55 were ultimately included. Exacerbation rates decreased by 6.7% (95% CI (4.4, 9.0); $p_B$ < 0.001) per year, or 50% (95% CI (36, 61)) per decade. Adjusting for available study and baseline characteristics such as forced expiratory volume in 1 s (FEV1) did not alter the observed trend considerably. Two subsets of studies, one using a true placebo group and the other allowing inhaled corticosteroids in the "placebo" group, also yielded consistent results. CONCLUSIONS: In conclusion, this meta-regression indicates that the rate of COPD exacerbations decreased over the past two decades to a clinically relevant extent independent of important prognostic factors. This suggests that care is needed in the design of new trials or when comparing results from older trials with more recent ones. Also a considerable effect of adjunct therapy on COPD exacerbations can be assumed. △ Less

Submitted 17 August, 2019; originally announced August 2019.

Comments: 11 pages, 4 figures

Journal ref: Respiratory Research 20:186, 2019

Blinded continuous information monitoring of recurrent event endpoints with time trends in clinical trials

Authors: Tobias Mütze, Susanna Salem, Norbert Benda, Heinz Schmidli, Tim Friede

Abstract: Blinded sample size re-estimation and information monitoring based on blinded data has been suggested to mitigate risks due to planning uncertainties regarding nuisance parameters. Motivated by a randomized controlled trial in pediatric multiple sclerosis (MS), a continuous monitoring procedure for overdispersed count data was proposed recently. However, this procedure assumed constant event rates… ▽ More Blinded sample size re-estimation and information monitoring based on blinded data has been suggested to mitigate risks due to planning uncertainties regarding nuisance parameters. Motivated by a randomized controlled trial in pediatric multiple sclerosis (MS), a continuous monitoring procedure for overdispersed count data was proposed recently. However, this procedure assumed constant event rates, an assumption often not met in practice. Here we extend the procedure to accommodate time trends in the event rates considering two blinded approaches: (a) the mixture approach modeling the number of events by a mixture of two negative binomial distributions, and (b) the lum** approach approximating the marginal distribution of the event counts by a negative binomial distribution. Through simulations the operating characteristics of the proposed procedures are investigated under decreasing event rates. We find that the type I error rate is not inflated relevantly by either of the monitoring procedures, with the exception of strong time dependencies where the procedure assuming constant rates exhibits some inflation. Furthermore, the procedure accommodating time trends has generally favorable power properties compared to the procedure based on constant rates which stops often too late. The proposed method is illustrated by the clinical trial in pediatric MS. △ Less

Submitted 6 March, 2019; originally announced March 2019.

Seamless phase II/III clinical trials using early outcomes for treatment or subgroup selection: Methods and aspects of their implementation

Authors: Tim Friede, Nigel Stallard, Nicholas Parsons

Abstract: Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, e.g. in terms of sample size and / or development time. It is well acknowledged that adaptive designs are more involved from a logistica… ▽ More Adaptive seamless designs combine confirmatory testing, a domain of phase III trials, with features such as treatment or subgroup selection, typically associated with phase II trials. They promise to increase the efficiency of development programmes of new drugs, e.g. in terms of sample size and / or development time. It is well acknowledged that adaptive designs are more involved from a logistical perspective and require more upfront planning, often in form of extensive simulation studies, than conventional approaches. Here we present a framework for adaptive treatment and subgroup selection using the same notation, which links the somewhat disparate literature on treatment selection on one side and on subgroup selection on the other. Furthermore, we introduce a flexible and yet efficient simulation model that serves both designs. As primary endpoints often take a long time to observe, interim analyses are frequently informed by early outcomes. Therefore, all methods presented accommodate both, interim analyses informed by the primary outcome or an early outcome. The R package asd, previously developed to simulate designs with treatment selection, was extended to include subpopulation selection (so-called adaptive enrichment designs). Here we describe the functionality of the R package asd and use it to present some worked-up examples motivated by clinical trials in chronic obstructive pulmonary disease and oncology. The examples illustrate various features of the R package providing at the same time insights into the operating characteristics of adaptive seamless studies. △ Less

Submitted 24 January, 2019; originally announced January 2019.

A Bayesian time-to-event pharmacokinetic model for sequential phase I dose-escalation trials with multiple schedules

Authors: Burak Kürsad Günhan, Sebastian Weber, Abdelkader Seroutou, Tim Friede

Abstract: Phase I dose-escalation trials constitute the first step in investigating the safety of potentially promising drugs in humans. Conventional methods for phase I dose-escalation trials are based on a single treatment schedule only. More recently, however, multiple schedules are more frequently investigated in the same trial. Here, we consider sequential phase I trials, where the trial proceeds with… ▽ More Phase I dose-escalation trials constitute the first step in investigating the safety of potentially promising drugs in humans. Conventional methods for phase I dose-escalation trials are based on a single treatment schedule only. More recently, however, multiple schedules are more frequently investigated in the same trial. Here, we consider sequential phase I trials, where the trial proceeds with a new schedule (e.g. daily or weekly dosing) once the dose escalation with another schedule has been completed. The aim is to utilize the information from both the completed and the ongoing dose-escalation trial to inform decisions on the dose level for the next dose cohort. For this purpose, we adapted the time-to-event pharmacokinetics (TITE-PK) model, which were originally developed for simultaneous investigation of multiple schedules. TITE-PK integrates information from multiple schedules using a pharmacokinetics (PK) model. In a simulation study, the developed appraoch is compared to the bridging continual reassessment method and the Bayesian logistic regression model using a meta-analytic-prior. TITE-PK results in better performance than comparators in terms of recommending acceptable dose and avoiding overly toxic doses for sequential phase I trials in most of the scenarios considered. Furthermore, better performance of TITE-PK is achieved while requiring similar number of patients in the simulated trials. For the scenarios involving one schedule, TITE-PK displays similar performance with alternatives in terms of acceptable dose recommendations. The \texttt{R} and \texttt{Stan} code for the implementation of an illustrative sequential phase I trial example is publicly available at https://github.com/gunhanb/TITEPK_sequential. △ Less

Submitted 20 August, 2020; v1 submitted 23 November, 2018; originally announced November 2018.

Journal ref: BMC medical research methodology, 2021

Recent advances in methodology for clinical trials in small populations: the InSPiRe project

Authors: T. Friede, M. Posch, S. Zohar, C. Alberti, N. Benda, E. Comets, S. Day, A. Dmitrenko, A. Graf, B. K. Günhan, S. W. Hee, F. Lentz, J. Madan, F. Miller, T. Ondra, M. Pearce, C. Röver, A. Tournazi, S. Unkel, M. Ursino, G. Wassmer, N. Stallard

Abstract: Where there are a limited number of patients, such as in a rare disease, clinical trials in these small populations present several challenges, including statistical issues. This led to an EU FP7 call for proposals in 2013. One of the three projects funded was the Innovative Methodology for Small Populations Research (InSPiRe) project. This paper summarizes the main results of the project, which w… ▽ More Where there are a limited number of patients, such as in a rare disease, clinical trials in these small populations present several challenges, including statistical issues. This led to an EU FP7 call for proposals in 2013. One of the three projects funded was the Innovative Methodology for Small Populations Research (InSPiRe) project. This paper summarizes the main results of the project, which was completed in 2017. The InSPiRe project has led to development of novel statistical methodology for clinical trials in small populations in four areas. We have explored new decision-making methods for small population clinical trials using a Bayesian decision-theoretic framework to compare costs with potential benefits, developed approaches for targeted treatment trials, enabling simultaneous identification of subgroups and confirmation of treatment effect for these patients, worked on early phase clinical trial design and on extrapolation from adult to pediatric studies, develo** methods to enable use of pharmacokinetics and pharmacodynamics data, and also developed improved robust meta-analysis methods for a small number of trials to support the planning, analysis and interpretation of a trial as well as enabling extrapolation between patient groups. In addition to scientific publications, we have contributed to regulatory guidance and produced free software in order to facilitate implementation of the novel methods. △ Less

Submitted 30 October, 2018; originally announced November 2018.

Comments: 9 pages, 3 figures

Journal ref: Orphanet Journal of Rare Diseases, 13:136, 2018

Contribution to the discussion of "When should meta-analysis avoid making hidden normality assumptions?": A Bayesian perspective

Authors: Christian Röver, Tim Friede

Abstract: Contribution to the discussion of "When should meta-analysis avoid making hidden normality assumptions?" by Dan Jackson and Ian R. White (2018; https://doi.org/10.1002/bimj.201800071). Contribution to the discussion of "When should meta-analysis avoid making hidden normality assumptions?" by Dan Jackson and Ian R. White (2018; https://doi.org/10.1002/bimj.201800071). △ Less

Submitted 17 September, 2018; originally announced September 2018.

Comments: 3 pages, 1 figure

Journal ref: Biometrical Journal, 60(6):1068-1070, 2018

Meta-analysis of few studies involving rare events

Authors: Burak Kürsad Günhan, Christian Röver, Tim Friede

Abstract: Meta-analyses of clinical trials targeting rare events face particular challenges when the data lack adequate numbers of events for all treatment arms. Especially when the number of studies is low, standard meta-analysis methods can lead to serious distortions because of such data sparsity. To overcome this, we suggest the use of weakly informative priors (WIP) for the treatment effect parameter o… ▽ More Meta-analyses of clinical trials targeting rare events face particular challenges when the data lack adequate numbers of events for all treatment arms. Especially when the number of studies is low, standard meta-analysis methods can lead to serious distortions because of such data sparsity. To overcome this, we suggest the use of weakly informative priors (WIP) for the treatment effect parameter of a Bayesian meta-analysis model, which may also be seen as a form of penalization. As a data model, we use a binomial-normal hierarchical model (BNHM) which does not require continuity corrections in case of zero counts in one or both arms. We suggest a normal prior for the log odds ratio with mean 0 and standard deviation 2.82, which is motivated (1) as a symmetric prior centred around unity and constraining the odds ratio to within a range from 1/250 to 250 with 95 % probability, and (2) as consistent with empirically observed effect estimates from a set of $\mbox{$37\,773$}$ meta-analyses from the Cochrane Database of Systematic Reviews. In a simulation study with rare events and few studies, our BNHM with a WIP outperformed a Bayesian method without a WIP and a maximum likelihood estimator in terms of smaller bias and shorter interval estimates with similar coverage. Furthermore, the methods are illustrated by a systematic review in immunosuppression of rare safety events following paediatric transplantation. A publicly available $\textbf{R}$ package, $\texttt{MetaStan}$, is developed to automate the $\textbf{Stan}$ implementation of meta-analysis models using WIPs. △ Less

Submitted 12 September, 2018; originally announced September 2018.

Journal ref: Research Synthesis Methods 2020; 11: 74-90

Likelihood-based meta-analysis with few studies: Empirical and simulation studies

Authors: Svenja E. Seide, Christian Röver, Tim Friede

Abstract: Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. We consider several likelihood-based inference methods. Confidence intervals are based on normal or Student-t approximations.… ▽ More Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. We consider several likelihood-based inference methods. Confidence intervals are based on normal or Student-t approximations. We extract an empirical data set of 40 meta-analyses from recent reviews published by the German Institute for Quality and Efficiency in Health Care (IQWiG). Methods are then compared empirically as well as in a simulation study, considering odds-ratio and risk ratio effect sizes. Empirically, a majority of the identified meta-analyses include only 2 studies. In the simulation study, coverage probability is, in the presence of heterogeneity and few studies, below the nominal level for all frequentist methods based on normal approximation, in particular when sizes in meta-analyses are not balanced, but improve when confidence intervals are adjusted. Bayesian methods result in better coverage than the frequentist methods with normal approximation in all scenarios. Credible intervals are empirically and in the simulation study wider than unadjusted confidence intervals, but considerably narrower than adjusted ones. Confidence intervals based on the generalized linear mixed models are in general, slightly narrower than those from other frequentist methods. Certain methods turned out impractical due to frequent numerical problems. In the presence of between-study heterogeneity, especially with unbalanced study sizes, caution is needed in applying meta-analytical methods to few studies, as either coverage probabilities might be compromised, or intervals are inconclusively wide. Bayesian estimation with a sensibly chosen prior for between-trial heterogeneity may offer a promising compromise. △ Less

Submitted 30 October, 2018; v1 submitted 24 July, 2018; originally announced July 2018.

Comments: 22 pages

Journal ref: BMC Medical Research Methodology 19:16, 2019

Dynamically borrowing strength from another study through shrinkage estimation

Authors: Christian Röver, Tim Friede

Abstract: Meta-analytic methods may be used to combine evidence from different sources of information. Quite commonly, the normal-normal hierarchical model (NNHM) including a random-effect to account for between-study heterogeneity is utilized for such analyses. The same modeling framework may also be used to not only derive a combined estimate, but also to borrow strength for a particular study from anothe… ▽ More Meta-analytic methods may be used to combine evidence from different sources of information. Quite commonly, the normal-normal hierarchical model (NNHM) including a random-effect to account for between-study heterogeneity is utilized for such analyses. The same modeling framework may also be used to not only derive a combined estimate, but also to borrow strength for a particular study from another by deriving a shrinkage estimate. For instance, a small-scale randomized controlled trial could be supported by a non-randomized study, e.g. a clinical registry. This would be particularly attractive in the context of rare diseases. We demonstrate that a meta-analysis still makes sense in this extreme two-study setup, as illustrated using a recent trial and a clinical registry in Creutzfeld-Jakob disease. Derivation of a shrinkage estimate within a Bayesian random-effects meta-analysis may substantially improve a given estimate even based on only a single additional estimate while accounting for potential effect heterogeneity between the studies. Alternatively, inference may equivalently be motivated via a model specification that does not require a common overall mean parameter but considers the treatment effect in one study, and the difference in effects between the studies. The proposed approach is quite generally applicable to combine different types of evidence originating e.g. from meta-analyses or individual studies. An application of this more general setup is provided in immunosuppression following liver transplantation in children. △ Less

Submitted 4 December, 2018; v1 submitted 4 June, 2018; originally announced June 2018.

Comments: 23 pages, 3 figures, 4 tables

Journal ref: Statistical Methods in Medical Research, 29(1):293-308, 2020

Model averaging for robust extrapolation in evidence synthesis

Authors: Christian Röver, Simon Wandel, Tim Friede

Abstract: Extrapolation from a source to a target, e.g., from adults to children, is a promising approach to utilizing external information when data are sparse. In the context of meta-analysis, one is commonly faced with a small number of studies, while potentially relevant additional information may also be available. Here we describe a simple extrapolation strategy using heavy-tailed mixture priors for e… ▽ More Extrapolation from a source to a target, e.g., from adults to children, is a promising approach to utilizing external information when data are sparse. In the context of meta-analysis, one is commonly faced with a small number of studies, while potentially relevant additional information may also be available. Here we describe a simple extrapolation strategy using heavy-tailed mixture priors for effect estimation in meta-analysis, which effectively results in a model-averaging technique. The described method is robust in the sense that a potential prior-data conflict, i.e., a discrepancy between source and target data, is explicitly anticipated. The aim of this paper to develop a solution for this particular application, to showcase the ease of implementation by providing R code, and to demonstrate the robustness of the general approach in simulations. △ Less

Submitted 17 September, 2018; v1 submitted 28 May, 2018; originally announced May 2018.

Comments: 18 pages, 7 figures, 5 tables

Journal ref: Statistics in Medicine, 38(4):674-694, 2019