-
Contextual aggregation and rapid updating of trial outcomes within a user-friendly open-source environment
Authors:
František Bartoš,
Eric-Jan Wagenmakers,
Christiaan H. Vinkers,
Kees P. J. Braun,
Willem M. Otte
Abstract:
The delayed and incomplete availability of historical findings and the lack of integrative and user-friendly software hampers the reliable interpretation of new clinical data. We developed a free, open, and user-friendly clinical trial aggregation program combining a large and representative sample of existing trial data with the latest classical and Bayesian meta-analytical models, including clea…
▽ More
The delayed and incomplete availability of historical findings and the lack of integrative and user-friendly software hampers the reliable interpretation of new clinical data. We developed a free, open, and user-friendly clinical trial aggregation program combining a large and representative sample of existing trial data with the latest classical and Bayesian meta-analytical models, including clear output visualizations. Our software is of particular interest for (post-graduate) educational programs (e.g., medicine, epidemiology) and global health initiatives. We demonstrate the database, interface, and plot functionality with a recent randomized controlled trial on effective epileptic seizure reduction in children treated for a parasitic brain infection. The single trial data is placed into context and we show how to interpret new results against existing knowledge instantaneously. Our program is of particular interest to those working on the contextualizing of medical findings. It may facilitate the advancement of global clinical progress as efficiently and openly as possible and simulate further bridging clinical data with the latest biostatistical models.
△ Less
Submitted 24 June, 2023;
originally announced June 2023.
-
Empirical prior distributions for Bayesian meta-analyses of binary and time to event outcomes
Authors:
František Bartoš,
Willem M. Otte,
Quentin F. Gronau,
Bram Timmers,
Alexander Ly,
Eric-Jan Wagenmakers
Abstract:
Bayesian model-averaged meta-analysis allows quantification of evidence for both treatment effectiveness $μ$ and across-study heterogeneity $τ$. We use the Cochrane Database of Systematic Reviews to develop discipline-wide empirical prior distributions for $μ$ and $τ$ for meta-analyses of binary and time-to-event clinical trial outcomes. First, we use 50% of the database to estimate parameters of…
▽ More
Bayesian model-averaged meta-analysis allows quantification of evidence for both treatment effectiveness $μ$ and across-study heterogeneity $τ$. We use the Cochrane Database of Systematic Reviews to develop discipline-wide empirical prior distributions for $μ$ and $τ$ for meta-analyses of binary and time-to-event clinical trial outcomes. First, we use 50% of the database to estimate parameters of different required parametric families. Second, we use the remaining 50% of the database to select the best-performing parametric families and explore essential assumptions about the presence or absence of the treatment effectiveness and across-study heterogeneity in real data. We find that most meta-analyses of binary outcomes are more consistent with the absence of the meta-analytic effect or heterogeneity while meta-analyses of time-to-event outcomes are more consistent with the presence of the meta-analytic effect or heterogeneity. Finally, we use the complete database - with close to half a million trial outcomes - to propose specific empirical prior distributions, both for the field in general and for specific medical subdisciplines. An example from acute respiratory infections demonstrates how the proposed prior distributions can be used to conduct a Bayesian model-averaged meta-analysis in the open-source software R and JASP.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics
Authors:
František Bartoš,
Maximilian Maier,
Eric-Jan Wagenmakers,
Franziska Nippold,
Hristos Doucouliagos,
John P. A. Ioannidis,
Willem M. Otte,
Martina Sladekova,
Teshome K. Deresssa,
Stephan B. Bruns,
Daniele Fanelli,
T. D. Stanley
Abstract:
Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely cont…
▽ More
Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.
△ Less
Submitted 26 September, 2023; v1 submitted 25 August, 2022;
originally announced August 2022.
-
Bayesian Model-Averaged Meta-Analysis in Medicine
Authors:
František Bartoš,
Quentin F. Gronau,
Bram Timmers,
Willem M. Otte,
Alexander Ly,
Eric-Jan Wagenmakers
Abstract:
We outline a Bayesian model-averaged meta-analysis for standardized mean differences in order to quantify evidence for both treatment effectiveness $δ$ and across-study heterogeneity $τ$. We construct four competing models by orthogonally combining two present-absent assumptions, one for the treatment effect and one for across-study heterogeneity. To inform the choice of prior distributions for th…
▽ More
We outline a Bayesian model-averaged meta-analysis for standardized mean differences in order to quantify evidence for both treatment effectiveness $δ$ and across-study heterogeneity $τ$. We construct four competing models by orthogonally combining two present-absent assumptions, one for the treatment effect and one for across-study heterogeneity. To inform the choice of prior distributions for the model parameters, we used 50% of the Cochrane Database of Systematic Reviews to specify rival prior distributions for $δ$ and $τ$. The relative predictive performance of the competing models and rival prior distributions was assessed using the remaining 50\% of the Cochrane Database. On average, $\mathcal{H}_1^r$ -- the model that assumes the presence of a treatment effect as well as across-study heterogeneity -- outpredicted the other models, but not by a large margin. Within $\mathcal{H}_1^r$, predictive adequacy was relatively constant across the rival prior distributions. We propose specific empirical prior distributions, both for the field in general and for each of 46 specific medical subdisciplines. An example from oral health demonstrates how the proposed prior distributions can be used to conduct a Bayesian model-averaged meta-analysis in the open-source software R and JASP. The preregistered analysis plan is available at https://osf.io/zs3df/.
△ Less
Submitted 3 October, 2021;
originally announced October 2021.
