Showing 1–10 of 10 results for author: Armstrong, C

Development and Validation of a Deep-Learning Model for Differential Treatment Benefit Prediction for Adults with Major Depressive Disorder Deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study

Authors: David Benrimoh, Caitrin Armstrong, Joseph Mehltretter, Robert Fratila, Kelly Perlman, Sonia Israel, Adam Kapelner, Sagar V. Parikh, Jordan F. Karp, Katherine Heller, Gustavo Turecki

Abstract: INTRODUCTION: The pharmacological treatment of Major Depressive Disorder (MDD) relies on a trial-and-error approach. We introduce an artificial intelligence (AI) model aiming to personalize treatment and improve outcomes, which was deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study. OBJECTIVES: 1) Develop a model capable of predicting probabilities of remiss… ▽ More INTRODUCTION: The pharmacological treatment of Major Depressive Disorder (MDD) relies on a trial-and-error approach. We introduce an artificial intelligence (AI) model aiming to personalize treatment and improve outcomes, which was deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study. OBJECTIVES: 1) Develop a model capable of predicting probabilities of remission across multiple pharmacological treatments for adults with at least moderate major depression. 2) Validate model predictions and examine them for amplification of harmful biases. METHODS: Data from previous clinical trials of antidepressant medications were standardized into a common framework and included 9,042 adults with moderate to severe major depression. Feature selection retained 25 clinical and demographic variables. Using Bayesian optimization, a deep learning model was trained on the training set, refined using the validation set, and tested once on the held-out test set. RESULTS: In the evaluation on the held-out test set, the model demonstrated achieved an AUC of 0.65. The model outperformed a null model on the test set (p = 0.01). The model demonstrated clinical utility, achieving an absolute improvement in population remission rate in hypothetical and actual improvement testing. While the model did identify one drug (escitalopram) as generally outperforming the other drugs (consistent with the input data), there was otherwise significant variation in drug rankings. On bias testing, the model did not amplify potentially harmful biases. CONCLUSIONS: We demonstrate the first model capable of predicting outcomes for 10 different treatment options for patients with MDD, intended to be used at or near the start of treatment to personalize treatment. The model was put into clinical practice during the AIDME randomized controlled trial whose results are reported separately. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Towards Outcome-Driven Patient Subgroups: A Machine Learning Analysis Across Six Depression Treatment Studies

Authors: David Benrimoh, Akiva Kleinerman, Toshi A. Furukawa, Charles F. Reynolds III, Eric Lenze, Jordan Karp, Benoit Mulsant, Caitrin Armstrong, Joseph Mehltretter, Robert Fratila, Kelly Perlman, Sonia Israel, Myriam Tanguay-Sela, Christina Popescu, Grace Golden, Sabrina Qassim, Alexandra Anacleto, Adam Kapelner, Ariel Rosenfeld, Gustavo Turecki

Abstract: Major depressive disorder (MDD) is a heterogeneous condition; multiple underlying neurobiological substrates could be associated with treatment response variability. Understanding the sources of this variability and predicting outcomes has been elusive. Machine learning has shown promise in predicting treatment response in MDD, but one limitation has been the lack of clinical interpretability of m… ▽ More Major depressive disorder (MDD) is a heterogeneous condition; multiple underlying neurobiological substrates could be associated with treatment response variability. Understanding the sources of this variability and predicting outcomes has been elusive. Machine learning has shown promise in predicting treatment response in MDD, but one limitation has been the lack of clinical interpretability of machine learning models. We analyzed data from six clinical trials of pharmacological treatment for depression (total n = 5438) using the Differential Prototypes Neural Network (DPNN), a neural network model that derives patient prototypes which can be used to derive treatment-relevant patient clusters while learning to generate probabilities for differential treatment response. A model classifying remission and outputting individual remission probabilities for five first-line monotherapies and three combination treatments was trained using clinical and demographic data. Model validity and clinical utility were measured based on area under the curve (AUC) and expected improvement in sample remission rate with model-guided treatment, respectively. Post-hoc analyses yielded clusters (subgroups) based on patient prototypes learned during training. Prototypes were evaluated for interpretability by assessing differences in feature distributions and treatment-specific outcomes. A 3-prototype model achieved an AUC of 0.66 and an expected absolute improvement in population remission rate compared to the sample remission rate. We identified three treatment-relevant patient clusters which were clinically interpretable. It is possible to produce novel treatment-relevant patient profiles using machine learning models; doing so may improve precision medicine for depression. Note: This model is not currently the subject of any active clinical trials and is not intended for clinical use. △ Less

Submitted 30 March, 2023; v1 submitted 24 March, 2023; originally announced March 2023.

Applying Artificial Intelligence to Clinical Decision Support in Mental Health: What Have We Learned?

Authors: Grace Golden, Christina Popescu, Sonia Israel, Kelly Perlman, Caitrin Armstrong, Robert Fratila, Myriam Tanguay-Sela, David Benrimoh

Abstract: Clinical decision support systems (CDSS) augmented with artificial intelligence (AI) models are emerging as potentially valuable tools in healthcare. Despite their promise, the development and implementation of these systems typically encounter several barriers, hindering the potential for widespread adoption. Here we present a case study of a recently developed AI-CDSS, Aifred Health, aimed at su… ▽ More Clinical decision support systems (CDSS) augmented with artificial intelligence (AI) models are emerging as potentially valuable tools in healthcare. Despite their promise, the development and implementation of these systems typically encounter several barriers, hindering the potential for widespread adoption. Here we present a case study of a recently developed AI-CDSS, Aifred Health, aimed at supporting the selection and management of treatment in major depressive disorder. We consider both the principles espoused during development and testing of this AI-CDSS, as well as the practical solutions developed to facilitate implementation. We also propose recommendations to consider throughout the building, validation, training, and implementation process of an AI-CDSS. These recommendations include: identifying the key problem, selecting the type of machine learning approach based on this problem, determining the type of data required, determining the format required for a CDSS to provide clinical utility, gathering physician and patient feedback, and validating the tool across multiple settings. Finally, we explore the potential benefits of widespread adoption of these systems, while balancing these against implementation challenges such as ensuring systems do not disrupt the clinical workflow, and designing systems in a manner that engenders trust on the part of end users. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making

Authors: Oskar Wysocki, Jessica Katharine Davies, Markel Vigo, Anne Caroline Armstrong, Dónal Landers, Rebecca Lee, André Freitas

Abstract: This paper contributes with a pragmatic evaluation framework for explainable Machine Learning (ML) models for clinical decision support. The study revealed a more nuanced role for ML explanation models, when these are pragmatically embedded in the clinical context. Despite the general positive attitude of healthcare professionals (HCPs) towards explanations as a safety and trust mechanism, for a s… ▽ More This paper contributes with a pragmatic evaluation framework for explainable Machine Learning (ML) models for clinical decision support. The study revealed a more nuanced role for ML explanation models, when these are pragmatically embedded in the clinical context. Despite the general positive attitude of healthcare professionals (HCPs) towards explanations as a safety and trust mechanism, for a significant set of participants there were negative effects associated with confirmation bias, accentuating model over-reliance and increased effort to interact with the model. Also, contradicting one of its main intended functions, standard explanatory models showed limited ability to support a critical understanding of the limitations of the model. However, we found new significant positive effects which repositions the role of explanations within a clinical context: these include reduction of automation bias, addressing ambiguous clinical cases (cases where HCPs were not certain about their decision) and support of less experienced HCPs in the acquisition of new domain knowledge. △ Less

Submitted 27 October, 2022; v1 submitted 11 April, 2022; originally announced April 2022.

Comments: supplementary information in the main pdf

Smartphone-based Hard-braking Event Detection at Scale for Road Safety Services

Authors: Luyang Liu, David Racz, Kara Vaillancourt, Julie Michelman, Matt Barnes, Stefan Mellem, Paul Eastham, Bradley Green, Charles Armstrong, Rishi Bal, Shawn O'Banion, Feng Guo

Abstract: Road crashes are the sixth leading cause of lost disability-adjusted life-years (DALYs) worldwide. One major challenge in traffic safety research is the sparsity of crashes, which makes it difficult to achieve a fine-grain understanding of crash causations and predict future crash risk in a timely manner. Hard-braking events have been widely used as a safety surrogate due to their relatively high… ▽ More Road crashes are the sixth leading cause of lost disability-adjusted life-years (DALYs) worldwide. One major challenge in traffic safety research is the sparsity of crashes, which makes it difficult to achieve a fine-grain understanding of crash causations and predict future crash risk in a timely manner. Hard-braking events have been widely used as a safety surrogate due to their relatively high prevalence and ease of detection with embedded vehicle sensors. As an alternative to using sensors fixed in vehicles, this paper presents a scalable approach for detecting hard-braking events using the kinematics data collected from smartphone sensors. We train a Transformer-based machine learning model for hard-braking event detection using concurrent sensor readings from smartphones and vehicle sensors from drivers who connect their phone to the vehicle while navigating in Google Maps. The detection model shows superior performance with a $0.83$ Area under the Precision-Recall Curve (PR-AUC), which is $3.8\times$better than a GPS speed-based heuristic model, and $166.6\times$better than an accelerometer-based heuristic model. The detected hard-braking events are strongly correlated with crashes from publicly available datasets, supporting their use as a safety surrogate. In addition, we conduct model fairness and selection bias evaluation to ensure that the safety benefits are equally shared. The developed methodology can benefit many safety applications such as identifying safety hot spots at road network level, evaluating the safety of new user interfaces, as well as using routing to improve traffic safety. △ Less

Submitted 3 February, 2022; originally announced February 2022.

Legends: Folklore on Reddit

Authors: Caitrin Armstrong, Derek Ruths

Abstract: In this paper we introduce Reddit legends, a collection of venerated old posts that have become famous on Reddit. To establish the utility of Reddit legends for both computational science/HCI and folkloristics, we investigate two main questions: (1) whether they can be considered folklore, i.e. if they have consistent form, cultural significance, and undergo spontaneous transmission, and (2) wheth… ▽ More In this paper we introduce Reddit legends, a collection of venerated old posts that have become famous on Reddit. To establish the utility of Reddit legends for both computational science/HCI and folkloristics, we investigate two main questions: (1) whether they can be considered folklore, i.e. if they have consistent form, cultural significance, and undergo spontaneous transmission, and (2) whether they can be studied in a systematic manner. Through several subtasks, including the creation of a typology, an analysis of references to Reddit legends, and an examination of some of the textual characteristics of referencing behaviour, we show that Reddit legends can indeed be considered as folklore and that they are amendable to systematic text-based approaches. We discuss how these results will enable future analyses of folklore on Reddit, including tracking subreddit-wide and individual-user behaviour, and the relationship of this behaviour to other cultural markers. △ Less

Submitted 1 July, 2020; originally announced July 2020.

The Residence History Inference Problem

Authors: Derek Ruths, Caitrin Armstrong

Abstract: The use of online user traces for studies of human mobility has received significant attention in recent years. This growing body of work, and the more general importance of human migration patterns to government and industry, motivates the need for a formalized approach to the computational modeling of human mobility - in particular how and when individuals change their place of residence - from… ▽ More The use of online user traces for studies of human mobility has received significant attention in recent years. This growing body of work, and the more general importance of human migration patterns to government and industry, motivates the need for a formalized approach to the computational modeling of human mobility - in particular how and when individuals change their place of residence - from online traces. Prior work on this topic has skirted the underlying computational modeling of residence inference, focusing on migration patterns themselves. As a result, to our knowledge, all prior work has employed heuristics to compute something like residence histories. Here, we formalize the residence assignment problem, which seeks, under constraints associated with the minimum length-of-stay at a residence, the most parsimonious sequence of residence periods and places that explains the movement history of an individual. Here we provide an exact solution for this problem and establish its algorithmic complexity. Because the calculation of optimal residence histories (under the assumptions of the model) is tractable, we believe that this method will be a valuable tool for future work on this topic. △ Less

Submitted 9 March, 2020; originally announced March 2020.

Big Data Analytics and AI in Mental Healthcare

Authors: Ariel Rosenfeld, David Benrimoh, Caitrin Armstrong, Nykan Mirchi, Timothe Langlois-Therrien, Colleen Rollins, Myriam Tanguay-Sela, Joseph Mehltretter, Robert Fratila, Sonia Israel, Emily Snook, Kelly Perlman, Akiva Kleinerman, Bechara Saab, Mark Thoburn, Cheryl Gabbay, Amit Yaniv-Rosenfeld

Abstract: Mental health conditions cause a great deal of distress or impairment; depression alone will affect 11% of the world's population. The application of Artificial Intelligence (AI) and big-data technologies to mental health has great potential for personalizing treatment selection, prognosticating, monitoring for relapse, detecting and hel** to prevent mental health conditions before they reach cl… ▽ More Mental health conditions cause a great deal of distress or impairment; depression alone will affect 11% of the world's population. The application of Artificial Intelligence (AI) and big-data technologies to mental health has great potential for personalizing treatment selection, prognosticating, monitoring for relapse, detecting and hel** to prevent mental health conditions before they reach clinical-level symptomatology, and even delivering some treatments. However, unlike similar applications in other fields of medicine, there are several unique challenges in mental health applications which currently pose barriers towards the implementation of these technologies. Specifically, there are very few widely used or validated biomarkers in mental health, leading to a heavy reliance on patient and clinician derived questionnaire data as well as interpretation of new signals such as digital phenoty**. In addition, diagnosis also lacks the same objective 'gold standard' as in other conditions such as oncology, where clinicians and researchers can often rely on pathological analysis for confirmation of diagnosis. In this chapter we discuss the major opportunities, limitations and techniques used for improving mental healthcare through AI and big-data. We explore both the computational, clinical and ethical considerations and best practices as well as lay out the major researcher directions for the near future. △ Less

Submitted 12 March, 2019; originally announced March 2019.

Comments: Chapter in the "Big Data in Healthcare" book (Elsevier) [exp. 2019]

Characterizing short-term stability for Boolean networks over any distribution of transfer functions

Authors: C. Seshadhri, Andrew M. Smith, Yevgeniy Vorobeychik, Jackson Mayo, Robert C. Armstrong

Abstract: We present a characterization of short-term stability of random Boolean networks under \emph{arbitrary} distributions of transfer functions. Given any distribution of transfer functions for a random Boolean network, we present a formula that decides whether short-term chaos (damage spreading) will happen. We provide a formal proof for this formula, and empirically show that its predictions are acc… ▽ More We present a characterization of short-term stability of random Boolean networks under \emph{arbitrary} distributions of transfer functions. Given any distribution of transfer functions for a random Boolean network, we present a formula that decides whether short-term chaos (damage spreading) will happen. We provide a formal proof for this formula, and empirically show that its predictions are accurate. Previous work only works for special cases of balanced families. It has been observed that these characterizations fail for unbalanced families, yet such families are widespread in real biological networks. △ Less

Submitted 15 September, 2014; originally announced September 2014.

Journal ref: Phys. Rev. E 94, 012301 (2016)

Influence and Dynamic Behavior in Random Boolean Networks

Authors: C. Seshadhri, Yevgeniy Vorobeychik, Jackson R. Mayo, Robert C. Armstrong, Joseph R. Ruthruff

Abstract: We present a rigorous mathematical framework for analyzing dynamics of a broad class of Boolean network models. We use this framework to provide the first formal proof of many of the standard critical transition results in Boolean network analysis, and offer analogous characterizations for novel classes of random Boolean networks. We precisely connect the short-run dynamic behavior of a Boolean ne… ▽ More We present a rigorous mathematical framework for analyzing dynamics of a broad class of Boolean network models. We use this framework to provide the first formal proof of many of the standard critical transition results in Boolean network analysis, and offer analogous characterizations for novel classes of random Boolean networks. We precisely connect the short-run dynamic behavior of a Boolean network to the average influence of the transfer functions. We show that some of the assumptions traditionally made in the more common mean-field analysis of Boolean networks do not hold in general. For example, we offer some evidence that imbalance, or expected internal inhomogeneity, of transfer functions is a crucial feature that tends to drive quiescent behavior far more strongly than previously observed. △ Less

Submitted 19 July, 2011; originally announced July 2011.

Comments: To appear as a Letter in Physical Review Letters 8 pages, 4 figures

Journal ref: Phys. Rev. Lett. 107, 108701 (2011)