-
Multi-site, Multi-domain Airway Tree Modeling (ATM'22): A Public Benchmark for Pulmonary Airway Segmentation
Authors:
Minghui Zhang,
Yangqian Wu,
Hanxiao Zhang,
Yulei Qin,
Hao Zheng,
Wen Tang,
Corey Arnold,
Chenhao Pei,
Pengxin Yu,
Yang Nan,
Guang Yang,
Simon Walsh,
Dominic C. Marshall,
Matthieu Komorowski,
Puyang Wang,
Dazhou Guo,
Dakai **,
Ya'nan Wu,
Shuiqing Zhao,
Runsheng Chang,
Boyu Zhang,
Xing Lv,
Abdul Qayyum,
Moona Mazher,
Qi Su
, et al. (11 additional authors not shown)
Abstract:
Open international challenges are becoming the de facto standard for assessing computer vision and image analysis algorithms. In recent years, new methods have extended the reach of pulmonary airway segmentation that is closer to the limit of image resolution. Since EXACT'09 pulmonary airway segmentation, limited effort has been directed to quantitative comparison of newly emerged algorithms drive…
▽ More
Open international challenges are becoming the de facto standard for assessing computer vision and image analysis algorithms. In recent years, new methods have extended the reach of pulmonary airway segmentation that is closer to the limit of image resolution. Since EXACT'09 pulmonary airway segmentation, limited effort has been directed to quantitative comparison of newly emerged algorithms driven by the maturity of deep learning based approaches and clinical drive for resolving finer details of distal airways for early intervention of pulmonary diseases. Thus far, public annotated datasets are extremely limited, hindering the development of data-driven methods and detailed performance evaluation of new algorithms. To provide a benchmark for the medical imaging community, we organized the Multi-site, Multi-domain Airway Tree Modeling (ATM'22), which was held as an official challenge event during the MICCAI 2022 conference. ATM'22 provides large-scale CT scans with detailed pulmonary airway annotation, including 500 CT scans (300 for training, 50 for validation, and 150 for testing). The dataset was collected from different sites and it further included a portion of noisy COVID-19 CTs with ground-glass opacity and consolidation. Twenty-three teams participated in the entire phase of the challenge and the algorithms for the top ten teams are reviewed in this paper. Quantitative and qualitative results revealed that deep learning models embedded with the topological continuity enhancement achieved superior performance in general. ATM'22 challenge holds as an open-call design, the training data and the gold standard evaluation are available upon successful registration via its homepage.
△ Less
Submitted 27 June, 2023; v1 submitted 10 March, 2023;
originally announced March 2023.
-
Beyond Low Earth Orbit: Biological Research, Artificial Intelligence, and Self-Driving Labs
Authors:
Lauren M. Sanders,
Jason H. Yang,
Ryan T. Scott,
Amina Ann Qutub,
Hector Garcia Martin,
Daniel C. Berrios,
Jaden J. A. Hastings,
Jon Rask,
Graham Mackintosh,
Adrienne L. Hoarfrost,
Stuart Chalk,
John Kalantari,
Kia Khezeli,
Erik L. Antonsen,
Joel Babdor,
Richard Barker,
Sergio E. Baranzini,
Afshin Beheshti,
Guillermo M. Delgado-Aparicio,
Benjamin S. Glicksberg,
Casey S. Greene,
Melissa Haendel,
Arif A. Hamid,
Philip Heller,
Daniel Jamieson
, et al. (31 additional authors not shown)
Abstract:
Space biology research aims to understand fundamental effects of spaceflight on organisms, develop foundational knowledge to support deep space exploration, and ultimately bioengineer spacecraft and habitats to stabilize the ecosystem of plants, crops, microbes, animals, and humans for sustained multi-planetary life. To advance these aims, the field leverages experiments, platforms, data, and mode…
▽ More
Space biology research aims to understand fundamental effects of spaceflight on organisms, develop foundational knowledge to support deep space exploration, and ultimately bioengineer spacecraft and habitats to stabilize the ecosystem of plants, crops, microbes, animals, and humans for sustained multi-planetary life. To advance these aims, the field leverages experiments, platforms, data, and model organisms from both spaceborne and ground-analog studies. As research is extended beyond low Earth orbit, experiments and platforms must be maximally autonomous, light, agile, and intelligent to expedite knowledge discovery. Here we present a summary of recommendations from a workshop organized by the National Aeronautics and Space Administration on artificial intelligence, machine learning, and modeling applications which offer key solutions toward these space biology challenges. In the next decade, the synthesis of artificial intelligence into the field of space biology will deepen the biological understanding of spaceflight effects, facilitate predictive modeling and analytics, support maximally autonomous and reproducible experiments, and efficiently manage spaceborne data and metadata, all with the goal to enable life to thrive in deep space.
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
-
Beyond Low Earth Orbit: Biomonitoring, Artificial Intelligence, and Precision Space Health
Authors:
Ryan T. Scott,
Erik L. Antonsen,
Lauren M. Sanders,
Jaden J. A. Hastings,
Seung-min Park,
Graham Mackintosh,
Robert J. Reynolds,
Adrienne L. Hoarfrost,
Aenor Sawyer,
Casey S. Greene,
Benjamin S. Glicksberg,
Corey A. Theriot,
Daniel C. Berrios,
Jack Miller,
Joel Babdor,
Richard Barker,
Sergio E. Baranzini,
Afshin Beheshti,
Stuart Chalk,
Guillermo M. Delgado-Aparicio,
Melissa Haendel,
Arif A. Hamid,
Philip Heller,
Daniel Jamieson,
Katelyn J. Jarvis
, et al. (31 additional authors not shown)
Abstract:
Human space exploration beyond low Earth orbit will involve missions of significant distance and duration. To effectively mitigate myriad space health hazards, paradigm shifts in data and space health systems are necessary to enable Earth-independence, rather than Earth-reliance. Promising developments in the fields of artificial intelligence and machine learning for biology and health can address…
▽ More
Human space exploration beyond low Earth orbit will involve missions of significant distance and duration. To effectively mitigate myriad space health hazards, paradigm shifts in data and space health systems are necessary to enable Earth-independence, rather than Earth-reliance. Promising developments in the fields of artificial intelligence and machine learning for biology and health can address these needs. We propose an appropriately autonomous and intelligent Precision Space Health system that will monitor, aggregate, and assess biomedical statuses; analyze and predict personalized adverse health outcomes; adapt and respond to newly accumulated data; and provide preventive, actionable, and timely insights to individual deep space crew members and iterative decision support to their crew medical officer. Here we present a summary of recommendations from a workshop organized by the National Aeronautics and Space Administration, on future applications of artificial intelligence in space biology and health. In the next decade, biomonitoring technology, biomarker science, spacecraft hardware, intelligent software, and streamlined data management must mature and be woven together into a Precision Space Health system to enable humanity to thrive in deep space.
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
-
ToFFi -- Toolbox for Frequency-based Fingerprinting of Brain Signals
Authors:
Michał K. Komorowski,
Krzysztof Rykaczewski,
Tomasz Piotrowski,
Katarzyna Jurewicz,
Jakub Wojciechowski,
Anne Keitel,
Joanna Dreszer,
Włodzisław Duch
Abstract:
Spectral fingerprints (SFs) are unique power spectra signatures of human brain regions of interest (ROIs, Keitel & Gross, 2016). SFs allow for accurate ROI identification and can serve as biomarkers of differences exhibited by non-neurotypical groups. At present, there are no open-source, versatile tools to calculate spectral fingerprints. We have filled this gap by creating a modular, highly-conf…
▽ More
Spectral fingerprints (SFs) are unique power spectra signatures of human brain regions of interest (ROIs, Keitel & Gross, 2016). SFs allow for accurate ROI identification and can serve as biomarkers of differences exhibited by non-neurotypical groups. At present, there are no open-source, versatile tools to calculate spectral fingerprints. We have filled this gap by creating a modular, highly-configurable MATLAB Toolbox for Frequency-based Fingerprinting (ToFFi). It can transform MEG/EEG signals into unique spectral representations using ROIs provided by anatomical (AAL, Desikan-Killiany), functional (Schaefer), or other custom volumetric brain parcellations. Toolbox design supports reproducibility and parallel computations.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
Enabling risk-aware Reinforcement Learning for medical interventions through uncertainty decomposition
Authors:
Paul Festor,
Giulia Luise,
Matthieu Komorowski,
A. Aldo Faisal
Abstract:
Reinforcement Learning (RL) is emerging as tool for tackling complex control and decision-making problems. However, in high-risk environments such as healthcare, manufacturing, automotive or aerospace, it is often challenging to bridge the gap between an apparently optimal policy learnt by an agent and its real-world deployment, due to the uncertainties and risk associated with it. Broadly speakin…
▽ More
Reinforcement Learning (RL) is emerging as tool for tackling complex control and decision-making problems. However, in high-risk environments such as healthcare, manufacturing, automotive or aerospace, it is often challenging to bridge the gap between an apparently optimal policy learnt by an agent and its real-world deployment, due to the uncertainties and risk associated with it. Broadly speaking RL agents face two kinds of uncertainty, 1. aleatoric uncertainty, which reflects randomness or noise in the dynamics of the world, and 2. epistemic uncertainty, which reflects the bounded knowledge of the agent due to model limitations and finite amount of information/data the agent has acquired about the world. These two types of uncertainty carry fundamentally different implications for the evaluation of performance and the level of risk or trust. Yet these aleatoric and epistemic uncertainties are generally confounded as standard and even distributional RL is agnostic to this difference. Here we propose how a distributional approach (UA-DQN) can be recast to render uncertainties by decomposing the net effects of each uncertainty. We demonstrate the operation of this method in grid world examples to build intuition and then show a proof of concept application for an RL agent operating as a clinical decision support system in critical care
△ Less
Submitted 27 April, 2022; v1 submitted 16 September, 2021;
originally announced September 2021.
-
Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPs
Authors:
Luchen Li,
Matthieu Komorowski,
Aldo A. Faisal
Abstract:
Health-related data is noisy and stochastic in implying the true physiological states of patients, limiting information contained in single-moment observations for sequential clinical decision making. We model patient-clinician interactions as partially observable Markov decision processes (POMDPs) and optimize sequential treatment based on belief states inferred from history sequence. To facilita…
▽ More
Health-related data is noisy and stochastic in implying the true physiological states of patients, limiting information contained in single-moment observations for sequential clinical decision making. We model patient-clinician interactions as partially observable Markov decision processes (POMDPs) and optimize sequential treatment based on belief states inferred from history sequence. To facilitate inference, we build a variational generative model and boost state representation with a recurrent neural network (RNN), incorporating an auxiliary loss from sequence auto-encoding. Meanwhile, we optimize a continuous policy of drug levels with an actor-critic method where policy gradients are obtained from a stablized off-policy estimate of advantage function, with the value of belief state backed up by parallel best-first suffix trees. We exploit our methodology in optimizing dosages of vasopressor and intravenous fluid for sepsis patients using a retrospective intensive care dataset and evaluate the learned policy with off-policy policy evaluation (OPPE). The results demonstrate that modelling as POMDPs yields better performance than MDPs, and that incorporating heuristic search improves sample efficiency.
△ Less
Submitted 17 May, 2019;
originally announced May 2019.
-
Understanding the Artificial Intelligence Clinician and optimal treatment strategies for sepsis in intensive care
Authors:
Matthieu Komorowski,
Leo A. Celi,
Omar Badawi,
Anthony C. Gordon,
A. Aldo Faisal
Abstract:
In this document, we explore in more detail our published work (Komorowski, Celi, Badawi, Gordon, & Faisal, 2018) for the benefit of the AI in Healthcare research community. In the above paper, we developed the AI Clinician system, which demonstrated how reinforcement learning could be used to make useful recommendations towards optimal treatment decisions from intensive care data. Since publicati…
▽ More
In this document, we explore in more detail our published work (Komorowski, Celi, Badawi, Gordon, & Faisal, 2018) for the benefit of the AI in Healthcare research community. In the above paper, we developed the AI Clinician system, which demonstrated how reinforcement learning could be used to make useful recommendations towards optimal treatment decisions from intensive care data. Since publication a number of authors have reviewed our work (e.g. Abbasi, 2018; Bos, Azoulay, & Martin-Loeches, 2019; Saria, 2018). Given the difference of our framework to previous work, the fact that we are bridging two very different academic communities (intensive care and machine learning) and that our work has impact on a number of other areas with more traditional computer-based approaches (biosignal processing and control, biomedical engineering), we are providing here additional details on our recent publication.
△ Less
Submitted 6 March, 2019;
originally announced March 2019.
-
Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning
Authors:
Xuefeng Peng,
Yi Ding,
David Wihl,
Omer Gottesman,
Matthieu Komorowski,
Li-wei H. Lehman,
Andrew Ross,
Aldo Faisal,
Finale Doshi-Velez
Abstract:
Sepsis is the leading cause of mortality in the ICU. It is challenging to manage because individual patients respond differently to treatment. Thus, tailoring treatment to the individual patient is essential for the best outcomes. In this paper, we take steps toward this goal by applying a mixture-of-experts framework to personalize sepsis treatment. The mixture model selectively alternates betwee…
▽ More
Sepsis is the leading cause of mortality in the ICU. It is challenging to manage because individual patients respond differently to treatment. Thus, tailoring treatment to the individual patient is essential for the best outcomes. In this paper, we take steps toward this goal by applying a mixture-of-experts framework to personalize sepsis treatment. The mixture model selectively alternates between neighbor-based (kernel) and deep reinforcement learning (DRL) experts depending on patient's current history. On a large retrospective cohort, this mixture-based approach outperforms physician, kernel only, and DRL-only experts.
△ Less
Submitted 15 January, 2019;
originally announced January 2019.
-
Model-Based Reinforcement Learning for Sepsis Treatment
Authors:
Aniruddh Raghu,
Matthieu Komorowski,
Sumeetpal Singh
Abstract:
Sepsis is a dangerous condition that is a leading cause of patient mortality. Treating sepsis is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we explore the use of continuous state-space model-based reinforcement learning (RL) to discover high-quality treatment policies…
▽ More
Sepsis is a dangerous condition that is a leading cause of patient mortality. Treating sepsis is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we explore the use of continuous state-space model-based reinforcement learning (RL) to discover high-quality treatment policies for sepsis patients. Our quantitative evaluation reveals that by blending the treatment strategy discovered with RL with what clinicians follow, we can obtain improved policies, potentially allowing for better medical treatment for sepsis.
△ Less
Submitted 23 November, 2018;
originally announced November 2018.
-
Information-theoretic analysis of multivariate single - cell signaling responses using SLEMI
Authors:
Tomasz Jetka,
Tomasz Winarski,
Karol Nienaltowski,
Slawomir Blonski,
Michal Komorowski
Abstract:
Mathematical methods of information theory constitute essential tools to describe how stimuli are encoded in activities of signaling effectors. Exploring the information-theoretic perspective, however, remains conceptually, experimentally and computationally challenging. Specifically, existing computational tools enable efficient analysis of relatively simple systems, usually with one input and ou…
▽ More
Mathematical methods of information theory constitute essential tools to describe how stimuli are encoded in activities of signaling effectors. Exploring the information-theoretic perspective, however, remains conceptually, experimentally and computationally challenging. Specifically, existing computational tools enable efficient analysis of relatively simple systems, usually with one input and output only. Moreover, their robust and readily applicable implementations are missing. Here, we propose a novel algorithm to analyze signaling data within the framework of information theory. Our approach enables robust as well as statistically and computationally efficient analysis of signaling systems with high-dimensional outputs and a large number of input values. Analysis of the NF-kB single - cell signaling responses to TNF-a uniquely reveals that the NF-kB signaling dynamics improves discrimination of high concentrations of TNF-a with a modest impact on discrimination of low concentrations. Our readily applicable R-package, SLEMI - statistical learning based estimation of mutual information, allows the approach to be used by computational biologists with only elementary knowledge of information theory.
△ Less
Submitted 16 August, 2018;
originally announced August 2018.
-
Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters
Authors:
Aniruddh Raghu,
Omer Gottesman,
Yao Liu,
Matthieu Komorowski,
Aldo Faisal,
Finale Doshi-Velez,
Emma Brunskill
Abstract:
In this work, we consider the problem of estimating a behaviour policy for use in Off-Policy Policy Evaluation (OPE) when the true behaviour policy is unknown. Via a series of empirical studies, we demonstrate how accurate OPE is strongly dependent on the calibration of estimated behaviour policy models: how precisely the behaviour policy is estimated from data. We show how powerful parametric mod…
▽ More
In this work, we consider the problem of estimating a behaviour policy for use in Off-Policy Policy Evaluation (OPE) when the true behaviour policy is unknown. Via a series of empirical studies, we demonstrate how accurate OPE is strongly dependent on the calibration of estimated behaviour policy models: how precisely the behaviour policy is estimated from data. We show how powerful parametric models such as neural networks can result in highly uncalibrated behaviour policy models on a real-world medical dataset, and illustrate how a simple, non-parametric, k-nearest neighbours model produces better calibrated behaviour policy estimates and can be used to obtain superior importance sampling-based OPE estimates.
△ Less
Submitted 10 July, 2018; v1 submitted 3 July, 2018;
originally announced July 2018.
-
Evaluating Reinforcement Learning Algorithms in Observational Health Settings
Authors:
Omer Gottesman,
Fredrik Johansson,
Joshua Meier,
Jack Dent,
Donghun Lee,
Srivatsan Srinivasan,
Linying Zhang,
Yi Ding,
David Wihl,
Xuefeng Peng,
Jiayu Yao,
Isaac Lage,
Christopher Mosch,
Li-wei H. Lehman,
Matthieu Komorowski,
Matthieu Komorowski,
Aldo Faisal,
Leo Anthony Celi,
David Sontag,
Finale Doshi-Velez
Abstract:
Much attention has been devoted recently to the development of machine learning algorithms with the goal of improving treatment policies in healthcare. Reinforcement learning (RL) is a sub-field within machine learning that is concerned with learning how to make sequences of decisions so as to optimize long-term effects. Already, RL algorithms have been proposed to identify decision-making strateg…
▽ More
Much attention has been devoted recently to the development of machine learning algorithms with the goal of improving treatment policies in healthcare. Reinforcement learning (RL) is a sub-field within machine learning that is concerned with learning how to make sequences of decisions so as to optimize long-term effects. Already, RL algorithms have been proposed to identify decision-making strategies for mechanical ventilation, sepsis management and treatment of schizophrenia. However, before implementing treatment policies learned by black-box algorithms in high-stakes clinical decision problems, special care must be taken in the evaluation of these policies.
In this document, our goal is to expose some of the subtleties associated with evaluating RL algorithms in healthcare. We aim to provide a conceptual starting point for clinical and computational researchers to ask the right questions when designing and evaluating algorithms for new ways of treating patients. In the following, we describe how choices about how to summarize a history, variance of statistical estimators, and confounders in more ad-hoc measures can result in unreliable, even misleading estimates of the quality of a treatment policy. We also provide suggestions for mitigating these effects---for while there is much promise for mining observational health data to uncover better treatment policies, evaluation must be performed thoughtfully.
△ Less
Submitted 30 May, 2018;
originally announced May 2018.
-
The Actor Search Tree Critic (ASTC) for Off-Policy POMDP Learning in Medical Decision Making
Authors:
Luchen Li,
Matthieu Komorowski,
Aldo A. Faisal
Abstract:
Off-policy reinforcement learning enables near-optimal policy from suboptimal experience, thereby provisions opportunity for artificial intelligence applications in healthcare. Previous works have mainly framed patient-clinician interactions as Markov decision processes, while true physiological states are not necessarily fully observable from clinical data. We capture this situation with partiall…
▽ More
Off-policy reinforcement learning enables near-optimal policy from suboptimal experience, thereby provisions opportunity for artificial intelligence applications in healthcare. Previous works have mainly framed patient-clinician interactions as Markov decision processes, while true physiological states are not necessarily fully observable from clinical data. We capture this situation with partially observable Markov decision process, in which an agent optimises its actions in a belief represented as a distribution of patient states inferred from individual history trajectories. A Gaussian mixture model is fitted for the observed data. Moreover, we take into account the fact that nuance in pharmaceutical dosage could presumably result in significantly different effect by modelling a continuous policy through a Gaussian approximator directly in the policy space, i.e. the actor. To address the challenge of infinite number of possible belief states which renders exact value iteration intractable, we evaluate and plan for only every encountered belief, through heuristic search tree by tightly maintaining lower and upper bounds of the true value of belief. We further resort to function approximations to update value bounds estimation, i.e. the critic, so that the tree search can be improved through more compact bounds at the fringe nodes that will be back-propagated to the root. Both actor and critic parameters are learned via gradient-based approaches. Our proposed policy trained from real intensive care unit data is capable of dictating dosing on vasopressors and intravenous fluids for sepsis patients that lead to the best patient outcomes.
△ Less
Submitted 3 June, 2018; v1 submitted 29 May, 2018;
originally announced May 2018.
-
Representation Balancing MDPs for Off-Policy Policy Evaluation
Authors:
Yao Liu,
Omer Gottesman,
Aniruddh Raghu,
Matthieu Komorowski,
Aldo Faisal,
Finale Doshi-Velez,
Emma Brunskill
Abstract:
We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value estimates from MDP models. Using this upper bound as an objective, we develop a l…
▽ More
We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value estimates from MDP models. Using this upper bound as an objective, we develop a learning algorithm of an MDP model with a balanced representation, and show that our approach can yield substantially lower MSE in common synthetic benchmarks and a HIV treatment simulation domain.
△ Less
Submitted 17 April, 2019; v1 submitted 23 May, 2018;
originally announced May 2018.
-
Deep Reinforcement Learning for Sepsis Treatment
Authors:
Aniruddh Raghu,
Matthieu Komorowski,
Imran Ahmed,
Leo Celi,
Peter Szolovits,
Marzyeh Ghassemi
Abstract:
Sepsis is a leading cause of mortality in intensive care units and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we propose an approach to deduce treatment policies for septic patients by using continuous sta…
▽ More
Sepsis is a leading cause of mortality in intensive care units and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we propose an approach to deduce treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Our model learns clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. The learned policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.
△ Less
Submitted 27 November, 2017;
originally announced November 2017.
-
Random Binary Trees for Approximate Nearest Neighbour Search in Binary Space
Authors:
Michal Komorowski,
Tomasz Trzcinski
Abstract:
Approximate nearest neighbour (ANN) search is one of the most important problems in computer science fields such as data mining or computer vision. In this paper, we focus on ANN for high-dimensional binary vectors and we propose a simple yet powerful search method that uses Random Binary Search Trees (RBST). We apply our method to a dataset of 1.25M binary local feature descriptors obtained from…
▽ More
Approximate nearest neighbour (ANN) search is one of the most important problems in computer science fields such as data mining or computer vision. In this paper, we focus on ANN for high-dimensional binary vectors and we propose a simple yet powerful search method that uses Random Binary Search Trees (RBST). We apply our method to a dataset of 1.25M binary local feature descriptors obtained from a real-life image-based localisation system provided by Google as a part of Project Tango. An extensive evaluation of our method against the state-of-the-art variations of Locality Sensitive Hashing (LSH), namely Uniform LSH and Multi-probe LSH, shows the superiority of our method in terms of retrieval precision with performance boost of over 20%
△ Less
Submitted 24 March, 2019; v1 submitted 9 August, 2017;
originally announced August 2017.
-
Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach
Authors:
Aniruddh Raghu,
Matthieu Komorowski,
Leo Anthony Celi,
Peter Szolovits,
Marzyeh Ghassemi
Abstract:
Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient's physiological state at a given time could hold the key to effecti…
▽ More
Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient's physiological state at a given time could hold the key to effective treatment policies. In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Learning treatment policies over continuous spaces is important, because we retain more of the patient's physiological information. Our model is able to learn clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. Evaluating our algorithm on past ICU patient data, we find that our model could reduce patient mortality in the hospital by up to 3.6% over observed clinical policies, from a baseline mortality of 13.7%. The learned treatment policies could be used to aid intensive care clinicians in medical decision making and improve the likelihood of patient survival.
△ Less
Submitted 23 May, 2017;
originally announced May 2017.
-
Quantifying intrinsic and extrinsic noise in gene transcription using the linear noise approximation: An application to single cell data
Authors:
Bärbel Finkenstädt,
Dan J. Woodcock,
Michal Komorowski,
Claire V. Harper,
Julian R. E. Davis,
Mike R. H. White,
David A. Rand
Abstract:
A central challenge in computational modeling of dynamic biological systems is parameter inference from experimental time course measurements. However, one would not only like to infer kinetic parameters but also study their variability from cell to cell. Here we focus on the case where single-cell fluorescent protein imaging time series data are available for a population of cells. Based on van K…
▽ More
A central challenge in computational modeling of dynamic biological systems is parameter inference from experimental time course measurements. However, one would not only like to infer kinetic parameters but also study their variability from cell to cell. Here we focus on the case where single-cell fluorescent protein imaging time series data are available for a population of cells. Based on van Kampen's linear noise approximation, we derive a dynamic state space model for molecular populations which is then extended to a hierarchical model. This model has potential to address the sources of variability relevant to single-cell data, namely, intrinsic noise due to the stochastic nature of the birth and death processes involved in reactions and extrinsic noise arising from the cell-to-cell variation of kinetic parameters. In order to infer such a model from experimental data, one must also quantify the measurement process where one has to allow for nonmeasurable molecular species as well as measurement noise of unknown level and variance. The availability of multiple single-cell time series data here provides a unique testbed to fit such a model and quantify these different sources of variation from experimental data.
△ Less
Submitted 8 January, 2014;
originally announced January 2014.
-
StochDecomp - Matlab package for noise decomposition in stochastic biochemical systems
Authors:
Tomasz Jetka,
Agata Charzynska,
Anna Gambin,
Michael P. H. Stumpf,
Michal Komorowski
Abstract:
Stochasticity is an indispensable aspect of biochemical processes at the cellular level. Studies on how the noise enters and propagates in biochemical systems provided us with nontrivial insights into the origins of stochasticity, in total however they constitute a patchwork of different theoretical analyses. Here we present a flexible and generally applicable noise decomposition tool, that allows…
▽ More
Stochasticity is an indispensable aspect of biochemical processes at the cellular level. Studies on how the noise enters and propagates in biochemical systems provided us with nontrivial insights into the origins of stochasticity, in total however they constitute a patchwork of different theoretical analyses. Here we present a flexible and generally applicable noise decomposition tool, that allows us to calculate contributions of individual reactions to the total variability of a system's output. With the package it is therefore possible to quantify how the noise enters and propagates in biochemical systems. We also demonstrate and exemplify using the JAK-STAT signalling pathway that it is possible to infer noise contributions resulting from individual reactions directly from experimental data. This is the first computational tool that allows to decompose noise into contributions resulting from individual reactions.
△ Less
Submitted 14 August, 2013;
originally announced August 2013.
-
Functional redundancy in the NF-κB signalling pathway
Authors:
Michał Włodarczyk,
Tomasz Lipniacki,
Michał Komorowski
Abstract:
The ability to represent intracellular biochemical dynamics via deterministic and stochastic modelling is one of the crucial components to move biological sciences in the observe-predict-control-design knowledge ladder. Compared to the engineering or physics problems, dynamical models in quantitative biology typically dependent on a relatively large number of parameters. Therefore, the relationshi…
▽ More
The ability to represent intracellular biochemical dynamics via deterministic and stochastic modelling is one of the crucial components to move biological sciences in the observe-predict-control-design knowledge ladder. Compared to the engineering or physics problems, dynamical models in quantitative biology typically dependent on a relatively large number of parameters. Therefore, the relationship between model parameters and dynamics is often prohibitively difficult to determine. We developed a method to depict the input-output relationship for multi-parametric stochastic and deterministic models via information-theoretic quantification of similarity between model parameters and modules. Identification of most information-theoretically orthogonal biological components, provided mathematical language to precisely communicate and visualise compensation like phenomena such as biological robustness, sloppiness and statistical non-identifiability. A comprehensive analysis of the multi-parameter NF-$κ$B signalling pathway demonstrates that the information-theoretic similarity reflects a topological structure of the network. Examination of the currently available experimental data on this system reveals the number of identifiable parameters and suggests informative experimental protocols.
△ Less
Submitted 13 March, 2013;
originally announced March 2013.
-
Dependence of MAPK mediated signaling on Erk isoforms and differences in nuclear shuttling
Authors:
Heather A. Harrington,
Michał Komorowski,
Mariano Beguerisse Díaz,
Gian Michele Ratto,
Michael P. H. Stumpf
Abstract:
The mitogen activated protein kinase (MAPK) family of proteins is involved in regulating cellular fate activities such as proliferation, differentiation and apoptosis. Their fundamental importance has attracted considerable attention on different aspects of the MAPK signaling dynamics; this is particularly true for the Erk/Mek system, which has become the canonical example for MAPK signaling syste…
▽ More
The mitogen activated protein kinase (MAPK) family of proteins is involved in regulating cellular fate activities such as proliferation, differentiation and apoptosis. Their fundamental importance has attracted considerable attention on different aspects of the MAPK signaling dynamics; this is particularly true for the Erk/Mek system, which has become the canonical example for MAPK signaling systems. Erk exists in many different isoforms, of which the most widely studied are Erk1 and Erk2. Until recently, these two kinases were considered equivalent as they differ only subtly at the sequence level; however, these isoforms exhibit radically different trafficking between cytoplasm and nucleus. Here we use spatially resolved data on Erk1/2 to develop and analyze spatio-temporal models of these cascades; and we discuss how sensitivity analysis can be used to discriminate between mechanisms. We are especially interested in understanding why two such similar proteins should co-exist in the same organism, as their functional roles appear to be different. Our models elucidate some of the factors governing the interplay between processes and the Erk1/2 localization in different cellular compartments, including competition between isoforms. This methodology is applicable to a wide range of systems, such as activation cascades, where translocation of species occurs via signal pathways. Furthermore, our work may motivate additional emphasis for considering potentially different roles for isoforms that differ subtly at the sequence level.
△ Less
Submitted 18 November, 2011; v1 submitted 17 October, 2011;
originally announced October 2011.
-
Decomposing Noise in Biochemical Signalling Systems Highlights the Role of Protein Degradation
Authors:
Michal Komorowski,
Jacek Miekisz,
Michael P. H. Stumpf
Abstract:
The phenomena of stochasticity in biochemical processes have been intriguing life scientists for the past few decades. We now know that living cells take advantage of stochasticity in some cases and counteract stochastic effects in others. The source of intrinsic stochasticity in biomolecular systems are random timings of individual reactions, which cumulatively drive the variability in outputs of…
▽ More
The phenomena of stochasticity in biochemical processes have been intriguing life scientists for the past few decades. We now know that living cells take advantage of stochasticity in some cases and counteract stochastic effects in others. The source of intrinsic stochasticity in biomolecular systems are random timings of individual reactions, which cumulatively drive the variability in outputs of such systems. Despite the acknowledged relevance of stochasticity in the functioning of living cells no rigorous method have been proposed to precisely identify sources of variability. In this paper we propose a novel methodology that allows us to calculate contributions of individual reactions into the variability of a system's output. We demonstrate that some reactions have dramatically different effects on noise than others. Surprisingly, in the class of open conversion systems that serve as an approximate model of signal transduction, the degradation of an output contributes half of the total noise. We also demonstrate the importance of degradation in other relevant systems and propose a degradation feedback control mechanism that has the capability of an effective noise suppression. Application of our method to some well studied biochemical systems such as: gene expression, Michaelis-Menten enzyme kinetics, and the p53 system indicates that our methodology reveals an unprecedented insight into the origins of variability in biochemical systems. For many systems an analytical decomposition is not available; therefore the method has been implemented as a Matlab package and is available from the authors upon request.
△ Less
Submitted 8 June, 2011;
originally announced June 2011.
-
Sensitivity, robustness and identifiability in stochastic chemical kinetics models
Authors:
Michal Komorowski,
Maria J. Costa,
David A. Rand,
Michael Stumpf
Abstract:
We present a novel and simple method to numerically calculate Fisher Information Matrices for stochastic chemical kinetics models. The linear noise approximation is used to derive model equations and a likelihood function which leads to an efficient computational algorithm. Our approach reduces the problem of calculating the Fisher Information Matrix to solving a set of ordinary differential equat…
▽ More
We present a novel and simple method to numerically calculate Fisher Information Matrices for stochastic chemical kinetics models. The linear noise approximation is used to derive model equations and a likelihood function which leads to an efficient computational algorithm. Our approach reduces the problem of calculating the Fisher Information Matrix to solving a set of ordinary differential equations. {This is the first method to compute Fisher Information for stochastic chemical kinetics models without the need for Monte Carlo simulations.} This methodology is then used to study sensitivity, robustness and parameter identifiability in stochastic chemical kinetics models. We show that significant differences exist between stochastic and deterministic models as well as between stochastic models with time-series and time-point measurements. We demonstrate that these discrepancies arise from the variability in molecule numbers, correlations between species, and temporal correlations and show how this approach can be used in the analysis and design of experiments probing stochastic processes at the cellular level. The algorithm has been implemented as a Matlab package and is available from the authors upon request.
△ Less
Submitted 7 April, 2011;
originally announced April 2011.
-
Bayesian inference of biochemical kinetic parameters using the linear noise approximation
Authors:
Michal Komorowski,
Barbel Finkenstadt,
Claire V. Harper,
David A. Rand
Abstract:
Fluorescent and luminescent gene reporters allow us to dynamically quantify changes in molecular species concentration over time on the single cell level. The mathematical modeling of their interaction through multivariate dynamical models requires the development of effective statistical methods to calibrate such models against available data. Given the prevalence of stochasticity and noise in…
▽ More
Fluorescent and luminescent gene reporters allow us to dynamically quantify changes in molecular species concentration over time on the single cell level. The mathematical modeling of their interaction through multivariate dynamical models requires the development of effective statistical methods to calibrate such models against available data. Given the prevalence of stochasticity and noise in biochemical systems inference for stochastic models is of special interest. In this paper we present a simple and computationally efficient algorithm for the estimation of biochemical kinetic parameters from gene reporter data. We use the linear noise approximation to model biochemical reactions through a stochastic dynamic model which essentially approximates a diffusion model by an ordinary differential equation model with an appropriately defined noise process. An explicit formula for the likelihood function can be derived allowing for computationally efficient parameter estimation. The proposed algorithm is embedded in a Bayesian framework and inference is performed using Markov chain Monte Carlo. The major advantage of the method is that in contrast to the more established diffusion approximation based methods the computationally costly methods of data augmentation are not necessary. Our approach also allows for unobserved variables and measurement error. The application of the method to both simulated and experimental data shows that the proposed methodology provides a useful alternative to diffusion approximation based methods.
△ Less
Submitted 4 July, 2009;
originally announced July 2009.