-
Principal Component Pursuit for Pattern Identification in Environmental Mixtures
Authors:
Elizabeth A. Gibson,
Junhui Zhang,
**gkai Yan,
Lawrence Chillrud,
Jaime Benavides,
Yanelli Nunez,
Julie B. Herbstman,
Jeff Goldsmith,
John Wright,
Marianthi-Anna Kioumourtzoglou
Abstract:
Environmental health researchers often aim to identify sources/behaviors that give rise to potentially harmful exposures. We adapted principal component pursuit (PCP)-a robust technique for dimensionality reduction in computer vision and signal processing-to identify patterns in environmental mixtures. PCP decomposes the exposure mixture into a low-rank matrix containing consistent exposure patter…
▽ More
Environmental health researchers often aim to identify sources/behaviors that give rise to potentially harmful exposures. We adapted principal component pursuit (PCP)-a robust technique for dimensionality reduction in computer vision and signal processing-to identify patterns in environmental mixtures. PCP decomposes the exposure mixture into a low-rank matrix containing consistent exposure patterns across pollutants and a sparse matrix isolating unique exposure events. We adapted PCP to accommodate non-negative and missing data, and values below a given limit of detection (LOD). We simulated data to represent environmental mixtures of two sizes with increasing proportions <LOD and three noise structures. We compared PCP-LOD to principal component analysis (PCA) to evaluate performance. We next applied PCP-LOD to a mixture of 21 persistent organic pollutants (POPs) measured in 1,000 U.S. adults from the 2001-2002 National Health and Nutrition Examination Survey. We applied singular value decomposition to the estimated low-rank matrix to characterize the patterns. PCP-LOD recovered the true number of patterns through cross-validation for all simulations; based on an a priori specified criterion, PCA recovered the true number of patterns in 32% of simulations. PCP-LOD achieved lower relative predictive error than PCA for all simulated datasets with up to 50% of the data <LOD. When 75% of values were <LOD, PCP-LOD outperformed PCA only when noise was low. In the POP mixture, PCP-LOD identified a rank-three underlying structure and separated 6% of values as unique events. One pattern represented comprehensive exposure to all POPs. The other patterns grouped chemicals based on known structure and toxicity. PCP-LOD serves as a useful tool to express multi-dimensional exposures as consistent patterns that, if found to be related to adverse health, are amenable to targeted interventions.
△ Less
Submitted 29 October, 2021;
originally announced November 2021.
-
Quantifying and Leveraging Predictive Uncertainty for Medical Image Assessment
Authors:
Florin C. Ghesu,
Bogdan Georgescu,
Awais Mansoor,
Young** Yoo,
Eli Gibson,
R. S. Vishwanath,
Abishek Balachandran,
James M. Balter,
Yue Cao,
Ramandeep Singh,
Subba R. Digumarthy,
Mannudeep K. Kalra,
Sasa Grbic,
Dorin Comaniciu
Abstract:
The interpretation of medical images is a challenging task, often complicated by the presence of artifacts, occlusions, limited contrast and more. Most notable is the case of chest radiography, where there is a high inter-rater variability in the detection and classification of abnormalities. This is largely due to inconclusive evidence in the data or subjective definitions of disease appearance.…
▽ More
The interpretation of medical images is a challenging task, often complicated by the presence of artifacts, occlusions, limited contrast and more. Most notable is the case of chest radiography, where there is a high inter-rater variability in the detection and classification of abnormalities. This is largely due to inconclusive evidence in the data or subjective definitions of disease appearance. An additional example is the classification of anatomical views based on 2D Ultrasound images. Often, the anatomical context captured in a frame is not sufficient to recognize the underlying anatomy. Current machine learning solutions for these problems are typically limited to providing probabilistic predictions, relying on the capacity of underlying models to adapt to limited information and the high degree of label noise. In practice, however, this leads to overconfident systems with poor generalization on unseen data. To account for this, we propose a system that learns not only the probabilistic estimate for classification, but also an explicit uncertainty measure which captures the confidence of the system in the predicted output. We argue that this approach is essential to account for the inherent ambiguity characteristic of medical images from different radiologic exams including computed radiography, ultrasonography and magnetic resonance imaging. In our experiments we demonstrate that sample rejection based on the predicted uncertainty can significantly improve the ROC-AUC for various tasks, e.g., by 8% to 0.91 with an expected rejection rate of under 25% for the classification of different abnormalities in chest radiographs. In addition, we show that using uncertainty-driven bootstrap** to filter the training data, one can achieve a significant increase in robustness and accuracy.
△ Less
Submitted 8 July, 2020;
originally announced July 2020.
-
Accelerated Learning with Robustness to Adversarial Regressors
Authors:
Joseph E. Gaudio,
Anuradha M. Annaswamy,
José M. Moreu,
Michael A. Bolender,
Travis E. Gibson
Abstract:
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms…
▽ More
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms which can be proven stable in the presence of adversarial time-varying regressors, as is commonplace in control theory. In this paper, we propose a new discrete time algorithm which 1) provides stability and asymptotic convergence guarantees in the presence of adversarial regressors by leveraging insights from adaptive control theory and 2) provides non-asymptotic accelerated learning guarantees leveraging insights from convex optimization. In particular, our algorithm reaches an $ε$ sub-optimal point in at most $\tilde{\mathcal{O}}(1/\sqrtε)$ iterations when regressors are constant - matching lower bounds due to Nesterov of $Ω(1/\sqrtε)$, up to a $\log(1/ε)$ factor and provides guaranteed bounds for stability when regressors are time-varying. We provide numerical experiments for a variant of Nesterov's provably hard convex optimization problem with time-varying regressors, as well as the problem of recovering an image with a time-varying blur and noise using streaming data.
△ Less
Submitted 4 June, 2021; v1 submitted 4 May, 2020;
originally announced May 2020.
-
Graph Attention Network based Pruning for Reconstructing 3D Liver Vessel Morphology from Contrasted CT Images
Authors:
Donghao Zhang,
Siqi Liu,
Shikha Chaganti,
Eli Gibson,
Zhoubing Xu,
Sasa Grbic,
Weidong Cai,
Dorin Comaniciu
Abstract:
With the injection of contrast material into blood vessels, multi-phase contrasted CT images can enhance the visibility of vessel networks in the human body. Reconstructing the 3D geometric morphology of liver vessels from the contrasted CT images can enable multiple liver preoperative surgical planning applications. Automatic reconstruction of liver vessel morphology remains a challenging problem…
▽ More
With the injection of contrast material into blood vessels, multi-phase contrasted CT images can enhance the visibility of vessel networks in the human body. Reconstructing the 3D geometric morphology of liver vessels from the contrasted CT images can enable multiple liver preoperative surgical planning applications. Automatic reconstruction of liver vessel morphology remains a challenging problem due to the morphological complexity of liver vessels and the inconsistent vessel intensities among different multi-phase contrasted CT images. On the other side, high integrity is required for the 3D reconstruction to avoid decision making biases. In this paper, we propose a framework for liver vessel morphology reconstruction using both a fully convolutional neural network and a graph attention network. A fully convolutional neural network is first trained to produce the liver vessel centerline heatmap. An over-reconstructed liver vessel graph model is then traced based on the heatmap using an image processing based algorithm. We use a graph attention network to prune the false-positive branches by predicting the presence probability of each segmented branch in the initial reconstruction using the aggregated CNN features. We evaluated the proposed framework on an in-house dataset consisting of 418 multi-phase abdomen CT images with contrast. The proposed graph network pruning improves the overall reconstruction F1 score by 6.4% over the baseline. It also outperformed the other state-of-the-art curvilinear structure reconstruction algorithms.
△ Less
Submitted 17 March, 2020;
originally announced March 2020.
-
No Surprises: Training Robust Lung Nodule Detection for Low-Dose CT Scans by Augmenting with Adversarial Attacks
Authors:
Siqi Liu,
Arnaud Arindra Adiyoso Setio,
Florin C. Ghesu,
Eli Gibson,
Sasa Grbic,
Bogdan Georgescu,
Dorin Comaniciu
Abstract:
Detecting malignant pulmonary nodules at an early stage can allow medical interventions which may increase the survival rate of lung cancer patients. Using computer vision techniques to detect nodules can improve the sensitivity and the speed of interpreting chest CT for lung cancer screening. Many studies have used CNNs to detect nodule candidates. Though such approaches have been shown to outper…
▽ More
Detecting malignant pulmonary nodules at an early stage can allow medical interventions which may increase the survival rate of lung cancer patients. Using computer vision techniques to detect nodules can improve the sensitivity and the speed of interpreting chest CT for lung cancer screening. Many studies have used CNNs to detect nodule candidates. Though such approaches have been shown to outperform the conventional image processing based methods regarding the detection accuracy, CNNs are also known to be limited to generalize on under-represented samples in the training set and prone to imperceptible noise perturbations. Such limitations can not be easily addressed by scaling up the dataset or the models. In this work, we propose to add adversarial synthetic nodules and adversarial attack samples to the training data to improve the generalization and the robustness of the lung nodule detection systems. To generate hard examples of nodules from a differentiable nodule synthesizer, we use projected gradient descent (PGD) to search the latent code within a bounded neighbourhood that would generate nodules to decrease the detector response. To make the network more robust to unanticipated noise perturbations, we use PGD to search for noise patterns that can trigger the network to give over-confident mistakes. By evaluating on two different benchmark datasets containing consensus annotations from three radiologists, we show that the proposed techniques can improve the detection performance on real CT data. To understand the limitations of both the conventional networks and the proposed augmented networks, we also perform stress-tests on the false positive reduction networks by feeding different types of artificially produced patches. We show that the augmented networks are more robust to both under-represented nodules as well as resistant to noise perturbations.
△ Less
Submitted 28 October, 2020; v1 submitted 8 March, 2020;
originally announced March 2020.
-
Conditional Segmentation in Lieu of Image Registration
Authors:
Yipeng Hu,
Eli Gibson,
Dean C. Barratt,
Mark Emberton,
J. Alison Noble,
Tom Vercauteren
Abstract:
Classical pairwise image registration methods search for a spatial transformation that optimises a numerical measure that indicates how well a pair of moving and fixed images are aligned. Current learning-based registration methods have adopted the same paradigm and typically predict, for any new input image pair, dense correspondences in the form of a dense displacement field or parameters of a s…
▽ More
Classical pairwise image registration methods search for a spatial transformation that optimises a numerical measure that indicates how well a pair of moving and fixed images are aligned. Current learning-based registration methods have adopted the same paradigm and typically predict, for any new input image pair, dense correspondences in the form of a dense displacement field or parameters of a spatial transformation model. However, in many applications of registration, the spatial transformation itself is only required to propagate points or regions of interest (ROIs). In such cases, detailed pixel- or voxel-level correspondence within or outside of these ROIs often have little clinical value. In this paper, we propose an alternative paradigm in which the location of corresponding image-specific ROIs, defined in one image, within another image is learnt. This results in replacing image registration by a conditional segmentation algorithm, which can build on typical image segmentation networks and their widely-adopted training strategies. Using the registration of 3D MRI and ultrasound images of the prostate as an example to demonstrate this new approach, we report a median target registration error (TRE) of 2.1 mm between the ground-truth ROIs defined on intraoperative ultrasound images and those propagated from the preoperative MR images. Significantly lower (>34%) TREs were obtained using the proposed conditional segmentation compared with those obtained from a previously-proposed spatial-transformation-predicting registration network trained with the same multiple ROI labels for individual image pairs. We conclude this work by using a quantitative bias-variance analysis to provide one explanation of the observed improvement in registration accuracy.
△ Less
Submitted 30 June, 2019;
originally announced July 2019.
-
Connections Between Adaptive Control and Optimization in Machine Learning
Authors:
Joseph E. Gaudio,
Travis E. Gibson,
Anuradha M. Annaswamy,
Michael A. Bolender,
Eugene Lavretsky
Abstract:
This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts,…
▽ More
This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis are provided. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
Provably Correct Learning Algorithms in the Presence of Time-Varying Features Using a Variational Perspective
Authors:
Joseph E. Gaudio,
Travis E. Gibson,
Anuradha M. Annaswamy,
Michael A. Bolender
Abstract:
Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when…
▽ More
Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when time-varying features are present, and demonstrates provable performance guarantees. In particular, we develop a unified variational perspective within a continuous time algorithm. This variational perspective includes higher order learning concepts and normalization, both of which stem from adaptive control, and allows stability to be established for dynamical machine learning problems where time-varying features are present. These higher order algorithms are also examined for provably correct learning in adaptive control and identification. Simulations are provided to verify the theoretical results.
△ Less
Submitted 27 May, 2019; v1 submitted 11 March, 2019;
originally announced March 2019.
-
Sign Stability via Root Locus Analysis
Authors:
Travis E. Gibson
Abstract:
With the rise of network science old topics in ecology and economics are resurfacing. One such topic is structural stability (often referred to as qualitative stability or sign stability). A system is deemed structurally stable if the system remains stable for all possible parameter variations so long as the parameters do not change sign. This type of stability analysis is appealing when studying…
▽ More
With the rise of network science old topics in ecology and economics are resurfacing. One such topic is structural stability (often referred to as qualitative stability or sign stability). A system is deemed structurally stable if the system remains stable for all possible parameter variations so long as the parameters do not change sign. This type of stability analysis is appealing when studying real systems as the underlying stability result only requires the scientist or engineer to know the sign of the parameters in the model and not the specific values. The necessary and sufficient conditions for qualitative stability however are opaque. In order to shed light on those conditions root locus analysis is employed. This technique allows us to illustrate the necessary conditions for qualitative stability.
△ Less
Submitted 18 December, 2015;
originally announced December 2015.
-
Convergence Properties of Adaptive Systems and the Definition of Exponential Stability
Authors:
Benjamin M. Jenkins,
Anuradha M. Annaswamy,
Eugene Lavretsky,
Travis E. Gibson
Abstract:
The convergence properties of adaptive systems in terms of excitation conditions on the regressor vector are well known. With persistent excitation of the regressor vector in model reference adaptive control the state error and the adaptation error are globally exponentially stable, or equivalently, exponentially stable in the large. When the excitation condition however is imposed on the referenc…
▽ More
The convergence properties of adaptive systems in terms of excitation conditions on the regressor vector are well known. With persistent excitation of the regressor vector in model reference adaptive control the state error and the adaptation error are globally exponentially stable, or equivalently, exponentially stable in the large. When the excitation condition however is imposed on the reference input or the reference model state it is often incorrectly concluded that the persistent excitation in those signals also implies exponential stability in the large. The definition of persistent excitation is revisited so as to address some possible confusion in the adaptive control literature. It is then shown that persistent excitation of the reference model only implies local persistent excitation (weak persistent excitation). Weak persistent excitation of the regressor is still sufficient for uniform asymptotic stability in the large, but not exponential stability in the large. We show that there exists an infinite region in the state-space of adaptive systems where the state rate is bounded. This infinite region with finite rate of convergence is shown to exist not only in classic open-loop reference model adaptive systems, but also in a new class of closed-loop reference model adaptive systems.
△ Less
Submitted 10 November, 2015;
originally announced November 2015.
-
On the Origins and Control of Community Types in the Human Microbiome
Authors:
Travis E. Gibson,
Amir Bashan,
Hong-Tai Cao,
Scott T. Weiss,
Yang-Yu Liu
Abstract:
Microbiome-based stratification of healthy individuals into compositional categories, referred to as "community types", holds promise for drastically improving personalized medicine. Despite this potential, the existence of community types and the degree of their distinctness have been highly debated. Here we adopted a dynamic systems approach and found that heterogeneity in the interspecific inte…
▽ More
Microbiome-based stratification of healthy individuals into compositional categories, referred to as "community types", holds promise for drastically improving personalized medicine. Despite this potential, the existence of community types and the degree of their distinctness have been highly debated. Here we adopted a dynamic systems approach and found that heterogeneity in the interspecific interactions or the presence of strongly interacting species is sufficient to explain community types, independent of the topology of the underlying ecological network. By controlling the presence or absence of these strongly interacting species we can steer the microbial ecosystem to any desired community type. This open-loop control strategy still holds even when the community types are not distinct but appear as dense regions within a continuous gradient. This finding can be used to develop viable therapeutic strategies for shifting the microbial composition to a healthy configuration
△ Less
Submitted 21 January, 2016; v1 submitted 16 June, 2015;
originally announced June 2015.
-
Adaptive Output Feedback based on Closed-loop Reference Models
Authors:
Travis E. Gibson,
Zheng Qu,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
This note presents the design and analysis of an adaptive controller for a class of linear plants in the presence of output feedback. This controller makes use of a closed-loop reference model as an observer, and guarantees global stability and asymptotic output tracking.
This note presents the design and analysis of an adaptive controller for a class of linear plants in the presence of output feedback. This controller makes use of a closed-loop reference model as an observer, and guarantees global stability and asymptotic output tracking.
△ Less
Submitted 7 October, 2014;
originally announced October 2014.
-
On Adaptive Control with Closed-loop Reference Models: Transients, Oscillations, and Peaking
Authors:
Travis E. Gibson,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
One of the main features of adaptive systems is an oscillatory convergence that exacerbates with the speed of adaptation. Recently it has been shown that Closed-loop Reference Models (CRMs) can result in improved transient performance over their open-loop counterparts in model reference adaptive control. In this paper, we quantify both the transient performance in the classical adaptive systems an…
▽ More
One of the main features of adaptive systems is an oscillatory convergence that exacerbates with the speed of adaptation. Recently it has been shown that Closed-loop Reference Models (CRMs) can result in improved transient performance over their open-loop counterparts in model reference adaptive control. In this paper, we quantify both the transient performance in the classical adaptive systems and their improvement with CRMs. In addition to deriving bounds on L-2 norms of the derivatives of the adaptive parameters which are shown to be smaller, an optimal design of CRMs is proposed which minimizes an underlying peaking phenomenon. The analytical tools proposed are shown to be applicable for a range of adaptive control problems including direct control and composite control with observer feedback. The presence of CRMs in adaptive backstep** and adaptive robot control are also discussed. Simulation results are presented throughout the paper to support the theoretical derivations.
△ Less
Submitted 8 August, 2013; v1 submitted 26 April, 2013;
originally announced April 2013.
-
Closed-loop Reference Models for Output-Feedback Adaptive Systems
Authors:
Travis E. Gibson,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
Closed-loop reference models have recently been proposed for states accessible adaptive systems. They have been shown to have improved transient response over their open loop counter parts. The results in the states accessible case are extended to single input single output plants of arbitrary relative degree.
Closed-loop reference models have recently been proposed for states accessible adaptive systems. They have been shown to have improved transient response over their open loop counter parts. The results in the states accessible case are extended to single input single output plants of arbitrary relative degree.
△ Less
Submitted 27 November, 2012; v1 submitted 30 October, 2012;
originally announced October 2012.
-
Adaptive Systems with Closed-loop Reference Models: Stability, Robustness and Transient Performance
Authors:
Travis E. Gibson,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
This paper explores the properties of adaptive systems with closed-loop reference models. Using additional design freedom available in closed-loop reference models, we design new adaptive controllers that are (a) stable, and (b) have improved transient properties. Numerical studies that complement theoretical derivations are also reported.
This paper explores the properties of adaptive systems with closed-loop reference models. Using additional design freedom available in closed-loop reference models, we design new adaptive controllers that are (a) stable, and (b) have improved transient properties. Numerical studies that complement theoretical derivations are also reported.
△ Less
Submitted 30 October, 2012; v1 submitted 23 January, 2012;
originally announced January 2012.
-
Projection Operator in Adaptive Systems
Authors:
Eugene Lavretsky,
Travis E. Gibson
Abstract:
The projection algorithm is frequently used in adaptive control and this note presents a detailed analysis of its properties.
The projection algorithm is frequently used in adaptive control and this note presents a detailed analysis of its properties.
△ Less
Submitted 16 October, 2012; v1 submitted 18 December, 2011;
originally announced December 2011.
