-
Unraveling Anomalies in Time: Unsupervised Discovery and Isolation of Anomalous Behavior in Bio-regenerative Life Support System Telemetry
Authors:
Ferdinand Rewicki,
Jakob Gawlikowski,
Julia Niebling,
Joachim Denzler
Abstract:
The detection of abnormal or critical system states is essential in condition monitoring. While much attention is given to promptly identifying anomalies, a retrospective analysis of these anomalies can significantly enhance our comprehension of the underlying causes of observed undesired behavior. This aspect becomes particularly critical when the monitored system is deployed in a vital environme…
▽ More
The detection of abnormal or critical system states is essential in condition monitoring. While much attention is given to promptly identifying anomalies, a retrospective analysis of these anomalies can significantly enhance our comprehension of the underlying causes of observed undesired behavior. This aspect becomes particularly critical when the monitored system is deployed in a vital environment. In this study, we delve into anomalies within the domain of Bio-Regenerative Life Support Systems (BLSS) for space exploration and analyze anomalies found in telemetry data stemming from the EDEN ISS space greenhouse in Antarctica. We employ time series clustering on anomaly detection results to categorize various types of anomalies in both uni- and multivariate settings. We then assess the effectiveness of these methods in identifying systematic anomalous behavior. Additionally, we illustrate that the anomaly detection methods MDI and DAMP produce complementary results, as previously indicated by research.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
The Unreasonable Effectiveness of Deep Evidential Regression
Authors:
Nis Meinert,
Jakob Gawlikowski,
Alexander Lavin
Abstract:
There is a significant need for principled uncertainty reasoning in machine learning systems as they are increasingly deployed in safety-critical domains. A new approach with uncertainty-aware regression-based neural networks (NNs), based on learning evidential distributions for aleatoric and epistemic uncertainties, shows promise over traditional deterministic methods and typical Bayesian NNs, no…
▽ More
There is a significant need for principled uncertainty reasoning in machine learning systems as they are increasingly deployed in safety-critical domains. A new approach with uncertainty-aware regression-based neural networks (NNs), based on learning evidential distributions for aleatoric and epistemic uncertainties, shows promise over traditional deterministic methods and typical Bayesian NNs, notably with the capabilities to disentangle aleatoric and epistemic uncertainties. Despite some empirical success of Deep Evidential Regression (DER), there are important gaps in the mathematical foundation that raise the question of why the proposed technique seemingly works. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a heuristic rather than an exact uncertainty quantification. We go on to discuss corrections and redefinitions of how aleatoric and epistemic uncertainties should be extracted from NNs.
△ Less
Submitted 16 July, 2023; v1 submitted 20 May, 2022;
originally announced May 2022.
-
A Survey of Uncertainty in Deep Neural Networks
Authors:
Jakob Gawlikowski,
Cedrique Rovile Njieutcheu Tassi,
Mohsin Ali,
Jongseok Lee,
Matthias Humt,
Jianxiang Feng,
Anna Kruspe,
Rudolph Triebel,
Peter Jung,
Ribana Roscher,
Muhammad Shahzad,
Wen Yang,
Richard Bamler,
Xiao Xiang Zhu
Abstract:
Due to their increasing spread, confidence in neural network predictions became more and more important. However, basic neural networks do not deliver certainty estimates or suffer from over or under confidence. Many researchers have been working on understanding and quantifying uncertainty in a neural network's prediction. As a result, different types and sources of uncertainty have been identifi…
▽ More
Due to their increasing spread, confidence in neural network predictions became more and more important. However, basic neural networks do not deliver certainty estimates or suffer from over or under confidence. Many researchers have been working on understanding and quantifying uncertainty in a neural network's prediction. As a result, different types and sources of uncertainty have been identified and a variety of approaches to measure and quantify uncertainty in neural networks have been proposed. This work gives a comprehensive overview of uncertainty estimation in neural networks, reviews recent advances in the field, highlights current challenges, and identifies potential research opportunities. It is intended to give anyone interested in uncertainty estimation in neural networks a broad overview and introduction, without presupposing prior knowledge in this field. A comprehensive introduction to the most crucial sources of uncertainty is given and their separation into reducible model uncertainty and not reducible data uncertainty is presented. The modeling of these uncertainties based on deterministic neural networks, Bayesian neural networks, ensemble of neural networks, and test-time data augmentation approaches is introduced and different branches of these fields as well as the latest developments are discussed. For a practical application, we discuss different measures of uncertainty, approaches for the calibration of neural networks and give an overview of existing baselines and implementations. Different examples from the wide spectrum of challenges in different fields give an idea of the needs and challenges regarding uncertainties in practical applications. Additionally, the practical limitations of current methods for mission- and safety-critical real world applications are discussed and an outlook on the next steps towards a broader usage of such methods is given.
△ Less
Submitted 18 January, 2022; v1 submitted 7 July, 2021;
originally announced July 2021.
-
Leveraging Graph and Deep Learning Uncertainties to Detect Anomalous Trajectories
Authors:
Sandeep Kumar Singh,
Jaya Shradha Fowdur,
Jakob Gawlikowski,
Daniel Medina
Abstract:
Understanding and representing traffic patterns are key to detecting anomalous trajectories in the transportation domain. However, some trajectories can exhibit heterogeneous maneuvering characteristics despite confining to normal patterns. Thus, we propose a novel graph-based trajectory representation and association scheme for extraction and confederation of traffic movement patterns, such that…
▽ More
Understanding and representing traffic patterns are key to detecting anomalous trajectories in the transportation domain. However, some trajectories can exhibit heterogeneous maneuvering characteristics despite confining to normal patterns. Thus, we propose a novel graph-based trajectory representation and association scheme for extraction and confederation of traffic movement patterns, such that data patterns and uncertainty can be learned by deep learning (DL) models. This paper proposes the usage of a recurrent neural network (RNN)-based evidential regression model, which can predict trajectory at future timesteps as well as estimate the data and model uncertainties associated, to detect maritime anomalous trajectories, such as unusual vessel maneuvering, using automatic identification system (AIS) data. Furthermore, we utilize evidential deep learning classifiers to detect unusual turns of vessels and the loss of transmitted signal using predicted class probabilities with associated uncertainties. Our experimental results suggest that the graphical representation of traffic patterns improves the ability of the DL models, such as evidential and Monte Carlo dropout, to learn the temporal-spatial correlation of data and associated uncertainties. Using different datasets and experiments, we demonstrate that the estimated prediction uncertainty yields fundamental information for the detection of traffic anomalies in the maritime and, possibly in other domains.
△ Less
Submitted 12 March, 2022; v1 submitted 4 July, 2021;
originally announced July 2021.
-
Out-of-distribution detection in satellite image classification
Authors:
Jakob Gawlikowski,
Sudipan Saha,
Anna Kruspe,
Xiao Xiang Zhu
Abstract:
In satellite image analysis, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data and differences in the geographic area. Deep learning based models may behave in unexpected manner when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. P…
▽ More
In satellite image analysis, distributional mismatch between the training and test data may arise due to several reasons, including unseen classes in the test data and differences in the geographic area. Deep learning based models may behave in unexpected manner when subjected to test data that has such distributional shifts from the training data, also called out-of-distribution (OOD) examples. Predictive uncertainly analysis is an emerging research topic which has not been explored much in context of satellite image analysis. Towards this, we adopt a Dirichlet Prior Network based model to quantify distributional uncertainty of deep learning models for remote sensing. The approach seeks to maximize the representation gap between the in-domain and OOD examples for a better identification of unknown examples at test time. Experimental results on three exemplary test scenarios show the efficacy of the model in satellite image analysis.
△ Less
Submitted 9 April, 2021;
originally announced April 2021.
