Showing 1–45 of 45 results for author: Pohl, K

Few Shot Part Segmentation Reveals Compositional Logic for Industrial Anomaly Detection

Authors: Soopil Kim, Sion An, Philip Chikontwe, Myeongkyun Kang, Ehsan Adeli, Kilian M. Pohl, Sang Hyun Park

Abstract: Logical anomalies (LA) refer to data violating underlying logical constraints e.g., the quantity, arrangement, or composition of components within an image. Detecting accurately such anomalies requires models to reason about various component types through segmentation. However, curation of pixel-level annotations for semantic segmentation is both time-consuming and expensive. Although there are s… ▽ More Logical anomalies (LA) refer to data violating underlying logical constraints e.g., the quantity, arrangement, or composition of components within an image. Detecting accurately such anomalies requires models to reason about various component types through segmentation. However, curation of pixel-level annotations for semantic segmentation is both time-consuming and expensive. Although there are some prior few-shot or unsupervised co-part segmentation algorithms, they often fail on images with industrial object. These images have components with similar textures and shapes, and a precise differentiation proves challenging. In this study, we introduce a novel component segmentation model for LA detection that leverages a few labeled samples and unlabeled images sharing logical constraints. To ensure consistent segmentation across unlabeled images, we employ a histogram matching loss in conjunction with an entropy loss. As segmentation predictions play a crucial role, we propose to enhance both local and global sample validity detection by capturing key aspects from visual semantics via three memory banks: class histograms, component composition embeddings and patch-level representations. For effective LA detection, we propose an adaptive scaling strategy to standardize anomaly scores from different memory banks in inference. Extensive experiments on the public benchmark MVTec LOCO AD reveal our method achieves 98.1% AUROC in LA detection vs. 89.6% from competing methods. △ Less

Submitted 15 April, 2024; v1 submitted 21 December, 2023; originally announced December 2023.

Comments: Accepted in AAAI2024

On nonlinear Feynman-Kac formulas for viscosity solutions of semilinear parabolic partial differential equations with gradient-dependent nonlinearities

Authors: Martin Hutzenthaler, Katharina Pohl

Abstract: The classical Feynman-Kac identity represents solutions of linear partial differential equations in terms of stochastic differential euqations. This representation has been generalized to nonlinear partial differential equations on the one hand via backward stochastic differential equations and on the other hand via stochastic fixed-point equations. In this article we generalize the representation… ▽ More The classical Feynman-Kac identity represents solutions of linear partial differential equations in terms of stochastic differential euqations. This representation has been generalized to nonlinear partial differential equations on the one hand via backward stochastic differential equations and on the other hand via stochastic fixed-point equations. In this article we generalize the representation via stochastic fixed-point equations to allow the nonlinearity in the semilinear partial differential equation to depend also on the gradient of the solution. △ Less

Submitted 27 October, 2023; originally announced October 2023.

Comments: 39 pages

Towards the decentralized coordination of multiple self-adaptive systems

Authors: Paul-Andrei Dragan, Andreas Metzger, Klaus Pohl

Abstract: When multiple self-adaptive systems share the same environment and have common goals, they may coordinate their adaptations at runtime to avoid conflicts and to satisfy their goals. There are two approaches to coordination. (1) Logically centralized, where a supervisor has complete control over the individual self-adaptive systems. Such approach is infeasible when the systems have different owners… ▽ More When multiple self-adaptive systems share the same environment and have common goals, they may coordinate their adaptations at runtime to avoid conflicts and to satisfy their goals. There are two approaches to coordination. (1) Logically centralized, where a supervisor has complete control over the individual self-adaptive systems. Such approach is infeasible when the systems have different owners or administrative domains. (2) Logically decentralized, where coordination is achieved through direct interactions. Because the individual systems have control over the information they share, decentralized coordination accommodates multiple administrative domains. However, existing techniques do not account simultaneously for both local concerns, e.g., preferences, and shared concerns, e.g., conflicts, which may lead to goals not being achieved as expected. Our idea to address this shortcoming is to express both types of concerns within the same constraint optimization problem. We propose CoADAPT, a decentralized coordination technique introducing two types of constraints: preference constraints, expressing local concerns, and consistency constraints, expressing shared concerns. At runtime, the problem is solved in a decentralized way using distributed constraint optimization algorithms implemented by each self-adaptive system. As a first step in realizing CoADAPT, we focus in this work on the coordination of adaptation planning strategies, traditionally addressed only with centralized techniques. We show the feasibility of CoADAPT in an exemplar from cloud computing and analyze experimentally its scalability. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Journal ref: 4th IEEE Intl. Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2023)

On existence and uniqueness properties for solutions of stochastic fixed point equations with gradient-dependent nonlinearities

Authors: Katharina Pohl, Martin Hutzenthaler

Abstract: The combination of the Itô formula and the Bismut-Elworthy-Li formula implies that suitable smooth solutions of semilinear Kolmogorov partial differential equations (PDEs) are also solutions to certain stochastic fixed point equations (SFPEs). In this paper we generalize known results on existence and uniqueness of solutions of SFPEs associated with PDEs with Lipschitz continuous, gradient-indepen… ▽ More The combination of the Itô formula and the Bismut-Elworthy-Li formula implies that suitable smooth solutions of semilinear Kolmogorov partial differential equations (PDEs) are also solutions to certain stochastic fixed point equations (SFPEs). In this paper we generalize known results on existence and uniqueness of solutions of SFPEs associated with PDEs with Lipschitz continuous, gradient-independent nonlinearities to the case of gradient-dependent nonlinearities. The main challenge arises from the fact that in the case of a non-differentiable terminal condition and a gradient-dependent nonlinearity the Bismut-Elworthy-Li formula leads to a singularity of the solution of the SFPE in the last time point. △ Less

Submitted 26 October, 2023; originally announced October 2023.

Comments: 32 pages

Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIs

Authors: Wei Peng, Tomas Bosschieter, Jiahong Ouyang, Robert Paul, Ehsan Adeli, Qingyu Zhao, Kilian M. Pohl

Abstract: Generative AI models hold great potential in creating synthetic brain MRIs that advance neuroimaging studies by, for example, enriching data diversity. However, the mainstay of AI research only focuses on optimizing the visual quality (such as signal-to-noise ratio) of the synthetic MRIs while lacking insights into their relevance to neuroscience. To gain these insights with respect to T1-weighted… ▽ More Generative AI models hold great potential in creating synthetic brain MRIs that advance neuroimaging studies by, for example, enriching data diversity. However, the mainstay of AI research only focuses on optimizing the visual quality (such as signal-to-noise ratio) of the synthetic MRIs while lacking insights into their relevance to neuroscience. To gain these insights with respect to T1-weighted MRIs, we first propose a new generative model, BrainSynth, to synthesize metadata-conditioned (e.g., age- and sex-specific) MRIs that achieve state-of-the-art visual quality. We then extend our evaluation with a novel procedure to quantify anatomical plausibility, i.e., how well the synthetic MRIs capture macrostructural properties of brain regions, and how accurately they encode the effects of age and sex. Results indicate that more than half of the brain regions in our synthetic MRIs are anatomically accurate, i.e., with a small effect size between real and synthetic MRIs. Moreover, the anatomical plausibility varies across cortical regions according to their geometric complexity. As is, our synthetic MRIs can significantly improve the training of a Convolutional Neural Network to identify accelerated aging effects in an independent study. These results highlight the opportunities of using generative AI to aid neuroimaging research and point to areas for further improvement. △ Less

Submitted 6 October, 2023; originally announced October 2023.

LSOR: Longitudinally-Consistent Self-Organized Representation Learning

Authors: Jiahong Ouyang, Qingyu Zhao, Ehsan Adeli, Wei Peng, Greg Zaharchuk, Kilian M. Pohl

Abstract: Interpretability is a key issue when applying deep learning models to longitudinal brain MRIs. One way to address this issue is by visualizing the high-dimensional latent spaces generated by deep learning via self-organizing maps (SOM). SOM separates the latent space into clusters and then maps the cluster centers to a discrete (typically 2D) grid preserving the high-dimensional relationship betwe… ▽ More Interpretability is a key issue when applying deep learning models to longitudinal brain MRIs. One way to address this issue is by visualizing the high-dimensional latent spaces generated by deep learning via self-organizing maps (SOM). SOM separates the latent space into clusters and then maps the cluster centers to a discrete (typically 2D) grid preserving the high-dimensional relationship between clusters. However, learning SOM in a high-dimensional latent space tends to be unstable, especially in a self-supervision setting. Furthermore, the learned SOM grid does not necessarily capture clinically interesting information, such as brain age. To resolve these issues, we propose the first self-supervised SOM approach that derives a high-dimensional, interpretable representation stratified by brain age solely based on longitudinal brain MRIs (i.e., without demographic or cognitive information). Called Longitudinally-consistent Self-Organized Representation learning (LSOR), the method is stable during training as it relies on soft clustering (vs. the hard cluster assignments used by existing SOM). Furthermore, our approach generates a latent space stratified according to brain age by aligning trajectories inferred from longitudinal MRIs to the reference vector associated with the corresponding SOM cluster. When applied to longitudinal MRIs of the Alzheimer's Disease Neuroimaging Initiative (ADNI, N=632), LSOR generates an interpretable latent space and achieves comparable or higher accuracy than the state-of-the-art representations with respect to the downstream tasks of classification (static vs. progressive mild cognitive impairment) and regression (determining ADAS-Cog score of all subjects). The code is available at https://github.com/ouyangjiahong/longitudinal-som-single-modality. △ Less

Submitted 29 September, 2023; originally announced October 2023.

Journal ref: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2023

Fuzz on the Beach: Fuzzing Solana Smart Contracts

Authors: Sven Smolka, Jens-Rene Giesen, Pascal Winkler, Oussama Draissi, Lucas Davi, Ghassan Karame, Klaus Pohl

Abstract: Solana has quickly emerged as a popular platform for building decentralized applications (DApps), such as marketplaces for non-fungible tokens (NFTs). A key reason for its success are Solana's low transaction fees and high performance, which is achieved in part due to its stateless programming model. Although the literature features extensive tooling support for smart contract security, current so… ▽ More Solana has quickly emerged as a popular platform for building decentralized applications (DApps), such as marketplaces for non-fungible tokens (NFTs). A key reason for its success are Solana's low transaction fees and high performance, which is achieved in part due to its stateless programming model. Although the literature features extensive tooling support for smart contract security, current solutions are largely tailored for the Ethereum Virtual Machine. Unfortunately, the very stateless nature of Solana's execution environment introduces novel attack patterns specific to Solana requiring a rethinking for building vulnerability analysis methods. In this paper, we address this gap and propose FuzzDelSol, the first binary-only coverage-guided fuzzing architecture for Solana smart contracts. FuzzDelSol faithfully models runtime specifics such as smart contract interactions. Moreover, since source code is not available for the large majority of Solana contracts, FuzzDelSol operates on the contract's binary code. Hence, due to the lack of semantic information, we carefully extracted low-level program and state information to develop a diverse set of bug oracles covering all major bug classes in Solana. Our extensive evaluation on 6049 smart contracts shows that FuzzDelSol's bug oracles find bugs with a high precision and recall. To the best of our knowledge, this is the largest evaluation of the security landscape on the Solana mainnet. △ Less

Submitted 4 October, 2023; v1 submitted 6 September, 2023; originally announced September 2023.

Comments: This paper will appear on the ACM CCS 2023 in November 2023

Imputing Brain Measurements Across Data Sets via Graph Neural Networks

Authors: Yixin Wang, Wei Peng, Susan F. Tapert, Qingyu Zhao, Kilian M. Pohl

Abstract: Publicly available data sets of structural MRIs might not contain specific measurements of brain Regions of Interests (ROIs) that are important for training machine learning models. For example, the curvature scores computed by Freesurfer are not released by the Adolescent Brain Cognitive Development (ABCD) Study. One can address this issue by simply reapplying Freesurfer to the data set. However,… ▽ More Publicly available data sets of structural MRIs might not contain specific measurements of brain Regions of Interests (ROIs) that are important for training machine learning models. For example, the curvature scores computed by Freesurfer are not released by the Adolescent Brain Cognitive Development (ABCD) Study. One can address this issue by simply reapplying Freesurfer to the data set. However, this approach is generally computationally and labor intensive (e.g., requiring quality control). An alternative is to impute the missing measurements via a deep learning approach. However, the state-of-the-art is designed to estimate randomly missing values rather than entire measurements. We therefore propose to re-frame the imputation problem as a prediction task on another (public) data set that contains the missing measurements and shares some ROI measurements with the data sets of interest. A deep learning model is then trained to predict the missing measurements from the shared ones and afterwards is applied to the other data sets. Our proposed algorithm models the dependencies between ROI measurements via a graph neural network (GNN) and accounts for demographic differences in brain measurements (e.g. sex) by feeding the graph encoding into a parallel architecture. The architecture simultaneously optimizes a graph decoder to impute values and a classifier in predicting demographic factors. We test the approach, called Demographic Aware Graph-based Imputation (DAGI), on imputing those missing Freesurfer measurements of ABCD (N=3760) by training the predictor on those publicly released by the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA, N=540)... △ Less

Submitted 19 August, 2023; originally announced August 2023.

Comments: Accepted at the 6th workshop on PRedictive Intelligence in Medicine (PRIME 2023) - MICCAI 2023

An Explainable Geometric-Weighted Graph Attention Network for Identifying Functional Networks Associated with Gait Impairment

Authors: Favour Nerrise, Qingyu Zhao, Kathleen L. Poston, Kilian M. Pohl, Ehsan Adeli

Abstract: One of the hallmark symptoms of Parkinson's Disease (PD) is the progressive loss of postural reflexes, which eventually leads to gait difficulties and balance problems. Identifying disruptions in brain function associated with gait impairment could be crucial in better understanding PD motor progression, thus advancing the development of more effective and personalized therapeutics. In this work,… ▽ More One of the hallmark symptoms of Parkinson's Disease (PD) is the progressive loss of postural reflexes, which eventually leads to gait difficulties and balance problems. Identifying disruptions in brain function associated with gait impairment could be crucial in better understanding PD motor progression, thus advancing the development of more effective and personalized therapeutics. In this work, we present an explainable, geometric, weighted-graph attention neural network (xGW-GAT) to identify functional networks predictive of the progression of gait difficulties in individuals with PD. xGW-GAT predicts the multi-class gait impairment on the MDS Unified PD Rating Scale (MDS-UPDRS). Our computational- and data-efficient model represents functional connectomes as symmetric positive definite (SPD) matrices on a Riemannian manifold to explicitly encode pairwise interactions of entire connectomes, based on which we learn an attention mask yielding individual- and group-level explainability. Applied to our resting-state functional MRI (rs-fMRI) dataset of individuals with PD, xGW-GAT identifies functional connectivity patterns associated with gait impairment in PD and offers interpretable explanations of functional subnetworks associated with motor impairment. Our model successfully outperforms several existing methods while simultaneously revealing clinically-relevant connectivity patterns. The source code is available at https://github.com/favour-nerrise/xGW-GAT . △ Less

Submitted 24 July, 2023; originally announced July 2023.

Comments: Accepted by the 26th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). MICCAI Student-Author Registration (STAR) Award. 11 pages, 2 figures, 1 table, appendix. Source Code: https://github.com/favour-nerrise/xGW-GAT

Automatically Reconciling the Trade-off between Prediction Accuracy and Earliness in Prescriptive Business Process Monitoring

Authors: Andreas Metzger, Tristan Kley, Aristide Rothweiler, Klaus Pohl

Abstract: Prescriptive business process monitoring provides decision support to process managers on when and how to adapt an ongoing business process to prevent or mitigate an undesired process outcome. We focus on the problem of automatically reconciling the trade-off between prediction accuracy and prediction earliness in determining when to adapt. Adaptations should happen sufficiently early to provide e… ▽ More Prescriptive business process monitoring provides decision support to process managers on when and how to adapt an ongoing business process to prevent or mitigate an undesired process outcome. We focus on the problem of automatically reconciling the trade-off between prediction accuracy and prediction earliness in determining when to adapt. Adaptations should happen sufficiently early to provide enough lead time for the adaptation to become effective. However, earlier predictions are typically less accurate than later predictions. This means that acting on less accurate predictions may lead to unnecessary adaptations or missed adaptations. Different approaches were presented in the literature to reconcile the trade-off between prediction accuracy and earliness. So far, these approaches were compared with different baselines, and evaluated using different data sets or even confidential data sets. This limits the comparability and replicability of the approaches and makes it difficult to choose a concrete approach in practice. We perform a comparative evaluation of the main alternative approaches for reconciling the trade-off between prediction accuracy and earliness. Using four public real-world event log data sets and two types of prediction models, we assess and compare the cost savings of these approaches. The experimental results indicate which criteria affect the effectiveness of an approach and help us state initial recommendations for the selection of a concrete approach in practice. △ Less

Submitted 12 July, 2023; originally announced July 2023.

Journal ref: Information Systems, 2023

A User Study on Explainable Online Reinforcement Learning for Adaptive Systems

Authors: Andreas Metzger, Jan Laufer, Felix Feit, Klaus Pohl

Abstract: Online reinforcement learning (RL) is increasingly used for realizing adaptive systems in the presence of design time uncertainty. Online RL facilitates learning from actual operational data and thereby leverages feedback only available at runtime. However, Online RL requires the definition of an effective and correct reward function, which quantifies the feedback to the RL algorithm and thereby g… ▽ More Online reinforcement learning (RL) is increasingly used for realizing adaptive systems in the presence of design time uncertainty. Online RL facilitates learning from actual operational data and thereby leverages feedback only available at runtime. However, Online RL requires the definition of an effective and correct reward function, which quantifies the feedback to the RL algorithm and thereby guides learning. With Deep RL gaining interest, the learned knowledge is no longer explicitly represented, but is represented as a neural network. For a human, it becomes practically impossible to relate the parametrization of the neural network to concrete RL decisions. Deep RL thus essentially appears as a black box, which severely limits the debugging of adaptive systems. We previously introduced the explainable RL technique XRL-DINE, which provides visual insights into why certain decisions were made at important time points. Here, we introduce an empirical user study involving 54 software engineers from academia and industry to assess (1) the performance of software engineers when performing different tasks using XRL-DINE and (2) the perceived usefulness and ease of use of XRL-DINE. △ Less

Submitted 9 July, 2023; originally announced July 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2210.05931

Generating Realistic Brain MRIs via a Conditional Diffusion Probabilistic Model

Authors: Wei Peng, Ehsan Adeli, Tomas Bosschieter, Sang Hyun Park, Qingyu Zhao, Kilian M. Pohl

Abstract: As acquiring MRIs is expensive, neuroscience studies struggle to attain a sufficient number of them for properly training deep learning models. This challenge could be reduced by MRI synthesis, for which Generative Adversarial Networks (GANs) are popular. GANs, however, are commonly unstable and struggle with creating diverse and high-quality data. A more stable alternative is Diffusion Probabilis… ▽ More As acquiring MRIs is expensive, neuroscience studies struggle to attain a sufficient number of them for properly training deep learning models. This challenge could be reduced by MRI synthesis, for which Generative Adversarial Networks (GANs) are popular. GANs, however, are commonly unstable and struggle with creating diverse and high-quality data. A more stable alternative is Diffusion Probabilistic Models (DPMs) with a fine-grained training strategy. To overcome their need for extensive computational resources, we propose a conditional DPM (cDPM) with a memory-efficient process that generates realistic-looking brain MRIs. To this end, we train a 2D cDPM to generate an MRI subvolume conditioned on another subset of slices from the same MRI. By generating slices using arbitrary combinations between condition and target slices, the model only requires limited computational resources to learn interdependencies between slices even if they are spatially far apart. After having learned these dependencies via an attention network, a new anatomy-consistent 3D brain MRI is generated by repeatedly applying the cDPM. Our experiments demonstrate that our method can generate high-quality 3D MRIs that share a similar distribution to real MRIs while still diversifying the training set. The code is available at https://github.com/xiaoiker/mask3DMRI_diffusion and also will be released as part of MONAI, at https://github.com/Project-MONAI/GenerativeModels. △ Less

Submitted 7 September, 2023; v1 submitted 15 December, 2022; originally announced December 2022.

Journal ref: MICCAI 2023

Joint Graph Convolution for Analyzing Brain Structural and Functional Connectome

Authors: Yueting Li, Qingyue Wei, Ehsan Adeli, Kilian M. Pohl, Qingyu Zhao

Abstract: The white-matter (micro-)structural architecture of the brain promotes synchrony among neuronal populations, giving rise to richly patterned functional connections. A fundamental problem for systems neuroscience is determining the best way to relate structural and functional networks quantified by diffusion tensor imaging and resting-state functional MRI. As one of the state-of-the-art approaches… ▽ More The white-matter (micro-)structural architecture of the brain promotes synchrony among neuronal populations, giving rise to richly patterned functional connections. A fundamental problem for systems neuroscience is determining the best way to relate structural and functional networks quantified by diffusion tensor imaging and resting-state functional MRI. As one of the state-of-the-art approaches for network analysis, graph convolutional networks (GCN) have been separately used to analyze functional and structural networks, but have not been applied to explore inter-network relationships. In this work, we propose to couple the two networks of an individual by adding inter-network edges between corresponding brain regions, so that the joint structure-function graph can be directly analyzed by a single GCN. The weights of inter-network edges are learnable, reflecting non-uniform structure-function coupling strength across the brain. We apply our Joint-GCN to predict age and sex of 662 participants from the public dataset of the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) based on their functional and micro-structural white-matter networks. Our results support that the proposed Joint-GCN outperforms existing multi-modal graph learning approaches for analyzing structural and functional networks. △ Less

Submitted 27 October, 2022; originally announced November 2022.

UMLsec4Edge: Extending UMLsec to model data-protection-compliant edge computing systems

Authors: Sven Smolka, Jan Laufer, Zoltán Ádám Mann, Klaus Pohl

Abstract: Edge computing enables the processing of data - frequently personal data - at the edge of the network. For personal data, legislation such as the European General Data Protection Regulation requires data protection by design. Hence, data protection has to be accounted for in the design of edge computing systems whenever personal data is involved. This leads to specific requirements for modeling th… ▽ More Edge computing enables the processing of data - frequently personal data - at the edge of the network. For personal data, legislation such as the European General Data Protection Regulation requires data protection by design. Hence, data protection has to be accounted for in the design of edge computing systems whenever personal data is involved. This leads to specific requirements for modeling the architecture of edge computing systems, e.g., representation of data and network properties. To the best of our knowledge, no existing modeling language fulfils all these requirements. In our previous work we showed that the commonly used UML profile UMLsec fulfils some of these requirements, and can thus serve as a starting point. The aim of this paper is to create a modeling language which meets all requirements concerning the design of the architecture of edge computing systems accounting for data protection. Thus, we extend UMLsec to satisfy all requirements. We call the resulting UML profile UMLsec4Edge. We follow a systematic approach to develop UMLsec4Edge. We apply UMLsec4Edge to real-world use cases from different domains, and create appropriate deployment diagrams and class diagrams. These diagrams show UMLsec4Edge is capable of meeting the requirements. △ Less

Submitted 17 October, 2022; originally announced October 2022.

Comments: 8 pages, 4 figures, 2 tables, published at the 48th Euromicro Conference Series on Software Engineering and Advanced Applications (SEAA)

Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems

Authors: Felix Feit, Andreas Metzger, Klaus Pohl

Abstract: Design time uncertainty poses an important challenge when develo** a self-adaptive system. As an example, defining how the system should adapt when facing a new environment state, requires understanding the precise effect of an adaptation, which may not be known at design time. Online reinforcement learning, i.e., employing reinforcement learning (RL) at runtime, is an emerging approach to reali… ▽ More Design time uncertainty poses an important challenge when develo** a self-adaptive system. As an example, defining how the system should adapt when facing a new environment state, requires understanding the precise effect of an adaptation, which may not be known at design time. Online reinforcement learning, i.e., employing reinforcement learning (RL) at runtime, is an emerging approach to realizing self-adaptive systems in the presence of design time uncertainty. By using Online RL, the self-adaptive system can learn from actual operational data and leverage feedback only available at runtime. Recently, Deep RL is gaining interest. Deep RL represents learned knowledge as a neural network whereby it can generalize over unseen inputs, as well as handle continuous environment states and adaptation actions. A fundamental problem of Deep RL is that learned knowledge is not explicitly represented. For a human, it is practically impossible to relate the parametrization of the neural network to concrete RL decisions and thus Deep RL essentially appears as a black box. Yet, understanding the decisions made by Deep RL is key to (1) increasing trust, and (2) facilitating debugging. Such debugging is especially relevant for self-adaptive systems, because the reward function, which quantifies the feedback to the RL algorithm, must be defined by developers. The reward function must be explicitly defined by developers, thus introducing a potential for human error. To explain Deep RL for self-adaptive systems, we enhance and combine two existing explainable RL techniques from the machine learning literature. The combined technique, XRL-DINE, overcomes the respective limitations of the individual techniques. We present a proof-of-concept implementation of XRL-DINE, as well as qualitative and quantitative results of applying XRL-DINE to a self-adaptive system exemplar. △ Less

Submitted 12 October, 2022; originally announced October 2022.

Comments: Published at 3rd Intl Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2022)

Multiple Instance Neuroimage Transformer

Authors: Ayush Singla, Qingyu Zhao, Daniel K. Do, Yuyin Zhou, Kilian M. Pohl, Ehsan Adeli

Abstract: For the first time, we propose using a multiple instance learning based convolution-free transformer model, called Multiple Instance Neuroimage Transformer (MINiT), for the classification of T1weighted (T1w) MRIs. We first present several variants of transformer models adopted for neuroimages. These models extract non-overlap** 3D blocks from the input volume and perform multi-headed self-attent… ▽ More For the first time, we propose using a multiple instance learning based convolution-free transformer model, called Multiple Instance Neuroimage Transformer (MINiT), for the classification of T1weighted (T1w) MRIs. We first present several variants of transformer models adopted for neuroimages. These models extract non-overlap** 3D blocks from the input volume and perform multi-headed self-attention on a sequence of their linear projections. MINiT, on the other hand, treats each of the non-overlap** 3D blocks of the input MRI as its own instance, splitting it further into non-overlap** 3D patches, on which multi-headed self-attention is computed. As a proof-of-concept, we evaluate the efficacy of our model by training it to identify sex from T1w-MRIs of two public datasets: Adolescent Brain Cognitive Development (ABCD) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA). The learned attention maps highlight voxels contributing to identifying sex differences in brain morphometry. The code is available at https://github.com/singlaayush/MINIT. △ Less

Submitted 19 August, 2022; originally announced August 2022.

Bridging the Gap between Deep Learning and Hypothesis-Driven Analysis via Permutation Testing

Authors: Magdalini Paschali, Qingyu Zhao, Ehsan Adeli, Kilian M. Pohl

Abstract: A fundamental approach in neuroscience research is to test hypotheses based on neuropsychological and behavioral measures, i.e., whether certain factors (e.g., related to life events) are associated with an outcome (e.g., depression). In recent years, deep learning has become a potential alternative approach for conducting such analyses by predicting an outcome from a collection of factors and ide… ▽ More A fundamental approach in neuroscience research is to test hypotheses based on neuropsychological and behavioral measures, i.e., whether certain factors (e.g., related to life events) are associated with an outcome (e.g., depression). In recent years, deep learning has become a potential alternative approach for conducting such analyses by predicting an outcome from a collection of factors and identifying the most "informative" ones driving the prediction. However, this approach has had limited impact as its findings are not linked to statistical significance of factors supporting hypotheses. In this article, we proposed a flexible and scalable approach based on the concept of permutation testing that integrates hypothesis testing into the data-driven deep learning analysis. We apply our approach to the yearly self-reported assessments of 621 adolescent participants of the National Consortium of Alcohol and Neurodevelopment in Adolescence (NCANDA) to predict negative valence, a symptom of major depressive disorder according to the NIMH Research Domain Criteria (RDoC). Our method successfully identifies categories of risk factors that further explain the symptom. △ Less

Submitted 28 July, 2022; originally announced July 2022.

Comments: Accepted at the 5th workshop on PRedictive Intelligence in Medicine (PRIME 2022) - MICCAI 2022

A Penalty Approach for Normalizing Feature Distributions to Build Confounder-Free Models

Authors: Anthony Vento, Qingyu Zhao, Robert Paul, Kilian M. Pohl, Ehsan Adeli

Abstract: Translating machine learning algorithms into clinical applications requires addressing challenges related to interpretability, such as accounting for the effect of confounding variables (or metadata). Confounding variables affect the relationship between input training data and target outputs. When we train a model on such data, confounding variables will bias the distribution of the learned featu… ▽ More Translating machine learning algorithms into clinical applications requires addressing challenges related to interpretability, such as accounting for the effect of confounding variables (or metadata). Confounding variables affect the relationship between input training data and target outputs. When we train a model on such data, confounding variables will bias the distribution of the learned features. A recent promising solution, MetaData Normalization (MDN), estimates the linear relationship between the metadata and each feature based on a non-trainable closed-form solution. However, this estimation is confined by the sample size of a mini-batch and thereby may cause the approach to be unstable during training. In this paper, we extend the MDN method by applying a Penalty approach (referred to as PDMN). We cast the problem into a bi-level nested optimization problem. We then approximate this optimization problem using a penalty method so that the linear parameters within the MDN layer are trainable and learned on all samples. This enables PMDN to be plugged into any architectures, even those unfit to run batch-level operations, such as transformers and recurrent models. We show improvement in model accuracy and greater independence from confounders using PMDN over MDN in a synthetic experiment and a multi-label, multi-site dataset of magnetic resonance images (MRIs). △ Less

Submitted 11 July, 2022; originally announced July 2022.

GaitForeMer: Self-Supervised Pre-Training of Transformers via Human Motion Forecasting for Few-Shot Gait Impairment Severity Estimation

Authors: Mark Endo, Kathleen L. Poston, Edith V. Sullivan, Li Fei-Fei, Kilian M. Pohl, Ehsan Adeli

Abstract: Parkinson's disease (PD) is a neurological disorder that has a variety of observable motor-related symptoms such as slow movement, tremor, muscular rigidity, and impaired posture. PD is typically diagnosed by evaluating the severity of motor impairments according to scoring systems such as the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Automated severity predic… ▽ More Parkinson's disease (PD) is a neurological disorder that has a variety of observable motor-related symptoms such as slow movement, tremor, muscular rigidity, and impaired posture. PD is typically diagnosed by evaluating the severity of motor impairments according to scoring systems such as the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Automated severity prediction using video recordings of individuals provides a promising route for non-intrusive monitoring of motor impairments. However, the limited size of PD gait data hinders model ability and clinical potential. Because of this clinical data scarcity and inspired by the recent advances in self-supervised large-scale language models like GPT-3, we use human motion forecasting as an effective self-supervised pre-training task for the estimation of motor impairment severity. We introduce GaitForeMer, Gait Forecasting and impairment estimation transforMer, which is first pre-trained on public datasets to forecast gait movements and then applied to clinical data to predict MDS-UPDRS gait impairment severity. Our method outperforms previous approaches that rely solely on clinical data by a large margin, achieving an F1 score of 0.76, precision of 0.79, and recall of 0.75. Using GaitForeMer, we show how public human movement data repositories can assist clinical use cases through learning universal motion representations. The code is available at https://github.com/markendo/GaitForeMer . △ Less

Submitted 30 June, 2022; originally announced July 2022.

Comments: Accepted as a conference paper at MICCAI (Medical Image Computing and Computer Assisted Intervention) 2022

Counterfactual Explanations for Predictive Business Process Monitoring

Authors: Tsung-Hao Huang, Andreas Metzger, Klaus Pohl

Abstract: Predictive business process monitoring increasingly leverages sophisticated prediction models. Although sophisticated models achieve consistently higher prediction accuracy than simple models, one major drawback is their lack of interpretability, which limits their adoption in practice. We thus see growing interest in explainable predictive business process monitoring, which aims to increase the i… ▽ More Predictive business process monitoring increasingly leverages sophisticated prediction models. Although sophisticated models achieve consistently higher prediction accuracy than simple models, one major drawback is their lack of interpretability, which limits their adoption in practice. We thus see growing interest in explainable predictive business process monitoring, which aims to increase the interpretability of prediction models. Existing solutions focus on giving factual explanations.While factual explanations can be helpful, humans typically do not ask why a particular prediction was made, but rather why it was made instead of another prediction, i.e., humans are interested in counterfactual explanations. While research in explainable AI produced several promising techniques to generate counterfactual explanations, directly applying them to predictive process monitoring may deliver unrealistic explanations, because they ignore the underlying process constraints. We propose LORELEY, a counterfactual explanation technique for predictive process monitoring, which extends LORE, a recent explainable AI technique. We impose control flow constraints to the explanation generation process to ensure realistic counterfactual explanations. Moreover, we extend LORE to enable explaining multi-class classification models. Experimental results using a real, public dataset indicate that LORELEY can approximate the prediction models with an average fidelity of 97.69\% and generate realistic counterfactual explanations. △ Less

Submitted 24 February, 2022; originally announced February 2022.

Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions

Authors: Martin Hutzenthaler, Arnulf Jentzen, Katharina Pohl, Adrian Riekert, Luca Scarpa

Abstract: In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day it remains an open problem of research to provide a mathematical convergence analysis which rigorously explains the success of SGD type optimization methods in the training of DNNs. In this work we study SGD type optimiz… ▽ More In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day it remains an open problem of research to provide a mathematical convergence analysis which rigorously explains the success of SGD type optimization methods in the training of DNNs. In this work we study SGD type optimization methods in the training of fully-connected feedforward DNNs with rectified linear unit (ReLU) activation. We first establish general regularity properties for the risk functions and their generalized gradient functions appearing in the training of such DNNs and, thereafter, we investigate the plain vanilla SGD optimization method in the training of such DNNs under the assumption that the target function under consideration is a constant function. Specifically, we prove under the assumption that the learning rates (the step sizes of the SGD optimization method) are sufficiently small but not $L^1$-summable and under the assumption that the target function is a constant function that the expectation of the riskof the considered SGD process converges in the training of such DNNs to zero as the number of SGD steps increases to infinity. △ Less

Submitted 22 June, 2023; v1 submitted 13 December, 2021; originally announced December 2021.

Comments: 71 pages, 5 figures, 2 tables, 4 Python source codes. To appear in Electronic Research Archive

Longitudinal Correlation Analysis for Decoding Multi-Modal Brain Development

Authors: Qingyu Zhao, Ehsan Adeli, Kilian M. Pohl

Abstract: Starting from childhood, the human brain restructures and rewires throughout life. Characterizing such complex brain development requires effective analysis of longitudinal and multi-modal neuroimaging data. Here, we propose such an analysis approach named Longitudinal Correlation Analysis (LCA). LCA couples the data of two modalities by first reducing the input from each modality to a latent repr… ▽ More Starting from childhood, the human brain restructures and rewires throughout life. Characterizing such complex brain development requires effective analysis of longitudinal and multi-modal neuroimaging data. Here, we propose such an analysis approach named Longitudinal Correlation Analysis (LCA). LCA couples the data of two modalities by first reducing the input from each modality to a latent representation based on autoencoders. A self-supervised strategy then relates the two latent spaces by jointly disentangling two directions, one in each space, such that the longitudinal changes in latent representations along those directions are maximally correlated between modalities. We applied LCA to analyze the longitudinal T1-weighted and diffusion-weighted MRIs of 679 youths from the National Consortium on Alcohol and Neurodevelopment in Adolescence. Unlike existing approaches that focus on either cross-sectional or single-modal modeling, LCA successfully unraveled coupled macrostructural and microstructural brain development from morphological and diffusivity features extracted from the data. A retesting of LCA on raw 3D image volumes of those subjects successfully replicated the findings from the feature-based analysis. Lastly, the developmental effects revealed by LCA were inline with the current understanding of maturational patterns of the adolescent brain. △ Less

Submitted 9 July, 2021; originally announced July 2021.

Metadata Normalization

Authors: Mandy Lu, Qingyu Zhao, Jiequan Zhang, Kilian M. Pohl, Li Fei-Fei, Juan Carlos Niebles, Ehsan Adeli

Abstract: Batch Normalization (BN) and its variants have delivered tremendous success in combating the covariate shift induced by the training step of deep learning methods. While these techniques normalize feature distributions by standardizing with batch statistics, they do not correct the influence on features from extraneous variables or multiple distributions. Such extra variables, referred to as metad… ▽ More Batch Normalization (BN) and its variants have delivered tremendous success in combating the covariate shift induced by the training step of deep learning methods. While these techniques normalize feature distributions by standardizing with batch statistics, they do not correct the influence on features from extraneous variables or multiple distributions. Such extra variables, referred to as metadata here, may create bias or confounding effects (e.g., race when classifying gender from face images). We introduce the Metadata Normalization (MDN) layer, a new batch-level operation which can be used end-to-end within the training framework, to correct the influence of metadata on feature distributions. MDN adopts a regression analysis technique traditionally used for preprocessing to remove (regress out) the metadata effects on model features during training. We utilize a metric based on distance correlation to quantify the distribution bias from the metadata and demonstrate that our method successfully removes metadata effects on four diverse settings: one synthetic, one 2D image, one video, and one 3D medical image dataset. △ Less

Submitted 5 May, 2021; v1 submitted 19 April, 2021; originally announced April 2021.

Comments: Accepted to CVPR 2021. Project page: https://mml.stanford.edu/MDN/

Self-Supervised Longitudinal Neighbourhood Embedding

Authors: Jiahong Ouyang, Qingyu Zhao, Ehsan Adeli, Edith V Sullivan, Adolf Pfefferbaum, Greg Zaharchuk, Kilian M Pohl

Abstract: Longitudinal MRIs are often used to capture the gradual deterioration of brain structure and function caused by aging or neurological diseases. Analyzing this data via machine learning generally requires a large number of ground-truth labels, which are often missing or expensive to obtain. Reducing the need for labels, we propose a self-supervised strategy for representation learning named Longitu… ▽ More Longitudinal MRIs are often used to capture the gradual deterioration of brain structure and function caused by aging or neurological diseases. Analyzing this data via machine learning generally requires a large number of ground-truth labels, which are often missing or expensive to obtain. Reducing the need for labels, we propose a self-supervised strategy for representation learning named Longitudinal Neighborhood Embedding (LNE). Motivated by concepts in contrastive learning, LNE explicitly models the similarity between trajectory vectors across different subjects. We do so by building a graph in each training iteration defining neighborhoods in the latent space so that the progression direction of a subject follows the direction of its neighbors. This results in a smooth trajectory field that captures the global morphological change of the brain while maintaining the local continuity. We apply LNE to longitudinal T1w MRIs of two neuroimaging studies: a dataset composed of 274 healthy subjects, and Alzheimer's Disease Neuroimaging Initiative (ADNI, N=632). The visualization of the smooth trajectory vector field and superior performance on downstream tasks demonstrate the strength of the proposed method over existing self-supervised methods in extracting information associated with normal aging and in revealing the impact of neurodegenerative disorders. The code is available at \url{https://github.com/ouyangjiahong/longitudinal-neighbourhood-embedding.git}. △ Less

Submitted 17 June, 2021; v1 submitted 5 March, 2021; originally announced March 2021.

Comments: Provisional Accepted by Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021

Representation Disentanglement for Multi-modal brain MR Analysis

Authors: Jiahong Ouyang, Ehsan Adeli, Kilian M. Pohl, Qingyu Zhao, Greg Zaharchuk

Abstract: Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can benefit from explicitly disentangling anatomical (shape) and modality (appearance) information into separate image presentations. In this work, we challenge mainstream strategies… ▽ More Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can benefit from explicitly disentangling anatomical (shape) and modality (appearance) information into separate image presentations. In this work, we challenge mainstream strategies by showing that they do not naturally lead to representation disentanglement both in theory and in practice. To address this issue, we propose a margin loss that regularizes the similarity in relationships of the representations across subjects and modalities. To enable robust training, we further use a conditional convolution to design a single model for encoding images of all modalities. Lastly, we propose a fusion function to combine the disentangled anatomical representations as a set of modality-invariant features for downstream tasks. We evaluate the proposed method on three multi-modal neuroimaging datasets. Experiments show that our proposed method can achieve superior disentangled representations compared to existing disentanglement strategies. Results also indicate that the fused anatomical representation has potential in the downstream task of zero-dose PET reconstruction and brain tumor segmentation. The code is available at \url{https://github.com/ouyangjiahong/representation-disentanglement}. △ Less

Submitted 11 June, 2021; v1 submitted 22 February, 2021; originally announced February 2021.

Comments: Accepted by Information Processing in Medical Imaging (IPMI) 2021

Going Beyond Saliency Maps: Training Deep Models to Interpret Deep Models

Authors: Zixuan Liu, Ehsan Adeli, Kilian M. Pohl, Qingyu Zhao

Abstract: Interpretability is a critical factor in applying complex deep learning models to advance the understanding of brain disorders in neuroimaging studies. To interpret the decision process of a trained classifier, existing techniques typically rely on saliency maps to quantify the voxel-wise or feature-level importance for classification through partial derivatives. Despite providing some level of lo… ▽ More Interpretability is a critical factor in applying complex deep learning models to advance the understanding of brain disorders in neuroimaging studies. To interpret the decision process of a trained classifier, existing techniques typically rely on saliency maps to quantify the voxel-wise or feature-level importance for classification through partial derivatives. Despite providing some level of localization, these maps are not human-understandable from the neuroscience perspective as they do not inform the specific meaning of the alteration linked to the brain disorder. Inspired by the image-to-image translation scheme, we propose to train simulator networks that can warp a given image to inject or remove patterns of the disease. These networks are trained such that the classifier produces consistently increased or decreased prediction logits for the simulated images. Moreover, we propose to couple all the simulators into a unified model based on conditional convolution. We applied our approach to interpreting classifiers trained on a synthetic dataset and two neuroimaging datasets to visualize the effect of the Alzheimer's disease and alcohol use disorder. Compared to the saliency maps generated by baseline approaches, our simulations and visualizations based on the Jacobian determinants of the war** field reveal meaningful and understandable patterns related to the diseases. △ Less

Submitted 25 June, 2021; v1 submitted 16 February, 2021; originally announced February 2021.

Vision-based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson's Disease Motor Severity

Authors: Mandy Lu, Kathleen Poston, Adolf Pfefferbaum, Edith V. Sullivan, Li Fei-Fei, Kilian M. Pohl, Juan Carlos Niebles, Ehsan Adeli

Abstract: Parkinson's disease (PD) is a progressive neurological disorder primarily affecting motor function resulting in tremor at rest, rigidity, bradykinesia, and postural instability. The physical severity of PD impairments can be quantified through the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a widely used clinical rating scale. Accurate and quantitative assessmen… ▽ More Parkinson's disease (PD) is a progressive neurological disorder primarily affecting motor function resulting in tremor at rest, rigidity, bradykinesia, and postural instability. The physical severity of PD impairments can be quantified through the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS), a widely used clinical rating scale. Accurate and quantitative assessment of disease progression is critical to develo** a treatment that slows or stops further advancement of the disease. Prior work has mainly focused on dopamine transport neuroimaging for diagnosis or costly and intrusive wearables evaluating motor impairments. For the first time, we propose a computer vision-based model that observes non-intrusive video recordings of individuals, extracts their 3D body skeletons, tracks them through time, and classifies the movements according to the MDS-UPDRS gait scores. Experimental results show that our proposed method performs significantly better than chance and competing methods with an F1-score of 0.83 and a balanced accuracy of 81%. This is the first benchmark for classifying PD patients based on MDS-UPDRS gait severity and could be an objective biomarker for disease severity. Our work demonstrates how computer-assisted technologies can be used to non-intrusively monitor patients and their motor impairments. The code is available at https://github.com/mlu355/PD-Motor-Severity-Estimation. △ Less

Submitted 17 July, 2020; originally announced July 2020.

Comments: Accepted as a conference paper at MICCAI (Medical Image Computing and Computer Assisted Intervention), Lima, Peru, October 2020. 11 pages, LaTeX

Longitudinal Self-Supervised Learning

Authors: Qingyu Zhao, Zixuan Liu, Ehsan Adeli, Kilian M. Pohl

Abstract: Machine learning analysis of longitudinal neuroimaging data is typically based on supervised learning, which requires a large number of ground-truth labels to be informative. As ground-truth labels are often missing or expensive to obtain in neuroscience, we avoid them in our analysis by combing factor disentanglement with self-supervised learning to identify changes and consistencies across the m… ▽ More Machine learning analysis of longitudinal neuroimaging data is typically based on supervised learning, which requires a large number of ground-truth labels to be informative. As ground-truth labels are often missing or expensive to obtain in neuroscience, we avoid them in our analysis by combing factor disentanglement with self-supervised learning to identify changes and consistencies across the multiple MRIs acquired of each individual over time. Specifically, we propose a new definition of disentanglement by formulating a multivariate map** between factors (e.g., brain age) associated with an MRI and a latent image representation. Then, factors that evolve across acquisitions of longitudinal sequences are disentangled from that map** by self-supervised learning in such a way that changes in a single factor induce change along one direction in the representation space. We implement this model, named Longitudinal Self-Supervised Learning (LSSL), via a standard autoencoding structure with a cosine loss to disentangle brain age from the image representation. We apply LSSL to two longitudinal neuroimaging studies to highlight its strength in extracting the brain-age information from MRI and revealing informative characteristics associated with neurodegenerative and neuropsychological disorders. Moreover, the representations learned by LSSL facilitate supervised classification by recording faster convergence and higher (or similar) prediction accuracy compared to several other representation learning techniques. △ Less

Submitted 26 June, 2021; v1 submitted 11 June, 2020; originally announced June 2020.

Longitudinal Pooling & Consistency Regularization to Model Disease Progression from MRIs

Authors: Jiahong Ouyang, Qingyu Zhao, Edith V Sullivan, Adolf Pfefferbaum, Susan F. Tapert, Ehsan Adeli, Kilian M Pohl

Abstract: Many neurological diseases are characterized by gradual deterioration of brain structure and function. Large longitudinal MRI datasets have revealed such deterioration, in part, by applying machine and deep learning to predict diagnosis. A popular approach is to apply Convolutional Neural Networks (CNN) to extract informative features from each visit of the longitudinal MRI and then use those feat… ▽ More Many neurological diseases are characterized by gradual deterioration of brain structure and function. Large longitudinal MRI datasets have revealed such deterioration, in part, by applying machine and deep learning to predict diagnosis. A popular approach is to apply Convolutional Neural Networks (CNN) to extract informative features from each visit of the longitudinal MRI and then use those features to classify each visit via Recurrent Neural Networks (RNNs). Such modeling neglects the progressive nature of the disease, which may result in clinically implausible classifications across visits. To avoid this issue, we propose to combine features across visits by coupling feature extraction with a novel longitudinal pooling layer and enforce consistency of the classification across visits in line with disease progression. We evaluate the proposed method on the longitudinal structural MRIs from three neuroimaging datasets: Alzheimer's Disease Neuroimaging Initiative (ADNI, N=404), a dataset composed of 274 normal controls and 329 patients with Alcohol Use Disorder (AUD), and 255 youths from the National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA). In all three experiments our method is superior to other widely used approaches for longitudinal classification thus making a unique contribution towards more accurate tracking of the impact of conditions on the brain. The code is available at https://github.com/ouyangjiahong/longitudinal-pooling. △ Less

Submitted 26 May, 2021; v1 submitted 31 March, 2020; originally announced March 2020.

Comments: Accepted by Journal of Biomedical and Health Informatics (JBHI)

Journal ref: IEEE Journal of Biomedical and Health Informatics 2020

Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis

Authors: Soham Gadgil, Qingyu Zhao, Adolf Pfefferbaum, Edith V. Sullivan, Ehsan Adeli, Kilian M. Pohl

Abstract: The Blood-Oxygen-Level-Dependent (BOLD) signal of resting-state fMRI (rs-fMRI) records the temporal dynamics of intrinsic functional networks in the brain. However, existing deep learning methods applied to rs-fMRI either neglect the functional dependency between different brain regions in a network or discard the information in the temporal dynamics of brain activity. To overcome those shortcomin… ▽ More The Blood-Oxygen-Level-Dependent (BOLD) signal of resting-state fMRI (rs-fMRI) records the temporal dynamics of intrinsic functional networks in the brain. However, existing deep learning methods applied to rs-fMRI either neglect the functional dependency between different brain regions in a network or discard the information in the temporal dynamics of brain activity. To overcome those shortcomings, we propose to formulate functional connectivity networks within the context of spatio-temporal graphs. We train a spatio-temporal graph convolutional network (ST-GCN) on short sub-sequences of the BOLD time series to model the non-stationary nature of functional connectivity. Simultaneously, the model learns the importance of graph edges within ST-GCN to gain insight into the functional connectivities contributing to the prediction. In analyzing the rs-fMRI of the Human Connectome Project (HCP, N=1,091) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA, N=773), ST-GCN is significantly more accurate than common approaches in predicting gender and age based on BOLD signals. Furthermore, the brain regions and functional connections significantly contributing to the predictions of our model are important markers according to the neuroscience literature. △ Less

Submitted 28 June, 2021; v1 submitted 23 March, 2020; originally announced March 2020.

Representation Learning with Statistical Independence to Mitigate Bias

Authors: Ehsan Adeli, Qingyu Zhao, Adolf Pfefferbaum, Edith V. Sullivan, Li Fei-Fei, Juan Carlos Niebles, Kilian M. Pohl

Abstract: Presence of bias (in datasets or tasks) is inarguably one of the most critical challenges in machine learning applications that has alluded to pivotal debates in recent years. Such challenges range from spurious associations between variables in medical studies to the bias of race in gender or face recognition systems. Controlling for all types of biases in the dataset curation stage is cumbersome… ▽ More Presence of bias (in datasets or tasks) is inarguably one of the most critical challenges in machine learning applications that has alluded to pivotal debates in recent years. Such challenges range from spurious associations between variables in medical studies to the bias of race in gender or face recognition systems. Controlling for all types of biases in the dataset curation stage is cumbersome and sometimes impossible. The alternative is to use the available data and build models incorporating fair representation learning. In this paper, we propose such a model based on adversarial training with two competing objectives to learn features that have (1) maximum discriminative power with respect to the task and (2) minimal statistical mean dependence with the protected (bias) variable(s). Our approach does so by incorporating a new adversarial loss function that encourages a vanished correlation between the bias and the learned features. We apply our method to synthetic data, medical images (containing task bias), and a dataset for gender classification (containing dataset bias). Our results show that the learned features by our method not only result in superior prediction performance but also are unbiased. The code is available at https://github.com/QingyuZhao/BR-Net/. △ Less

Submitted 20 November, 2020; v1 submitted 8 October, 2019; originally announced October 2019.

Comments: WACV 2021

Confounder-Aware Visualization of ConvNets

Authors: Qingyu Zhao, Ehsan Adeli, Adolf Pfefferbaum, Edith V. Sullivan, Kilian M. Pohl

Abstract: With recent advances in deep learning, neuroimaging studies increasingly rely on convolutional networks (ConvNets) to predict diagnosis based on MR images. To gain a better understanding of how a disease impacts the brain, the studies visualize the salience maps of the ConvNet highlighting voxels within the brain majorly contributing to the prediction. However, these salience maps are generally co… ▽ More With recent advances in deep learning, neuroimaging studies increasingly rely on convolutional networks (ConvNets) to predict diagnosis based on MR images. To gain a better understanding of how a disease impacts the brain, the studies visualize the salience maps of the ConvNet highlighting voxels within the brain majorly contributing to the prediction. However, these salience maps are generally confounded, i.e., some salient regions are more predictive of confounding variables (such as age) than the diagnosis. To avoid such misinterpretation, we propose in this paper an approach that aims to visualize confounder-free saliency maps that only highlight voxels predictive of the diagnosis. The approach incorporates univariate statistical tests to identify confounding effects within the intermediate features learned by ConvNet. The influence from the subset of confounded features is then removed by a novel partial back-propagation procedure. We use this two-step approach to visualize confounder-free saliency maps extracted from synthetic and two real datasets. These experiments reveal the potential of our visualization in producing unbiased model-interpretation. △ Less

Submitted 25 June, 2021; v1 submitted 29 July, 2019; originally announced July 2019.

Feature-Model-Guided Online Learning for Self-Adaptive Systems

Authors: Andreas Metzger, Clément Quinton, Zoltán Ádám Mann, Luciano Baresi, Klaus Pohl

Abstract: A self-adaptive system can modify its own structure and behavior at runtime based on its perception of the environment, of itself and of its requirements. To develop a self-adaptive system, software developers codify knowledge about the system and its environment, as well as how adaptation actions impact on the system. However, the codified knowledge may be insufficient due to design time uncertai… ▽ More A self-adaptive system can modify its own structure and behavior at runtime based on its perception of the environment, of itself and of its requirements. To develop a self-adaptive system, software developers codify knowledge about the system and its environment, as well as how adaptation actions impact on the system. However, the codified knowledge may be insufficient due to design time uncertainty, and thus a self-adaptive system may execute adaptation actions that do not have the desired effect. Online learning is an emerging approach to address design time uncertainty by employing machine learning at runtime. Online learning accumulates knowledge at runtime by, for instance, exploring not-yet executed adaptation actions. We address two specific problems with respect to online learning for self-adaptive systems. First, the number of possible adaptation actions can be very large. Existing online learning techniques randomly explore the possible adaptation actions, but this can lead to slow convergence of the learning process. Second, the possible adaptation actions can change as a result of system evolution. Existing online learning techniques are unaware of these changes and thus do not explore new adaptation actions, but explore adaptation actions that are no longer valid. We propose using feature models to give structure to the set of adaptation actions and thereby guide the exploration process during online learning. Experimental results involving four real-world systems suggest that considering the hierarchical structure of feature models may speed up convergence by 7.2% on average. Considering the differences between feature models before and after an evolution step may speed up convergence by 64.6% on average. [...] △ Less

Submitted 22 July, 2019; originally announced July 2019.

Journal ref: Computing 2022

Variational AutoEncoder For Regression: Application to Brain Aging Analysis

Authors: Qingyu Zhao, Ehsan Adeli, Nicolas Honnorat, Tuo Leng, Kilian M. Pohl

Abstract: While unsupervised variational autoencoders (VAE) have become a powerful tool in neuroimage analysis, their application to supervised learning is under-explored. We aim to close this gap by proposing a unified probabilistic model for learning the latent space of imaging data and performing supervised regression. Based on recent advances in learning disentangled representations, the novel generativ… ▽ More While unsupervised variational autoencoders (VAE) have become a powerful tool in neuroimage analysis, their application to supervised learning is under-explored. We aim to close this gap by proposing a unified probabilistic model for learning the latent space of imaging data and performing supervised regression. Based on recent advances in learning disentangled representations, the novel generative process explicitly models the conditional distribution of latent representations with respect to the regression target variable. Performing a variational inference procedure on this model leads to joint regularization between the VAE and a neural-network regressor. In predicting the age of 245 subjects from their structural Magnetic Resonance (MR) images, our model is more accurate than state-of-the-art methods when applied to either region-of-interest (ROI) measurements or raw 3D volume images. More importantly, unlike simple feed-forward neural-networks, disentanglement of age in latent representations allows for intuitive interpretation of the structural developmental patterns of the human brain. △ Less

Submitted 11 July, 2019; v1 submitted 11 April, 2019; originally announced April 2019.

Truncated Gaussian-Mixture Variational AutoEncoder

Authors: Qingyu Zhao, Nicolas Honnorat, Ehsan Adeli, Kilian M. Pohl

Abstract: Variation Autoencoder (VAE) has become a powerful tool in modeling the non-linear generative process of data from a low-dimensional latent space. Recently, several studies have proposed to use VAE for unsupervised clustering by using mixture models to capture the multi-modal structure of latent representations. This strategy, however, is ineffective when there are outlier data samples whose latent… ▽ More Variation Autoencoder (VAE) has become a powerful tool in modeling the non-linear generative process of data from a low-dimensional latent space. Recently, several studies have proposed to use VAE for unsupervised clustering by using mixture models to capture the multi-modal structure of latent representations. This strategy, however, is ineffective when there are outlier data samples whose latent representations are meaningless, yet contaminating the estimation of key major clusters in the latent space. This exact problem arises in the context of resting-state fMRI (rs-fMRI) analysis, where clustering major functional connectivity patterns is often hindered by heavy noise of rs-fMRI and many minor clusters (rare connectivity patterns) of no interest to analysis. In this paper we propose a novel generative process, in which we use a Gaussian-mixture to model a few major clusters in the data, and use a non-informative uniform distribution to capture the remaining data. We embed this truncated Gaussian-Mixture model in a Variational AutoEncoder framework to obtain a general joint clustering and outlier detection approach, called tGM-VAE. We demonstrated the applicability of tGM-VAE on the MNIST dataset and further validated it in the context of rs-fMRI connectivity analysis. △ Less

Submitted 25 June, 2021; v1 submitted 10 February, 2019; originally announced February 2019.

End-To-End Alzheimer's Disease Diagnosis and Biomarker Identification

Authors: Soheil Esmaeilzadeh, Dimitrios Ioannis Belivanis, Kilian M. Pohl, Ehsan Adeli

Abstract: As shown in computer vision, the power of deep learning lies in automatically learning relevant and powerful features for any perdition task, which is made possible through end-to-end architectures. However, deep learning approaches applied for classifying medical images do not adhere to this architecture as they rely on several pre- and post-processing steps. This shortcoming can be explained by… ▽ More As shown in computer vision, the power of deep learning lies in automatically learning relevant and powerful features for any perdition task, which is made possible through end-to-end architectures. However, deep learning approaches applied for classifying medical images do not adhere to this architecture as they rely on several pre- and post-processing steps. This shortcoming can be explained by the relatively small number of available labeled subjects, the high dimensionality of neuroimaging data, and difficulties in interpreting the results of deep learning methods. In this paper, we propose a simple 3D Convolutional Neural Networks and exploit its model parameters to tailor the end-to-end architecture for the diagnosis of Alzheimer's disease (AD). Our model can diagnose AD with an accuracy of 94.1\% on the popular ADNI dataset using only MRI data, which outperforms the previous state-of-the-art. Based on the learned model, we identify the disease biomarkers, the results of which were in accordance with the literature. We further transfer the learned model to diagnose mild cognitive impairment (MCI), the prodromal stage of AD, which yield better results compared to other methods. △ Less

Submitted 1 October, 2018; originally announced October 2018.

Journal ref: Machine Learning in Medical Imaging. MLMI 2018. Lecture Notes in Computer Science, vol 11046. Springer, Cham

Classification of solvable Leibniz algebras with abelian nilradical and $k-1$ dimensional extension

Authors: R. K. Gaybullaev, A. Kh. Khudoyberdiyev, K. Pohl

Abstract: This work is devoted to the classification of solvable Leibniz algebras with an abelian nilradical. We consider $k-1$ dimensional extension of $k$-dimensional abelian algebras and classify all $2k-1$-dimensional solvable Leibniz algebras with an abelian nilradical of dimension $k$. This work is devoted to the classification of solvable Leibniz algebras with an abelian nilradical. We consider $k-1$ dimensional extension of $k$-dimensional abelian algebras and classify all $2k-1$-dimensional solvable Leibniz algebras with an abelian nilradical of dimension $k$. △ Less

Submitted 19 August, 2018; originally announced August 2018.

MSC Class: 17A32; 17A36; 17A60; 17A65; 17B30

Surface buckling of phosphorene materials: determination, origin and influence on electronic structure

Authors: Zhongwei Dai, Wencan **, Jie-Xiang Yu, Maxwell Grady, Jerzy T. Sadowski, Young Duck Kim, James Hone, Jiadong Zang, Richard M. Osgood, Jr., Karsten Pohl

Abstract: The surface structure of phosphorene crystals materials is determined using surface sensitive dynamical micro-spot low energy electron diffraction (μLEED) analysis using a high spatial resolution low energy electron microscopy (LEEM) system. Samples of (\textit{i}) crystalline cleaved black phosphorus (BP) at 300 K and (\textit{ii}) exfoliated few-layer phosphorene (FLP) of about 10 nm thicknes, w… ▽ More The surface structure of phosphorene crystals materials is determined using surface sensitive dynamical micro-spot low energy electron diffraction (μLEED) analysis using a high spatial resolution low energy electron microscopy (LEEM) system. Samples of (\textit{i}) crystalline cleaved black phosphorus (BP) at 300 K and (\textit{ii}) exfoliated few-layer phosphorene (FLP) of about 10 nm thicknes, which were annealed at 573 K in vacuum were studied. In both samples, a significant surface buckling of 0.22 Å and 0.30 Å, respectively, is measured, which is one order of magnitude larger than previously reported. Using first principle calculations, the presence of surface vacancies is attributed not only to the surface buckling in BP and FLP, but also the previously reported intrinsic hole do** of phosphorene materials. △ Less

Submitted 22 April, 2017; originally announced April 2017.

Journal ref: Phys. Rev. Materials 1, 074003 (2017)

Observation of oscillatory relaxation in the Sn-terminated surface of epitaxial rock-salt SnSe $\{111\}$ topological crystalline insulator

Authors: Wencan **, Suresh Vishwanath, Jianpeng Liu, Lingyuan Kong, Rui Lou, Zhongwei Dai, Jerzy T. Sadowski, Xinyu Liu, Huai-Hsun Lien, Alexander Chaney, Yimo Han, Micheal Cao, Junzhang Ma, Tian Qian, Jerry I. Dadap, Shancai Wang, Malgorzata Dobrowolska, Jacek Furdyna, David A. Muller, Karsten Pohl, Hong Ding, Huili Grace Xing, Richard M. Osgood, Jr

Abstract: Topological crystalline insulators have been recently predicted and observed in rock-salt structure SnSe $\{111\}$ thin films. Previous studies have suggested that the Se-terminated surface of this thin film with hydrogen passivation, has a reduced surface energy and is thus a preferred configuration. In this paper, synchrotron-based angle-resolved photoemission spectroscopy, along with density fu… ▽ More Topological crystalline insulators have been recently predicted and observed in rock-salt structure SnSe $\{111\}$ thin films. Previous studies have suggested that the Se-terminated surface of this thin film with hydrogen passivation, has a reduced surface energy and is thus a preferred configuration. In this paper, synchrotron-based angle-resolved photoemission spectroscopy, along with density functional theory calculations, are used to demonstrate conclusively that a rock-salt SnSe $\{111\}$ thin film epitaxially-grown on \ce{Bi2Se3} has a stable Sn-terminated surface. These observations are supported by low energy electron diffraction (LEED) intensity-voltage measurements and dynamical LEED calculations, which further show that the Sn-terminated SnSe $\{111\}$ thin film has undergone a surface structural relaxation of the interlayer spacing between the Sn and Se atomic planes. In sharp contrast to the Se-terminated counterpart, the observed Dirac surface state in the Sn-terminated SnSe $\{111\}$ thin film is shown to yield a high Fermi velocity, $0.50\times10^6$m/s, which suggests a potential mechanism of engineering the Dirac surface state of topological materials by tuning the surface configuration. △ Less

Submitted 10 April, 2017; originally announced April 2017.

Comments: 12 pages, 13 figures, supplementary materials included

Journal ref: Phys. Rev. X 7, 041020 (2017)

Active Mean Fields for Probabilistic Image Segmentation: Connections with Chan-Vese and Rudin-Osher-Fatemi Models

Authors: Marc Niethammer, Kilian M. Pohl, Firdaus Janoos, William M. Wells III

Abstract: Segmentation is a fundamental task for extracting semantically meaningful regions from an image. The goal of segmentation algorithms is to accurately assign object labels to each image location. However, image-noise, shortcomings of algorithms, and image ambiguities cause uncertainty in label assignment. Estimating the uncertainty in label assignment is important in multiple application domains, s… ▽ More Segmentation is a fundamental task for extracting semantically meaningful regions from an image. The goal of segmentation algorithms is to accurately assign object labels to each image location. However, image-noise, shortcomings of algorithms, and image ambiguities cause uncertainty in label assignment. Estimating the uncertainty in label assignment is important in multiple application domains, such as segmenting tumors from medical images for radiation treatment planning. One way to estimate these uncertainties is through the computation of posteriors of Bayesian models, which is computationally prohibitive for many practical applications. On the other hand, most computationally efficient methods fail to estimate label uncertainty. We therefore propose in this paper the Active Mean Fields (AMF) approach, a technique based on Bayesian modeling that uses a mean-field approximation to efficiently compute a segmentation and its corresponding uncertainty. Based on a variational formulation, the resulting convex model combines any label-likelihood measure with a prior on the length of the segmentation boundary. A specific implementation of that model is the Chan-Vese segmentation model (CV), in which the binary segmentation task is defined by a Gaussian likelihood and a prior regularizing the length of the segmentation boundary. Furthermore, the Euler-Lagrange equations derived from the AMF model are equivalent to those of the popular Rudin-Osher-Fatemi (ROF) model for image denoising. Solutions to the AMF model can thus be implemented by directly utilizing highly-efficient ROF solvers on log-likelihood ratio fields. We qualitatively assess the approach on synthetic data as well as on real natural and medical images. For a quantitative evaluation, we apply our approach to the icgbench dataset. △ Less

Submitted 4 October, 2016; v1 submitted 22 January, 2015; originally announced January 2015.

First-principles theory of low-energy electron diffraction and quantum interference in few-layer graphene

Authors: John F. McClain, Jiebing Sun, Karsten Pohl, Jian-Ming Tang

Abstract: We present a computationally efficient method to incorporate density-functional theory into the calculation of reflectivity in low-energy electron microscopy. The reflectivity is determined by matching plane waves representing the electron beams to the Kohn-Sham wave functions calculated for a finite slab in a supercell. We show that the observed quantum interference effects in the reflectivity sp… ▽ More We present a computationally efficient method to incorporate density-functional theory into the calculation of reflectivity in low-energy electron microscopy. The reflectivity is determined by matching plane waves representing the electron beams to the Kohn-Sham wave functions calculated for a finite slab in a supercell. We show that the observed quantum interference effects in the reflectivity spectra of a few layers of graphene on a substrate can be reproduced well by the calculations using a moderate slab thickness. △ Less

Submitted 12 November, 2013; originally announced November 2013.

Comments: 5 pages, 4 Figures

WESD - Weighted Spectral Distance for Measuring Shape Dissimilarity

Authors: Ender Konukoglu, Ben Glocker, Antonio Criminisi, Kilian M. Pohl

Abstract: This article presents a new distance for measuring shape dissimilarity between objects. Recent publications introduced the use of eigenvalues of the Laplace operator as compact shape descriptors. Here, we revisit the eigenvalues to define a proper distance, called Weighted Spectral Distance (WESD), for quantifying shape dissimilarity. The definition of WESD is derived through analysing the heat-tr… ▽ More This article presents a new distance for measuring shape dissimilarity between objects. Recent publications introduced the use of eigenvalues of the Laplace operator as compact shape descriptors. Here, we revisit the eigenvalues to define a proper distance, called Weighted Spectral Distance (WESD), for quantifying shape dissimilarity. The definition of WESD is derived through analysing the heat-trace. This analysis provides the proposed distance an intuitive meaning and mathematically links it to the intrinsic geometry of objects. We analyse the resulting distance definition, present and prove its important theoretical properties. Some of these properties include: i) WESD is defined over the entire sequence of eigenvalues yet it is guaranteed to converge, ii) it is a pseudometric, iii) it is accurately approximated with a finite number of eigenvalues, and iv) it can be mapped to the [0,1) interval. Lastly, experiments conducted on synthetic and real objects are presented. These experiments highlight the practical benefits of WESD for applications in vision and medical image analysis. △ Less

Submitted 24 August, 2012; originally announced August 2012.

Testing the link between terrestrial climate change and Galactic spiral arm transit

Authors: Andrew C. Overholt, Adrian L. Melott, Martin K. Pohl

Abstract: We re-examine past suggestions of a close link between terrestrial climate change and the Sun's transit of spiral arms in its path through the Milky Way galaxy. These links produced concrete fits, deriving the unknown spiral pattern speed from terrestrial climate correlations. We test these fits against new data on spiral structure based on CO data that does not make simplifying assumptions abou… ▽ More We re-examine past suggestions of a close link between terrestrial climate change and the Sun's transit of spiral arms in its path through the Milky Way galaxy. These links produced concrete fits, deriving the unknown spiral pattern speed from terrestrial climate correlations. We test these fits against new data on spiral structure based on CO data that does not make simplifying assumptions about symmetry and circular rotation. If we compare the times of these transits to changes in the climate of Earth, not only do the claimed correlations disappear, but also we find that they cannot be resurrected for any reasonable pattern speed. △ Less

Submitted 16 September, 2009; v1 submitted 15 June, 2009; originally announced June 2009.

Comments: Accepted for publication in ApJ Letters. Moderate revisions with clarification, no change in conclusions

Journal ref: Astrophys.J.705:L101-L103,2009

Band structure effects on the Be(0001) acoustic-surface-plasmon energy dispersion

Authors: V. M. Silkin, J. M. Pitarke, E. V. Chulkov, B. Diaconescu, K. Pohl, L. Vattuone, L. Savio, Ph. Hoffmann, D. Farias, M. Rocca, P. M. Echenique

Abstract: We report first-principles calculations of acoustic surface plasmons on the (0001) surface of Be, as obtained in the random-phase approximation of many-body theory. The energy dispersion of these collective excitations has been obtained along two symmetry directions. Our results show a considerable anisotropy of acoustic surface plasmons, and underline the capability of experimental measurements… ▽ More We report first-principles calculations of acoustic surface plasmons on the (0001) surface of Be, as obtained in the random-phase approximation of many-body theory. The energy dispersion of these collective excitations has been obtained along two symmetry directions. Our results show a considerable anisotropy of acoustic surface plasmons, and underline the capability of experimental measurements of these plasmons to {\it map} the electron-hole excitation spectrum of the quasi two-dimensional Shockley surface state band that is present on the Be(0001) surface. △ Less

Submitted 14 June, 2008; originally announced June 2008.

Comments: 9 pages, 2 figures

Journal ref: Phys. Stat. Sol. (a) 205, 1307 (2008)

Structural determination of a low-symmetry surface by low-energy electron diffraction and ab initio calculations: Bi(110)

Authors: J. Sun, A. Mikkelsen, M. Fuglsang Jensen, Y. M. Koroteev, G. Bihlmayer, E. V. Chulkov, D. L. Adams, Ph. Hofmann, K. Pohl

Abstract: The surface structure of Bi(110) has been investigated by low-energy electron diffraction (LEED) intensity analysis and by first-principles calculations. Diffraction patterns at a sample temperature of 110 K and normal incidence reveal a bulk truncated (1$\times$1) surface without indication of any structural reconstruction despite the presence of dangling bonds on the surface layer. Good agreem… ▽ More The surface structure of Bi(110) has been investigated by low-energy electron diffraction (LEED) intensity analysis and by first-principles calculations. Diffraction patterns at a sample temperature of 110 K and normal incidence reveal a bulk truncated (1$\times$1) surface without indication of any structural reconstruction despite the presence of dangling bonds on the surface layer. Good agreement is obtained between the calculated and measured diffraction intensities for this complex, low-symmetry surface containing only one mirror-plane symmetry element. No significant interlayer spacing relaxations are found. The Debye temperature for the surface layer is found to be lower than in the bulk, which is indicative of larger vibrational atomic amplitudes at the surface. Meanwhile, the second layer shows a Debye temperature close to the bulk value. The experimental results for the relaxations agree well with those of our first-principles calculation. △ Less

Submitted 30 June, 2006; originally announced June 2006.