-
Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging
Authors:
Bradley T. Baker,
Vince D. Calhoun,
Sergey M. Plis
Abstract:
Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In N…
▽ More
Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In Neuroimaging, the understanding of how such phenomena emerge is fundamental to preventing and informing users of the potential risks involved in practice. In this work, we present a novel introspection framework for Deep Learning on Neuroimaging data, which exploits the natural structure of gradient computations via the singular value decomposition of gradient components during reverse-mode auto-differentiation. Unlike post-hoc introspection techniques, which require fully-trained models for evaluation, our method allows for the study of training dynamics on the fly, and even more interestingly, allow for the decomposition of gradients based on which samples belong to particular groups of interest. We demonstrate how the gradient spectra for several common deep learning models differ between schizophrenia and control participants from the COBRE study, and illustrate how these trajectories may reveal specific training dynamics helpful for further analysis.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
Space-Filling Subset Selection for an Electric Battery Model
Authors:
Philipp Gesner,
Christian Gletter,
Florian Landenberger,
Frank Kirschbaum,
Lutz Morawietz,
Bernard Bäker
Abstract:
Dynamic models of the battery performance are an essential tool throughout the development process of automotive drive trains. The present study introduces a method making a large data set suitable for modeling the electrical impedance. When obtaining data-driven models, a usual assumption is that more observations produce better models. However, real driving data on the battery's behavior represe…
▽ More
Dynamic models of the battery performance are an essential tool throughout the development process of automotive drive trains. The present study introduces a method making a large data set suitable for modeling the electrical impedance. When obtaining data-driven models, a usual assumption is that more observations produce better models. However, real driving data on the battery's behavior represent a strongly non-uniform excitation of the system, which negatively affects the modeling. For that reason, a subset selection of the available data was developed. It aims at building accurate nonlinear autoregressive exogenous (NARX) models more efficiently. The algorithm selects those dynamic data points that fill the input space of the nonlinear model more homogeneously. It is shown, that this reduction of the training data leads to a higher model quality in comparison to a random subset and a faster training compared to modeling using all data points.
△ Less
Submitted 7 December, 2020;
originally announced December 2020.
-
Improved Differentially Private Decentralized Source Separation for fMRI Data
Authors:
Hafiz Imtiaz,
Jafar Mohammadi,
Rogers Silva,
Bradley Baker,
Sergey M. Plis,
Anand D. Sarwate,
Vince Calhoun
Abstract:
Blind source separation algorithms such as independent component analysis (ICA) are widely used in the analysis of neuroimaging data. In order to leverage larger sample sizes, different data holders/sites may wish to collaboratively learn feature representations. However, such datasets are often privacy-sensitive, precluding centralized analyses that pool the data at a single site. In this work, w…
▽ More
Blind source separation algorithms such as independent component analysis (ICA) are widely used in the analysis of neuroimaging data. In order to leverage larger sample sizes, different data holders/sites may wish to collaboratively learn feature representations. However, such datasets are often privacy-sensitive, precluding centralized analyses that pool the data at a single site. In this work, we propose a differentially private algorithm for performing ICA in a decentralized data setting. Conventional approaches to decentralized differentially private algorithms may introduce too much noise due to the typically small sample sizes at each site. We propose a novel protocol that uses correlated noise to remedy this problem. We show that our algorithm outperforms existing approaches on synthetic and real neuroimaging datasets and demonstrate that it can sometimes reach the same level of utility as the corresponding non-private algorithm. This indicates that it is possible to have meaningful utility while preserving privacy.
△ Less
Submitted 22 February, 2021; v1 submitted 28 October, 2019;
originally announced October 2019.
