-
Maximally informative feature selection using Information Imbalance: Application to COVID-19 severity prediction
Authors:
Romina Wild,
Emanuela Sozio,
Riccardo G. Margiotta,
Fabiana Dellai,
Angela Acquasanta,
Fabio Del Ben,
Carlo Tascini,
Francesco Curcio,
Alessandro Laio
Abstract:
Clinical databases typically include, for each patient, many heterogeneous features, for example blood exams, the clinical history before the onset of the disease, the evolution of the symptoms, the results of imaging exams, and many others. We here propose to exploit a recently developed statistical approach, the Information Imbalance, to compare different subsets of patient features, and automat…
▽ More
Clinical databases typically include, for each patient, many heterogeneous features, for example blood exams, the clinical history before the onset of the disease, the evolution of the symptoms, the results of imaging exams, and many others. We here propose to exploit a recently developed statistical approach, the Information Imbalance, to compare different subsets of patient features, and automatically select the set of features which is maximally informative for a given clinical purpose, especially in minority classes. We adapt the Information Imbalance approach to work in a clinical framework, where patient features are often categorical and are generally available only for a fraction of the patients. We apply this algorithm to a data set of ~ 1,300 patients treated for COVID-19 in Udine hospital before October 2021. Using this approach, we find combinations of features which, if used in combination, are maximally informative of the clinical fate and of the severity of the disease. The optimal number of features, which is determined automatically, turns out to be between 10 and 15. These features can be measured at admission. The approach can be used also if the features are available only for a fraction of the patients, does not require imputation and, importantly, is able to automatically select features with small inter-feature correlation. Clinical insights deriving from this study are also discussed.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
Attacks on Online Learners: a Teacher-Student Analysis
Authors:
Riccardo Giuseppe Margiotta,
Sebastian Goldt,
Guido Sanguinetti
Abstract:
Machine learning models are famously vulnerable to adversarial attacks: small ad-hoc perturbations of the data that can catastrophically alter the model predictions. While a large literature has studied the case of test-time attacks on pre-trained models, the important case of attacks in an online learning setting has received little attention so far. In this work, we use a control-theoretical per…
▽ More
Machine learning models are famously vulnerable to adversarial attacks: small ad-hoc perturbations of the data that can catastrophically alter the model predictions. While a large literature has studied the case of test-time attacks on pre-trained models, the important case of attacks in an online learning setting has received little attention so far. In this work, we use a control-theoretical perspective to study the scenario where an attacker may perturb data labels to manipulate the learning dynamics of an online learner. We perform a theoretical analysis of the problem in a teacher-student setup, considering different attack strategies, and obtaining analytical results for the steady state of simple linear learners. These results enable us to prove that a discontinuous transition in the learner's accuracy occurs when the attack strength exceeds a critical threshold. We then study empirically attacks on learners with complex architectures using real data, confirming the insights of our theoretical analysis. Our findings show that greedy attacks can be extremely efficient, especially when data stream in small batches.
△ Less
Submitted 29 October, 2023; v1 submitted 18 May, 2023;
originally announced May 2023.
-
Glassy dynamics on networks: local spectra and return probabilities
Authors:
Riccardo Giuseppe Margiotta,
Reimer Kühn,
Peter Sollich
Abstract:
The slow relaxation and aging of glassy systems can be modelled as a Markov process on a simplified rough energy landscape: energy minima where the system tends to get trapped are taken as nodes of a random network, and the dynamics are governed by the transition rates among these. In this work we consider the case of purely activated dynamics, where the transition rates only depend on the depth o…
▽ More
The slow relaxation and aging of glassy systems can be modelled as a Markov process on a simplified rough energy landscape: energy minima where the system tends to get trapped are taken as nodes of a random network, and the dynamics are governed by the transition rates among these. In this work we consider the case of purely activated dynamics, where the transition rates only depend on the depth of the departing trap. The random connectivity and the disorder in the trap depths make it impossible to solve the model analytically, so we base our analysis on the spectrum of eigenvalues $λ$ of the master operator. We compute the local density of states $ρ(λ|τ)$ for traps with a fixed lifetime $τ$ by means of the cavity method. This exhibits a power law behaviour $ρ(λ|τ)\simτ|λ|^T$ in the regime of small relaxation rates $|λ|$, which we rationalize using a simple analytical approximation. In the time domain, we find that the probabilities of return to a starting node have a power law-tail that is determined by the distribution of excursion times $F(t)\sim t^{-(T+1)}$. We show that these results arise only by the combination of finite configuration space connectivity and glassy disorder, and interpret them in a simple physical picture dominated by jumps to deep neighbouring traps.
△ Less
Submitted 18 June, 2019;
originally announced June 2019.
-
Spectral properties of the trap model on sparse networks
Authors:
Riccardo Giuseppe Margiotta,
Reimer Kühn,
Peter Sollich
Abstract:
One of the simplest models for the slow relaxation and aging of glasses is the trap model by Bouchaud and others, which represents a system as a point in configuration-space hop** between local energy minima. The time evolution depends on the transition rates and the network of allowed jumps between the minima. We consider the case of sparse configuration-space connectivity given by a random gra…
▽ More
One of the simplest models for the slow relaxation and aging of glasses is the trap model by Bouchaud and others, which represents a system as a point in configuration-space hop** between local energy minima. The time evolution depends on the transition rates and the network of allowed jumps between the minima. We consider the case of sparse configuration-space connectivity given by a random graph, and study the spectral properties of the resulting master operator. We develop a general approach using the cavity method that gives access to the density of states in large systems, as well as localisation properties of the eigenvectors, which are important for the dynamics. We illustrate how, for a system with sparse connectivity and finite temperature, the density of states and the average inverse participation ratio have attributes that arise from a non-trivial combination of the corresponding mean field (fully connected) and random walk (infinite temperature) limits. In particular, we find a range of eigenvalues for which the density of states is of mean-field form but localisation properties are not, and speculate that the corresponding eigenvectors may be concentrated on extensively many clusters of network sites.
△ Less
Submitted 18 May, 2018; v1 submitted 27 February, 2018;
originally announced February 2018.
