-
Signed-Perturbed Sums Estimation of ARX Systems: Exact Coverage and Strong Consistency (Extended Version)
Authors:
Algo Carè,
Erik Weyer,
Balázs Cs. Csáji,
Marco C. Campi
Abstract:
Sign-Perturbed Sums (SPS) is a system identification method that constructs confidence regions for the unknown system parameters. In this paper, we study SPS for ARX systems, and establish that the confidence regions are guaranteed to include the true model parameter with exact, user-chosen, probability under mild statistical assumptions, a property that holds true for any finite number of observe…
▽ More
Sign-Perturbed Sums (SPS) is a system identification method that constructs confidence regions for the unknown system parameters. In this paper, we study SPS for ARX systems, and establish that the confidence regions are guaranteed to include the true model parameter with exact, user-chosen, probability under mild statistical assumptions, a property that holds true for any finite number of observed input-output data. Furthermore, we prove the strong consistency of the method, that is, as the number of data points increases, the confidence region gets smaller and smaller and will asymptotically almost surely exclude any parameter value different from the true one. In addition, we also show that, asymptotically, the SPS region is included in an ellipsoid which is marginally larger than the confidence ellipsoid obtained from the asymptotic theory of system identification. The results are theoretically proven and illustrated in a simulation example.
△ Less
Submitted 18 February, 2024;
originally announced February 2024.
-
Three-dimensional numerical schemes for the segmentation of the psoas muscle in X-ray computed tomography images
Authors:
Giulio Paolucci,
Isabella Cama,
Cristina Campi,
Michele Piana
Abstract:
The analysis of the psoas muscle in morphological and functional imaging has proved to be an accurate approach to assess sarcopenia, i.e. a systemic loss of skeletal muscle mass and function that may be correlated to multifactorial etiological aspects. The inclusion of sarcopenia assessment into a radiological workflow would need the implementation of computational pipelines for image processing t…
▽ More
The analysis of the psoas muscle in morphological and functional imaging has proved to be an accurate approach to assess sarcopenia, i.e. a systemic loss of skeletal muscle mass and function that may be correlated to multifactorial etiological aspects. The inclusion of sarcopenia assessment into a radiological workflow would need the implementation of computational pipelines for image processing that guarantee segmentation reliability and a significant degree of automation. The present study utilizes three-dimensional numerical schemes for psoas segmentation in low-dose X-ray computed tomography images. Specifically, here we focused on the level set methodology and compared the performances of two standard approaches, a classical evolution model and a three-dimension geodesic model, with the performances of an original first-order modification of this latter one. The results of this analysis show that these gradient-based schemes guarantee reliability with respect to manual segmentation and that the first-order scheme requires a computational burden that is significantly smaller than the one needed by the second-order approach.
△ Less
Submitted 10 December, 2023;
originally announced December 2023.
-
A comprehensive theoretical framework for the optimization of neural networks classification performance with respect to weighted metrics
Authors:
Francesco Marchetti,
Sabrina Guastavino,
Cristina Campi,
Federico Benvenuto,
Michele Piana
Abstract:
In many contexts, customized and weighted classification scores are designed in order to evaluate the goodness of the predictions carried out by neural networks. However, there exists a discrepancy between the maximization of such scores and the minimization of the loss function in the training phase. In this paper, we provide a complete theoretical setting that formalizes weighted classification…
▽ More
In many contexts, customized and weighted classification scores are designed in order to evaluate the goodness of the predictions carried out by neural networks. However, there exists a discrepancy between the maximization of such scores and the minimization of the loss function in the training phase. In this paper, we provide a complete theoretical setting that formalizes weighted classification metrics and then allows the construction of losses that drive the model to optimize these metrics of interest. After a detailed theoretical analysis, we show that our framework includes as particular instances well-established approaches such as classical cost-sensitive learning, weighted cross entropy loss functions and value-weighted skill scores.
△ Less
Submitted 22 May, 2023;
originally announced May 2023.
-
Compression, Generalization and Learning
Authors:
Marco C. Campi,
Simone Garatti
Abstract:
A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this pape…
▽ More
A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change is interpreted that this observation brings in extra information and, in learning theory, this corresponds to misclassification, or misprediction. In this paper, we lay the foundations of a new theory that allows one to keep control on the probability of change of compression (which maps into the statistical "risk" in learning applications). Under suitable conditions, the cardinality of the compressed set is shown to be a consistent estimator of the probability of change of compression (without any upper limit on the size of the compressed set); moreover, unprecedentedly tight finite-sample bounds to evaluate the probability of change of compression are obtained under a generally applicable condition of preference. All results are usable in a fully agnostic setup, i.e., without requiring any a priori knowledge on the probability distribution of the observations. Not only these results offer a valid support to develop trust in observation-driven methodologies, they also play a fundamental role in learning techniques as a tool for hyper-parameter tuning.
△ Less
Submitted 8 January, 2024; v1 submitted 30 January, 2023;
originally announced January 2023.
-
Implementation paradigm for supervised flare forecasting studies: a deep learning application with video data
Authors:
Sabrina Guastavino,
Francesco Marchetti,
Federico Benvenuto,
Cristina Campi,
Michele Piana
Abstract:
Solar flare forecasting can be realized by means of the analysis of magnetic data through artificial intelligence techniques. The aim is to predict whether a magnetic active region (AR) will originate solar flares above a certain class within a certain amount of time. A crucial issue is concerned with the way the adopted machine learning method is implemented, since forecasting results strongly de…
▽ More
Solar flare forecasting can be realized by means of the analysis of magnetic data through artificial intelligence techniques. The aim is to predict whether a magnetic active region (AR) will originate solar flares above a certain class within a certain amount of time. A crucial issue is concerned with the way the adopted machine learning method is implemented, since forecasting results strongly depend on the criterion with which training, validation, and test sets are populated. In this paper we propose a general paradigm to generate these sets in such a way that they are independent from each other and internally well-balanced in terms of AR flaring effectiveness. This set generation process provides a ground for comparison for the performance assessment of machine learning algorithms. Finally, we use this implementation paradigm in the case of a deep neural network, which takes as input videos of magnetograms recorded by the Helioseismic and Magnetic Imager on-board the Solar Dynamics Observatory (SDO/HMI). To our knowledge, this is the first time that the solar flare forecasting problem is addressed by means of a deep neural network for video classification, which does not require any a priori extraction of features from the HMI magnetograms.
△ Less
Submitted 24 October, 2021;
originally announced October 2021.
-
Score-oriented loss (SOL) functions
Authors:
Francesco Marchetti,
Sabrina Guastavino,
Michele Piana,
Cristina Campi
Abstract:
Loss functions engineering and the assessment of forecasting performances are two crucial and intertwined aspects of supervised machine learning. This paper focuses on binary classification to introduce a class of loss functions that are defined on probabilistic confusion matrices and that allow an automatic and a priori maximization of the skill scores. The performances of these loss functions ar…
▽ More
Loss functions engineering and the assessment of forecasting performances are two crucial and intertwined aspects of supervised machine learning. This paper focuses on binary classification to introduce a class of loss functions that are defined on probabilistic confusion matrices and that allow an automatic and a priori maximization of the skill scores. The performances of these loss functions are validated during the training phase of two experimental forecasting problems, thus showing that the probability distribution function associated with the confusion matrices significantly impacts the outcome of the score maximization process.
△ Less
Submitted 29 March, 2021;
originally announced March 2021.
-
Oversampling errors in multimodal medical imaging are due to the Gibbs effect
Authors:
Davide Poggiali,
Diego Cecchin,
Cristina Campi,
Stefano De Marchi
Abstract:
To analyse multimodal 3-dimensional medical images, interpolation is required for resampling which - unavoidably - introduces an interpolation error. In this work we consider three segmented 3-dimensional images resampled with three different neuroimaging software tools for comparing undersampling and oversampling strategies and to identify where the oversampling error lies. The results indicate t…
▽ More
To analyse multimodal 3-dimensional medical images, interpolation is required for resampling which - unavoidably - introduces an interpolation error. In this work we consider three segmented 3-dimensional images resampled with three different neuroimaging software tools for comparing undersampling and oversampling strategies and to identify where the oversampling error lies. The results indicate that undersampling to the lowest image size is advantageous in terms of mean value per segment errors and that the oversampling error is larger where the gradient is steeper, showing a Gibbs effect.
△ Less
Submitted 7 May, 2021; v1 submitted 10 March, 2021;
originally announced March 2021.
-
A Theory of the Risk for Optimization with Relaxation and its Application to Support Vector Machines
Authors:
Marco C. Campi,
Simone Garatti
Abstract:
In this paper we consider optimization with relaxation, an ample paradigm to make data-driven designs. This approach was previously considered by the same authors of this work in Garatti and Campi (2019), a study that revealed a deep-seated connection between two concepts: risk (probability of not satisfying a new, out-of-sample, constraint) and complexity (according to a definition introduced in…
▽ More
In this paper we consider optimization with relaxation, an ample paradigm to make data-driven designs. This approach was previously considered by the same authors of this work in Garatti and Campi (2019), a study that revealed a deep-seated connection between two concepts: risk (probability of not satisfying a new, out-of-sample, constraint) and complexity (according to a definition introduced in paper Garatti and Campi (2019)). This connection was shown to have profound implications in applications because it implied that the risk can be estimated from the complexity, a quantity that can be measured from the data without any knowledge of the data-generation mechanism. In the present work we establish new results. First, we expand the scope of Garatti and Campi (2019) so as to embrace a more general setup that covers various algorithms in machine learning. Then, we study classical support vector methods - including SVM (Support Vector Machine), SVR (Support Vector Regression) and SVDD (Support Vector Data Description) - and derive new results for the ability of these methods to generalize. All results are valid for any finite size of the data set. When the sample size tends to infinity, we establish the unprecedented result that the risk approaches the ratio between the complexity and the cardinality of the data sample, regardless of the value of the complexity.
△ Less
Submitted 8 January, 2024; v1 submitted 13 April, 2020;
originally announced April 2020.
-
The scenario approach meets uncertain variational inequalities and game theory
Authors:
Dario Paccagnan,
Marco C. Campi
Abstract:
Variational inequalities are modelling tools used to capture a variety of decision-making problems arising in mathematical optimization, operations research, game theory. The scenario approach is a set of techniques developed to tackle stochastic optimization problems, take decisions based on historical data, and quantify their risk. The overarching goal of this manuscript is to bridge these two a…
▽ More
Variational inequalities are modelling tools used to capture a variety of decision-making problems arising in mathematical optimization, operations research, game theory. The scenario approach is a set of techniques developed to tackle stochastic optimization problems, take decisions based on historical data, and quantify their risk. The overarching goal of this manuscript is to bridge these two areas of research, and thus broaden the class of problems amenable to be studied under the lens of the scenario approach. First and foremost, we provide out-of-samples feasibility guarantees for the solution of variational and quasi variational inequality problems. Second, we apply these results to two classes of uncertain games. In the first class, the uncertainty enters in the constraint sets, while in the second class the uncertainty enters in the cost functions. Finally, we exemplify the quality and relevance of our bounds through numerical simulations on a demand-response model.
△ Less
Submitted 15 March, 2019;
originally announced March 2019.
-
A dynamical model of opinion formation in voting processes under bounded confidence
Authors:
Sergei Yu. Pilyugin,
M. C. Campi
Abstract:
In recent years, opinion dynamics has received an increasing attention, and various models have been introduced and evaluated mainly by simulation. In this study, we introduce and study a dynamical model inspired by the so-called `bounded confidence' approach where voters engaged in an electoral decision with two options are influenced by individuals sharing an opinion similar to their own. This m…
▽ More
In recent years, opinion dynamics has received an increasing attention, and various models have been introduced and evaluated mainly by simulation. In this study, we introduce and study a dynamical model inspired by the so-called `bounded confidence' approach where voters engaged in an electoral decision with two options are influenced by individuals sharing an opinion similar to their own. This model allows one to capture salient features of the evolution of opinions and results in final clusters of voters. The model is nonlinear and discontinuous. We provide a detailed study of the model, including a complete classification of fixed points of the appearing dynamical system and analysis of their stability. It is shown that any trajectory tends to a fixed point. The model highlights that the final electoral outcome depends on the level of interaction in the society, besides the initial opinion of each individual, so that a strongly interconnected society can reverse the electoral outcome as compared to a society with looser exchange.
△ Less
Submitted 6 November, 2018;
originally announced November 2018.
-
Sign-Perturbed Sums (SPS) with Instrumental Variables for the Identification of ARX Systems - Extended Version
Authors:
Valerio Volpe,
Balázs Cs. Csáji,
Algo Carè,
Erik Weyer,
Marco C. Campi
Abstract:
We propose a generalization of the recently developed system identification method called Sign-Perturbed Sums (SPS). The proposed construction is based on the instrumental variables estimate and, unlike the original SPS, it can construct non-asymptotic confidence regions for linear regression models where the regressors contain past values of the output. Hence, it is applicable to ARX systems, as…
▽ More
We propose a generalization of the recently developed system identification method called Sign-Perturbed Sums (SPS). The proposed construction is based on the instrumental variables estimate and, unlike the original SPS, it can construct non-asymptotic confidence regions for linear regression models where the regressors contain past values of the output. Hence, it is applicable to ARX systems, as well as systems with feedback. We show that this approach provides regions with exact confidence under weak assumptions, i.e., the true parameter is included in the regions with a (user-chosen) exact probability for any finite sample. The paper also proves the strong consistency of the method and proposes a computationally efficient generalization of the previously proposed ellipsoidal outer-approximation. Finally, the new method is demonstrated through numerical experiments, using both real-world and simulated data.
△ Less
Submitted 15 September, 2015;
originally announced September 2015.
