Showing 1–10 of 10 results for author: Scarpiniti, M

AFAFed -- Protocol analysis

Authors: Enzo Baccarelli, Michele Scarpiniti, Alireza Momenzadeh, Sima Sarv Ahrabi

Abstract: In this paper, we design, analyze the convergence properties and address the implementation aspects of AFAFed. This is a novel Asynchronous Fair Adaptive Federated learning framework for stream-oriented IoT application environments, which are featured by time-varying operating conditions, heterogeneous resource-limited devices (i.e., coworkers), non-i.i.d. local training data and unreliable commun… ▽ More In this paper, we design, analyze the convergence properties and address the implementation aspects of AFAFed. This is a novel Asynchronous Fair Adaptive Federated learning framework for stream-oriented IoT application environments, which are featured by time-varying operating conditions, heterogeneous resource-limited devices (i.e., coworkers), non-i.i.d. local training data and unreliable communication links. The key new of AFAFed is the synergic co-design of: (i) two sets of adaptively tuned tolerance thresholds and fairness coefficients at the coworkers and central server, respectively; and, (ii) a distributed adaptive mechanism, which allows each coworker to adaptively tune own communication rate. The convergence properties of AFAFed under (possibly) non-convex loss functions is guaranteed by a set of new analytical bounds, which formally unveil the impact on the resulting AFAFed convergence rate of a number of Federated Learning (FL) parameters, like, first and second moments of the per-coworker number of consecutive model updates, data skewness, communication packet-loss probability, and maximum/minimum values of the (adaptively tuned) mixing coefficient used for model aggregation. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Gomoku: analysis of the game and of the player Wine

Authors: Lorenzo Piazzo, Michele Scarpiniti, Enzo Baccarelli

Abstract: Gomoku, also known as five in a row, is a classical board game, ideally suited for quickly testing novel Artificial Intelligence (AI) techniques. With the aim of facilitating a developer willing to write a new Gomoku player, in this report we present an analysis of the main game concepts and strategies, which is wider and deeper than existing ones. Moreover, after discussing the general structure… ▽ More Gomoku, also known as five in a row, is a classical board game, ideally suited for quickly testing novel Artificial Intelligence (AI) techniques. With the aim of facilitating a developer willing to write a new Gomoku player, in this report we present an analysis of the main game concepts and strategies, which is wider and deeper than existing ones. Moreover, after discussing the general structure of an artificial player, we present and analyse a strong Gomoku player, named Wine, the code of which is freely available on the Internet and which is an excelent example of how a modern player is organised. △ Less

Submitted 1 November, 2021; originally announced November 2021.

Comments: 32 pages, 1 figure

A New Class of Efficient Adaptive Filters for Online Nonlinear Modeling

Authors: Danilo Comminiello, Alireza Nezamdoust, Simone Scardapane, Michele Scarpiniti, Amir Hussain, Aurelio Uncini

Abstract: Nonlinear models are known to provide excellent performance in real-world applications that often operate in non-ideal conditions. However, such applications often require online processing to be performed with limited computational resources. To address this problem, we propose a new class of efficient nonlinear models for online applications. The proposed algorithms are based on linear-in-the-pa… ▽ More Nonlinear models are known to provide excellent performance in real-world applications that often operate in non-ideal conditions. However, such applications often require online processing to be performed with limited computational resources. To address this problem, we propose a new class of efficient nonlinear models for online applications. The proposed algorithms are based on linear-in-the-parameters (LIP) nonlinear filters using functional link expansions. In order to make this class of functional link adaptive filters (FLAFs) efficient, we propose low-complexity expansions and frequency-domain adaptation of the parameters. Among this family of algorithms, we also define the partitioned-block frequency-domain FLAF, whose implementation is particularly suitable for online nonlinear modeling problems. We assess and compare frequency-domain FLAFs with different expansions providing the best possible tradeoff between performance and computational complexity. Experimental results prove that the proposed algorithms can be considered as an efficient and effective solution for online applications, such as the acoustic echo cancellation, even in the presence of adverse nonlinear conditions and with limited availability of computational resources. △ Less

Submitted 26 August, 2022; v1 submitted 19 April, 2021; originally announced April 2021.

Comments: This work has been accepted for publication in IEEE Transactions on Systems, Man, and Cybernetics: Systems. Copyright may be transferred without notice, after which this version may no longer be accessible

Why should we add early exits to neural networks?

Authors: Simone Scardapane, Michele Scarpiniti, Enzo Baccarelli, Aurelio Uncini

Abstract: Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant redu… ▽ More Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them. △ Less

Submitted 23 June, 2020; v1 submitted 27 April, 2020; originally announced April 2020.

Comments: Published in Cognitive Computation

Journal ref: Cognitive Computation, 2020

A Multimodal Deep Network for the Reconstruction of T2W MR Images

Authors: Antonio Falvo, Danilo Comminiello, Simone Scardapane, Michele Scarpiniti, Aurelio Uncini

Abstract: Multiple sclerosis is one of the most common chronic neurological diseases affecting the central nervous system. Lesions produced by the MS can be observed through two modalities of magnetic resonance (MR), known as T2W and FLAIR sequences, both providing useful information for formulating a diagnosis. However, long acquisition time makes the acquired MR image vulnerable to motion artifacts. This… ▽ More Multiple sclerosis is one of the most common chronic neurological diseases affecting the central nervous system. Lesions produced by the MS can be observed through two modalities of magnetic resonance (MR), known as T2W and FLAIR sequences, both providing useful information for formulating a diagnosis. However, long acquisition time makes the acquired MR image vulnerable to motion artifacts. This leads to the need of accelerating the execution of the MR analysis. In this paper, we present a deep learning method that is able to reconstruct subsampled MR images obtained by reducing the k-space data, while maintaining a high image quality that can be used to observe brain lesions. The proposed method exploits the multimodal approach of neural networks and it also focuses on the data acquisition and processing stages to reduce execution time of the MR analysis. Results prove the effectiveness of the proposed method in reconstructing subsampled MR images while saving execution time. △ Less

Submitted 24 February, 2020; v1 submitted 8 August, 2019; originally announced August 2019.

Comments: 29th Italian Neural Networks Workshop (WIRN 2019)

Journal ref: Progresses in Artificial Intelligence and Neural Systems. Smart Innovation, Systems and Technologies, vol 184. Springer, Singapore, Jul. 2020

Efficient data augmentation using graph imputation neural networks

Authors: Indro Spinelli, Simone Scardapane, Michele Scarpiniti, Aurelio Uncini

Abstract: Recently, data augmentation in the semi-supervised regime, where unlabeled data vastly outnumbers labeled data, has received a considerable attention. In this paper, we describe an efficient technique for this task, exploiting a recent framework we proposed for missing data imputation called graph imputation neural network (GINN). The key idea is to leverage both supervised and unsupervised data t… ▽ More Recently, data augmentation in the semi-supervised regime, where unlabeled data vastly outnumbers labeled data, has received a considerable attention. In this paper, we describe an efficient technique for this task, exploiting a recent framework we proposed for missing data imputation called graph imputation neural network (GINN). The key idea is to leverage both supervised and unsupervised data to build a graph of similarities between points in the dataset. Then, we augment the dataset by severely damaging a few of the nodes (up to 80\% of their features), and reconstructing them using a variation of GINN. On several benchmark datasets, we show that our method can obtain significant improvements compared to a fully-supervised model, and we are able to augment the datasets up to a factor of 10x. This points to the power of graph-based neural networks to represent structural affinities in the samples for tasks of data reconstruction and augmentation. △ Less

Submitted 20 June, 2019; originally announced June 2019.

Comments: Presented at the 2019 Italian Workshop on Neural Networks (WIRN'19)

EcoMobiFog -- Design and Dynamic Optimization of a 5G Mobile-Fog-Cloud Multi-Tier Ecosystem for the Real-Time Distributed Execution of Stream Applications

Authors: Enzo Baccarelli, Michele Scarpiniti, Alireza Momenzadeh

Abstract: The emerging 5G paradigm will enable multi-radio smartphones to run high-rate stream applications. However, since current smartphones remain resource and battery-limited, the 5G era opens new challenges on how to actually support these applications. In principle, the service orchestration capability of the Fog and Cloud Computing paradigms could be an effective means of dynamically providing resou… ▽ More The emerging 5G paradigm will enable multi-radio smartphones to run high-rate stream applications. However, since current smartphones remain resource and battery-limited, the 5G era opens new challenges on how to actually support these applications. In principle, the service orchestration capability of the Fog and Cloud Computing paradigms could be an effective means of dynamically providing resource-augmentation to smartphones. Motivated by these considerations, the peculiar focus of this paper is on the joint and adaptive optimization of the resource and task allocations of mobile stream applications in 5G-supported multi-tier Mobile-Fog-Cloud virtualized ecosystems. The objective is the minimization of the computing-plus-network energy of the overall ecosystem under hard constraints on the minimum streaming rate and the maximum computing-plus-networking resources. To this end: 1) we model the target ecosystem energy by explicitly accounting for the virtualized and multi-core nature of the Fog/Cloud servers; 2) since the resulting problem is non-convex and involves both continuous and discrete variables, we develop an optimality-preserving decomposition into the cascade of a (continuous) resource allocation sub-problem and a (discrete) task-allocation sub-problem; and 3) we numerically solve the first sub-problem through a suitably designed set of gradient-based adaptive iterations, while we approach the solution of the second sub-problem by resorting to an ad-hoc-developed elitary Genetic algorithm. Finally, we design the main blocks of EcoMobiFog, a technological virtualized platform for supporting the developed solver. The extensive numerical tests confirm that the energy-delay performance of the proposed solving framework is typically within a few per-cent the benchmark one of the exhaustive search-based solution. △ Less

Submitted 18 June, 2019; originally announced June 2019.

Comments: This is a longer version of the published paper on IEEE Access

Journal ref: IEEE Access, Vol. 7, pp. 55565-55608, 2019

Fog-supported delay-constrained energy-saving live migration of VMs over MultiPath TCP/IP 5G connections

Authors: Enzo Baccarelli, Michele Scarpiniti, Alireza Momenzadeh

Abstract: The incoming era of the Fifth-Generation Fog Computing-supported Radio Access Networks (shortly, 5G FOGRANs) aims at exploiting computing/networking resource virtualization, in order to augment the limited resources of wireless devices through the seamless live migration of Virtual Machines (VMs) towards nearby Fog data centers. For this purpose, the bandwidths of the multiple Wireless Network Int… ▽ More The incoming era of the Fifth-Generation Fog Computing-supported Radio Access Networks (shortly, 5G FOGRANs) aims at exploiting computing/networking resource virtualization, in order to augment the limited resources of wireless devices through the seamless live migration of Virtual Machines (VMs) towards nearby Fog data centers. For this purpose, the bandwidths of the multiple Wireless Network Interface Cards (WNICs) of the wireless devices may be aggregated under the control of the emerging MultiPathTCP (MPTCP) protocol. However, due to fading and mobility-induced phenomena, the energy consumptions of current state-of-the-art VM migration techniques may still offset their expected benefits. Motivated by these considerations, in this paper, we analytically characterize, implement in software and numerically test the optimal minimum-energy Settable-Complexity Bandwidth Manager (SCBM) for the live migration of VMs over 5G FOGRAN MPTCP connections. The key features of the proposed SCBM are that: (i) its implementation complexity is settable on-line on the basis of the target energy consumption-vs.-implementation complexity tradeoff; (ii) it minimizes the network energy consumed by the wireless device for sustaining the migration process under hard constraints on the tolerated migration times and downtimes; and, (iii) by leveraging a suitably designed adaptive mechanism, it is capable to quickly react to (possibly, unpredicted) fading and/or mobility-induced abrupt changes of the wireless environment without requiring forecasting. The actual effectiveness of the proposed SCBM is supported by extensive energy-vs.-delay performance comparisons, that cover: (i) a number of heterogeneous 3G/4G/WiFi FOGRAN scenarios; (ii) synthetic and real-world workloads; and, (iii) MPTCP and SinglePathTCP (SPTCP) wireless connections. △ Less

Submitted 31 May, 2018; originally announced May 2018.

Comments: A shorter version of this paper has been submitted to IEEE Access

Effective Blind Source Separation Based on the Adam Algorithm

Authors: Michele Scarpiniti, Simone Scardapane, Danilo Comminiello, Raffaele Parisi, Aurelio Uncini

Abstract: In this paper, we derive a modified InfoMax algorithm for the solution of Blind Signal Separation (BSS) problems by using advanced stochastic methods. The proposed approach is based on a novel stochastic optimization approach known as the Adaptive Moment Estimation (Adam) algorithm. The proposed BSS solution can benefit from the excellent properties of the Adam approach. In order to derive the new… ▽ More In this paper, we derive a modified InfoMax algorithm for the solution of Blind Signal Separation (BSS) problems by using advanced stochastic methods. The proposed approach is based on a novel stochastic optimization approach known as the Adaptive Moment Estimation (Adam) algorithm. The proposed BSS solution can benefit from the excellent properties of the Adam approach. In order to derive the new learning rule, the Adam algorithm is introduced in the derivation of the cost function maximization in the standard InfoMax algorithm. The natural gradient adaptation is also considered. Finally, some experimental results show the effectiveness of the proposed approach. △ Less

Submitted 26 September, 2016; v1 submitted 25 May, 2016; originally announced May 2016.

Comments: Revised version after review process. This paper has been presented at the 26-th Italian Workshop on Neural Networks (WIRN2016) May 18-20, Vietri sul Mare, Salerno, Italy. It will be published soon as a chapter in a book of the the Springer Smart Innovation, Systems and Technologies series

Learning activation functions from data using cubic spline interpolation

Authors: Simone Scardapane, Michele Scarpiniti, Danilo Comminiello, Aurelio Uncini

Abstract: Neural networks require a careful design in order to perform properly on a given task. In particular, selecting a good activation function (possibly in a data-dependent fashion) is a crucial step, which remains an open problem in the research community. Despite a large amount of investigations, most current implementations simply select one fixed function from a small set of candidates, which is n… ▽ More Neural networks require a careful design in order to perform properly on a given task. In particular, selecting a good activation function (possibly in a data-dependent fashion) is a crucial step, which remains an open problem in the research community. Despite a large amount of investigations, most current implementations simply select one fixed function from a small set of candidates, which is not adapted during training, and is shared among all neurons throughout the different layers. However, neither two of these assumptions can be supposed optimal in practice. In this paper, we present a principled way to have data-dependent adaptation of the activation functions, which is performed independently for each neuron. This is achieved by leveraging over past and present advances on cubic spline interpolation, allowing for local adaptation of the functions around their regions of use. The resulting algorithm is relatively cheap to implement, and overfitting is counterbalanced by the inclusion of a novel dam** criterion, which penalizes unwanted oscillations from a predefined shape. Experimental results validate the proposal over two well-known benchmarks. △ Less

Submitted 11 May, 2017; v1 submitted 18 May, 2016; originally announced May 2016.

Comments: Submitted to the 27th Italian Workshop on Neural Networks (WIRN 2017)

Journal ref: Neural Advances in Processing Nonlinear Dynamic Signals, 2017