Showing 1–50 of 57 results for author: Livi, L

Graph state-space models

Authors: Daniele Zambon, Andrea Cini, Lorenzo Livi, Cesare Alippi

Abstract: State-space models constitute an effective modeling tool to describe multivariate time series and operate by maintaining an updated representation of the system state from which predictions are made. Within this framework, relational inductive biases, e.g., associated with functional dependencies existing among signals, are not explicitly exploited leaving unattended great opportunities for effect… ▽ More State-space models constitute an effective modeling tool to describe multivariate time series and operate by maintaining an updated representation of the system state from which predictions are made. Within this framework, relational inductive biases, e.g., associated with functional dependencies existing among signals, are not explicitly exploited leaving unattended great opportunities for effective modeling approaches. The manuscript aims, for the first time, at filling this gap by matching state-space modeling and spatio-temporal data where the relational information, say the functional graph capturing latent dependencies, is learned directly from data and is allowed to change over time. Within a probabilistic formulation that accounts for the uncertainty in the data-generating process, an encoder-decoder architecture is proposed to learn the state-space model end-to-end on a downstream task. The proposed methodological framework generalizes several state-of-the-art methods and demonstrates to be effective in extracting meaningful relational information while achieving optimal forecasting performance in controlled environments. △ Less

Submitted 4 January, 2023; originally announced January 2023.

Embedding and trajectories of temporal networks

Authors: Chanon Thongprayoon, Lorenzo Livi, Naoki Masuda

Abstract: Temporal network data are increasingly available in various domains, and often represent highly complex systems with intricate structural and temporal evolutions. Due to the difficulty of processing such complex data, it may be useful to coarse grain temporal network data into a numeric trajectory embedded in a low-dimensional space. We refer to such a procedure as temporal network embedding, whic… ▽ More Temporal network data are increasingly available in various domains, and often represent highly complex systems with intricate structural and temporal evolutions. Due to the difficulty of processing such complex data, it may be useful to coarse grain temporal network data into a numeric trajectory embedded in a low-dimensional space. We refer to such a procedure as temporal network embedding, which is distinct from procedures that aim at embedding individual nodes. Temporal network embedding is a challenging task because we often have access only to discrete time-stamped events between node pairs, and, in general, the events occur with irregular intervals, making the construction of the network at a given time a nontrivial question already. We propose a method to generate trajectories of temporal networks embedded in a low-dimensional space given a sequence of time-stamped events as input. We realize this goal by combining the landmark multidimensional scaling, which is an out-of-sample extension of the well-known multidimensional scaling method, and the framework of tie-decay temporal networks. This combination enables us to obtain a continuous-time trajectory describing the evolution of temporal networks. We then study mathematical properties of the proposed temporal network embedding framework. Finally, we showcase the method with empirical data of social contacts to find temporal organization of contact events and loss of them over a single day and across different days. △ Less

Submitted 4 May, 2023; v1 submitted 4 August, 2022; originally announced August 2022.

Comments: 25 pages, 10 figures

Journal ref: in IEEE Access, vol. 11, pp. 41426-41443, 2023

Transferring Chemical and Energetic Knowledge Between Molecular Systems with Machine Learning

Authors: Sajjad Heydari, Stefano Raniolo, Lorenzo Livi, Vittorio Limongelli

Abstract: Predicting structural and energetic properties of a molecular system is one of the fundamental tasks in molecular simulations, and it has use cases in chemistry, biology, and medicine. In the past decade, the advent of machine learning algorithms has impacted on molecular simulations for various tasks, including property prediction of atomistic systems. In this paper, we propose a novel methodolog… ▽ More Predicting structural and energetic properties of a molecular system is one of the fundamental tasks in molecular simulations, and it has use cases in chemistry, biology, and medicine. In the past decade, the advent of machine learning algorithms has impacted on molecular simulations for various tasks, including property prediction of atomistic systems. In this paper, we propose a novel methodology for transferring knowledge obtained from simple molecular systems to a more complex one, possessing a significantly larger number of atoms and degrees of freedom. In particular, we focus on the classification of high and low free-energy states. Our approach relies on utilizing (i) a novel hypergraph representation of molecules, encoding all relevant information for characterizing the potential energy of a conformation, and (ii) novel message passing and pooling layers for processing and making predictions on such hypergraph-structured data. Despite the complexity of the problem, our results show a remarkable AUC of 0.92 for transfer learning from tri-alanine to the deca-alanine system. Moreover, we show that the very same transfer learning approach can be used to group, in an unsupervised way, various secondary structures of deca-alanine in clusters having similar free-energy values. Our study represents a proof of concept that reliable transfer learning models for molecular systems can be designed paving the way to unexplored routes in prediction of structural and energetic properties of biologically relevant systems. △ Less

Submitted 19 August, 2022; v1 submitted 6 May, 2022; originally announced May 2022.

Message Passing Neural Networks for Hypergraphs

Authors: Sajjad Heydari, Lorenzo Livi

Abstract: Hypergraph representations are both more efficient and better suited to describe data characterized by relations between two or more objects. In this work, we present a new graph neural network based on message passing capable of processing hypergraph-structured data. We show that the proposed model defines a design space for neural network models for hypergraphs, thus generalizing existing models… ▽ More Hypergraph representations are both more efficient and better suited to describe data characterized by relations between two or more objects. In this work, we present a new graph neural network based on message passing capable of processing hypergraph-structured data. We show that the proposed model defines a design space for neural network models for hypergraphs, thus generalizing existing models for hypergraphs. We report experiments on a benchmark dataset for node classification, highlighting the effectiveness of the proposed model with respect to other state-of-the-art methods for graphs and hypergraphs. We also discuss the benefits of using hypergraph representations and, at the same time, highlight the limitation of using equivalent graph representations when the underlying problem has relations among more than two objects. △ Less

Submitted 7 April, 2022; v1 submitted 31 March, 2022; originally announced March 2022.

A New Remote Monitor and Control System Based on Sigfox IoT Network

Authors: Lorenzo Francesco Livi, Jacopo Catani

Abstract: We describe a new, low-cost system designed to provide multi-sensor remote condition monitoring of modern scientific laboratories, as well as to allow users to perform actions from remote locations in case of detection of specified events. The system is battery operated and does not require the presence of a Local Area Network (LAN) or WiFi (which are typically not available in case of, e.g. power… ▽ More We describe a new, low-cost system designed to provide multi-sensor remote condition monitoring of modern scientific laboratories, as well as to allow users to perform actions from remote locations in case of detection of specified events. The system is battery operated and does not require the presence of a Local Area Network (LAN) or WiFi (which are typically not available in case of, e.g. power losses), as it exploits the growing infrastructure of Internet of Things (IoT) Low Power Wide Area Networks (LPWAN). In particular our system exploits the new SigFox ultra-narrow-bandwidth (UNB) infrastructure, and provides for a bidirectional link between the instrumentation and the remote user even in case of power line outages, which are among the most critical situations that a scientific laboratory can withstand. The system can detect the occurrence of predefined events in very short times, and either autonomously react with a series of predefined actions, also allowing a remote user to timely perform additional actions on the system through an user-friendly smartphone application or via a browser interface. The system also embeds a novel power-loss detection architecture, which detects power line failures in less than 2 ms. We provide a full characterization of the prototype, including reaction times, connection latencies, sensors sensitivity, and power consumption. △ Less

Submitted 3 September, 2021; originally announced October 2021.

Comments: Accepted for publication on Review of Scientific Instruments; 8 pages, 5 figures

Learning Graph Cellular Automata

Authors: Daniele Grattarola, Lorenzo Livi, Cesare Alippi

Abstract: Cellular automata (CA) are a class of computational models that exhibit rich dynamics emerging from the local interaction of cells arranged in a regular lattice. In this work we focus on a generalised version of typical CA, called graph cellular automata (GCA), in which the lattice structure is replaced by an arbitrary graph. In particular, we extend previous work that used convolutional neural ne… ▽ More Cellular automata (CA) are a class of computational models that exhibit rich dynamics emerging from the local interaction of cells arranged in a regular lattice. In this work we focus on a generalised version of typical CA, called graph cellular automata (GCA), in which the lattice structure is replaced by an arbitrary graph. In particular, we extend previous work that used convolutional neural networks to learn the transition rule of conventional CA and we use graph neural networks to learn a variety of transition rules for GCA. First, we present a general-purpose architecture for learning GCA, and we show that it can represent any arbitrary GCA with finite and discrete state space. Then, we test our approach on three different tasks: 1) learning the transition rule of a GCA on a Voronoi tessellation; 2) imitating the behaviour of a group of flocking agents; 3) learning a rule that converges to a desired target state. △ Less

Submitted 27 October, 2021; originally announced October 2021.

Comments: 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Flavour-selective localization in interacting lattice fermions via SU(N) symmetry breaking

Authors: Daniele Tusi, Lorenzo Franchi, Lorenzo Francesco Livi, Karla Baumann, Daniel Benedicto Orenes, Lorenzo Del Re, Rafael Emilio Barfknecht, Tianwei Zhou, Massimo Inguscio, Giacomo Cappellini, Massimo Capone, Jacopo Catani, Leonardo Fallani

Abstract: A large repulsion between particles in a quantum system can lead to their localization, as it happens for the electrons in Mott insulating materials. This paradigm has recently branched out into a new quantum state, the orbital-selective Mott insulator, where electrons in some orbitals are predicted to localize, while others remain itinerant. We provide a direct experimental realization of this ph… ▽ More A large repulsion between particles in a quantum system can lead to their localization, as it happens for the electrons in Mott insulating materials. This paradigm has recently branched out into a new quantum state, the orbital-selective Mott insulator, where electrons in some orbitals are predicted to localize, while others remain itinerant. We provide a direct experimental realization of this phenomenon, that we extend to a more general flavour-selective localization. By using an atom-based quantum simulator, we engineer SU(3) Fermi-Hubbard models breaking their symmetry via a tunable coupling between flavours, observing an enhancement of localization and the emergence of flavour-dependent correlations. Our realization of flavour-selective Mott physics opens the path to the quantum simulation of multicomponent materials, from superconductors to topological insulators. △ Less

Submitted 27 April, 2021; originally announced April 2021.

Comments: Main Text: 5 pages, 4 figures. Supplementary Material: 6 pages, 6 figures

Journal ref: Nature Physics, 18, pages 1201-1205 (2022)

Learn to Synchronize, Synchronize to Learn

Authors: Pietro Verzelli, Cesare Alippi, Lorenzo Livi

Abstract: In recent years, the machine learning community has seen a continuous growing interest in research aimed at investigating dynamical aspects of both training procedures and machine learning models. Of particular interest among recurrent neural networks we have the Reservoir Computing (RC) paradigm characterized by conceptual simplicity and a fast training scheme. Yet, the guiding principles under w… ▽ More In recent years, the machine learning community has seen a continuous growing interest in research aimed at investigating dynamical aspects of both training procedures and machine learning models. Of particular interest among recurrent neural networks we have the Reservoir Computing (RC) paradigm characterized by conceptual simplicity and a fast training scheme. Yet, the guiding principles under which RC operates are only partially understood. In this work, we analyze the role played by Generalized Synchronization (GS) when training a RC to solve a generic task. In particular, we show how GS allows the reservoir to correctly encode the system generating the input signal into its dynamics. We also discuss necessary and sufficient conditions for the learning to be feasible in this approach. Moreover, we explore the role that ergodicity plays in this process, showing how its presence allows the learning outcome to apply to multiple input trajectories. Finally, we show that satisfaction of the GS can be measured by means of the Mutual False Nearest Neighbors index, which makes effective to practitioners theoretical derivations. △ Less

Submitted 11 May, 2021; v1 submitted 6 October, 2020; originally announced October 2020.

Input-to-State Representation in linear reservoirs dynamics

Authors: Pietro Verzelli, Cesare Alippi, Lorenzo Livi, Peter Tino

Abstract: Reservoir computing is a popular approach to design recurrent neural networks, due to its training simplicity and approximation performance. The recurrent part of these networks is not trained (e.g., via gradient descent), making them appealing for analytical studies by a large community of researchers with backgrounds spanning from dynamical systems to neuroscience. However, even in the simple li… ▽ More Reservoir computing is a popular approach to design recurrent neural networks, due to its training simplicity and approximation performance. The recurrent part of these networks is not trained (e.g., via gradient descent), making them appealing for analytical studies by a large community of researchers with backgrounds spanning from dynamical systems to neuroscience. However, even in the simple linear case, the working principle of these networks is not fully understood and their design is usually driven by heuristics. A novel analysis of the dynamics of such networks is proposed, which allows the investigator to express the state evolution using the controllability matrix. Such a matrix encodes salient characteristics of the network dynamics; in particular, its rank represents an input-indepedent measure of the memory capacity of the network. Using the proposed approach, it is possible to compare different reservoir architectures and explain why a cyclic topology achieves favourable results as verified by practitioners. △ Less

Submitted 12 February, 2021; v1 submitted 23 March, 2020; originally announced March 2020.

The Echo Index and multistability in input-driven recurrent neural networks

Authors: Andrea Ceni, Peter Ashwin, Lorenzo Livi, Claire Postlethwaite

Abstract: A recurrent neural network (RNN) possesses the echo state property (ESP) if, for a given input sequence, it ``forgets'' any internal states of the driven (nonautonomous) system and asymptotically follows a unique, possibly complex trajectory. The lack of ESP is conventionally understood as a lack of reliable behaviour in RNNs. Here, we show that RNNs can reliably perform computations under a more… ▽ More A recurrent neural network (RNN) possesses the echo state property (ESP) if, for a given input sequence, it ``forgets'' any internal states of the driven (nonautonomous) system and asymptotically follows a unique, possibly complex trajectory. The lack of ESP is conventionally understood as a lack of reliable behaviour in RNNs. Here, we show that RNNs can reliably perform computations under a more general principle that accounts only for their local behaviour in phase space. To this end, we formulate a generalisation of the ESP and introduce an echo index to characterise the number of simultaneously stable responses of a driven RNN. We show that it is possible for the echo index to change with inputs, highlighting a potential source of computational errors in RNNs due to characteristics of the inputs driving the dynamics. △ Less

Submitted 19 May, 2020; v1 submitted 21 January, 2020; originally announced January 2020.

Comments: Revised version, 43 pages, 6 figures

Recurrence Quantification Analysis of Dynamic Brain Networks

Authors: Marinho A. Lopes, Jiaxiang Zhang, Dominik Krzemiński, Khalid Hamandi, Qi Chen, Lorenzo Livi, Naoki Masuda

Abstract: Evidence suggests that brain network dynamics is a key determinant of brain function and dysfunction. Here we propose a new framework to assess the dynamics of brain networks based on recurrence analysis. Our framework uses recurrence plots and recurrence quantification analysis to characterize dynamic networks. For resting-state magnetoencephalographic dynamic functional networks (dFNs), we have… ▽ More Evidence suggests that brain network dynamics is a key determinant of brain function and dysfunction. Here we propose a new framework to assess the dynamics of brain networks based on recurrence analysis. Our framework uses recurrence plots and recurrence quantification analysis to characterize dynamic networks. For resting-state magnetoencephalographic dynamic functional networks (dFNs), we have found that functional networks recur more quickly in people with epilepsy than healthy controls. This suggests that recurrence of dFNs may be used as a biomarker of epilepsy. For stereo electroencephalography data, we have found that dFNs involved in epileptic seizures emerge before seizure onset, and recurrence analysis allows us to detect seizures. We further observe distinct dFNs before and after seizures, which may inform neurostimulation strategies to prevent seizures. Our framework can also be used for understanding dFNs in healthy brain function and in other neurological disorders besides epilepsy. △ Less

Submitted 16 September, 2020; v1 submitted 11 January, 2020; originally announced January 2020.

Comments: 77 pages, 11 figures; note: the acknowledgments section is the most complete in this arxiv version (compared to the published version in EJN)

Hierarchical Representation Learning in Graph Neural Networks with Node Decimation Pooling

Authors: Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, Cesare Alippi

Abstract: In graph neural networks (GNNs), pooling operators compute local summaries of input graphs to capture their global properties, and they are fundamental for building deep GNNs that learn hierarchical representations. In this work, we propose the Node Decimation Pooling (NDP), a pooling operator for GNNs that generates coarser graphs while preserving the overall graph topology. During training, the… ▽ More In graph neural networks (GNNs), pooling operators compute local summaries of input graphs to capture their global properties, and they are fundamental for building deep GNNs that learn hierarchical representations. In this work, we propose the Node Decimation Pooling (NDP), a pooling operator for GNNs that generates coarser graphs while preserving the overall graph topology. During training, the GNN learns new node representations and fits them to a pyramid of coarsened graphs, which is computed offline in a pre-processing stage. NDP consists of three steps. First, a node decimation procedure selects the nodes belonging to one side of the partition identified by a spectral algorithm that approximates the \maxcut{} solution. Afterwards, the selected nodes are connected with Kron reduction to form the coarsened graph. Finally, since the resulting graph is very dense, we apply a sparsification procedure that prunes the adjacency matrix of the coarsened graph to reduce the computational cost in the GNN. Notably, we show that it is possible to remove many edges without significantly altering the graph structure. Experimental results show that NDP is more efficient compared to state-of-the-art graph pooling operators while reaching, at the same time, competitive performance on a significant variety of graph classification tasks. △ Less

Submitted 20 April, 2024; v1 submitted 24 October, 2019; originally announced October 2019.

Graph Random Neural Features for Distance-Preserving Graph Representations

Authors: Daniele Zambon, Cesare Alippi, Lorenzo Livi

Abstract: We present Graph Random Neural Features (GRNF), a novel embedding method from graph-structured data to real vectors based on a family of graph neural networks. The embedding naturally deals with graph isomorphism and preserves the metric structure of the graph domain, in probability. In addition to being an explicit embedding method, it also allows us to efficiently and effectively approximate gra… ▽ More We present Graph Random Neural Features (GRNF), a novel embedding method from graph-structured data to real vectors based on a family of graph neural networks. The embedding naturally deals with graph isomorphism and preserves the metric structure of the graph domain, in probability. In addition to being an explicit embedding method, it also allows us to efficiently and effectively approximate graph metric distances (as well as complete kernel functions); a criterion to select the embedding dimension trading off the approximation accuracy with the computational cost is also provided. GRNF can be used within traditional processing methods or as a training-free input layer of a graph neural network. The theoretical guarantees that accompany GRNF ensure that the considered graph distance is metric, hence allowing to distinguish any pair of non-isomorphic graphs. △ Less

Submitted 2 June, 2020; v1 submitted 9 September, 2019; originally announced September 2019.

Comments: to be published in Proceedings of the 37th International Conference on Machine Learning, 2020

Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere

Authors: Pietro Verzelli, Cesare Alippi, Lorenzo Livi

Abstract: Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behaviour. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of chaos. Fi… ▽ More Among the various architectures of Recurrent Neural Networks, Echo State Networks (ESNs) emerged due to their simplified and inexpensive training procedure. These networks are known to be sensitive to the setting of hyper-parameters, which critically affect their behaviour. Results show that their performance is usually maximized in a narrow region of hyper-parameter space called edge of chaos. Finding such a region requires searching in hyper-parameter space in a sensible way: hyper-parameter configurations marginally outside such a region might yield networks exhibiting fully developed chaos, hence producing unreliable computations. The performance gain due to optimizing hyper-parameters can be studied by considering the memory--nonlinearity trade-off, i.e., the fact that increasing the nonlinear behavior of the network degrades its ability to remember past inputs, and vice-versa. In this paper, we propose a model of ESNs that eliminates critical dependence on hyper-parameters, resulting in networks that provably cannot enter a chaotic regime and, at the same time, denotes nonlinear behaviour in phase space characterised by a large memory of past inputs, comparable to the one of linear networks. Our contribution is supported by experiments corroborating our theoretical findings, showing that the proposed model displays dynamics that are rich-enough to approximate many common nonlinear systems used for benchmarking. △ Less

Submitted 6 September, 2019; v1 submitted 27 March, 2019; originally announced March 2019.

Autoregressive Models for Sequences of Graphs

Authors: Daniele Zambon, Daniele Grattarola, Lorenzo Livi, Cesare Alippi

Abstract: This paper proposes an autoregressive (AR) model for sequences of graphs, which generalises traditional AR models. A first novelty consists in formalising the AR model for a very general family of graphs, characterised by a variable topology, and attributes associated with nodes and edges. A graph neural network (GNN) is also proposed to learn the AR function associated with the graph-generating p… ▽ More This paper proposes an autoregressive (AR) model for sequences of graphs, which generalises traditional AR models. A first novelty consists in formalising the AR model for a very general family of graphs, characterised by a variable topology, and attributes associated with nodes and edges. A graph neural network (GNN) is also proposed to learn the AR function associated with the graph-generating process (GGP), and subsequently predict the next graph in a sequence. The proposed method is compared with four baselines on synthetic GGPs, denoting a significantly better performance on all considered problems. △ Less

Submitted 18 March, 2019; originally announced March 2019.

Comments: International Joint Conference on Neural Networks (IJCNN) 2019

Deep Divergence-Based Approach to Clustering

Authors: Michael Kampffmeyer, Sigurd Løkse, Filippo M. Bianchi, Lorenzo Livi, Arnt-Børre Salberg, Robert Jenssen

Abstract: A promising direction in deep learning research consists in learning representations and simultaneously discovering cluster structure in unlabeled data by optimizing a discriminative loss function. As opposed to supervised deep learning, this line of research is in its infancy, and how to design and optimize suitable loss functions to train deep neural networks for clustering is still an open ques… ▽ More A promising direction in deep learning research consists in learning representations and simultaneously discovering cluster structure in unlabeled data by optimizing a discriminative loss function. As opposed to supervised deep learning, this line of research is in its infancy, and how to design and optimize suitable loss functions to train deep neural networks for clustering is still an open question. Our contribution to this emerging field is a new deep clustering network that leverages the discriminative power of information-theoretic divergence measures, which have been shown to be effective in traditional clustering. We propose a novel loss function that incorporates geometric regularization constraints, thus avoiding degenerate structures of the resulting clustering partition. Experiments on synthetic benchmarks and real datasets show that the proposed network achieves competitive performance with respect to other state-of-the-art methods, scales well to large datasets, and does not require pre-training steps. △ Less

Submitted 13 February, 2019; originally announced February 2019.

Graph Neural Networks with convolutional ARMA filters

Authors: Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, Cesare Alippi

Abstract: Popular graph neural networks implement convolution operations on graphs based on polynomial spectral filters. In this paper, we propose a novel graph convolutional layer inspired by the auto-regressive moving average (ARMA) filter that, compared to polynomial ones, provides a more flexible frequency response, is more robust to noise, and better captures the global graph structure. We propose a gr… ▽ More Popular graph neural networks implement convolution operations on graphs based on polynomial spectral filters. In this paper, we propose a novel graph convolutional layer inspired by the auto-regressive moving average (ARMA) filter that, compared to polynomial ones, provides a more flexible frequency response, is more robust to noise, and better captures the global graph structure. We propose a graph neural network implementation of the ARMA filter with a recursive and distributed formulation, obtaining a convolutional layer that is efficient to train, localized in the node space, and can be transferred to new graphs at test time. We perform a spectral analysis to study the filtering effect of the proposed ARMA layer and report experiments on four downstream tasks: semi-supervised node classification, graph signal classification, graph classification, and graph regression. Results show that the proposed ARMA layer brings significant improvements over graph neural networks based on polynomial filters. △ Less

Submitted 24 January, 2021; v1 submitted 4 January, 2019; originally announced January 2019.

Adversarial Autoencoders with Constant-Curvature Latent Manifolds

Authors: Daniele Grattarola, Lorenzo Livi, Cesare Alippi

Abstract: Constant-curvature Riemannian manifolds (CCMs) have been shown to be ideal embedding spaces in many application domains, as their non-Euclidean geometry can naturally account for some relevant properties of data, like hierarchy and circularity. In this work, we introduce the CCM adversarial autoencoder (CCM-AAE), a probabilistic generative model trained to represent a data distribution on a CCM. O… ▽ More Constant-curvature Riemannian manifolds (CCMs) have been shown to be ideal embedding spaces in many application domains, as their non-Euclidean geometry can naturally account for some relevant properties of data, like hierarchy and circularity. In this work, we introduce the CCM adversarial autoencoder (CCM-AAE), a probabilistic generative model trained to represent a data distribution on a CCM. Our method works by matching the aggregated posterior of the CCM-AAE with a probability distribution defined on a CCM, so that the encoder implicitly learns to represent data on the CCM to fool the discriminator network. The geometric constraint is also explicitly imposed by jointly training the CCM-AAE to maximise the membership degree of the embeddings to the CCM. While a few works in recent literature make use of either hyperspherical or hyperbolic manifolds for different learning tasks, ours is the first unified framework to seamlessly deal with CCMs of different curvatures. We show the effectiveness of our model on three different datasets characterised by non-trivial geometry: semi-supervised classification on MNIST, link prediction on two popular citation datasets, and graph-based molecule generation using the QM9 chemical database. Results show that our method improves upon other autoencoders based on Euclidean and non-Euclidean geometries on all tasks taken into account. △ Less

Submitted 11 April, 2019; v1 submitted 11 December, 2018; originally announced December 2018.

Comments: Submitted to Applied Soft Computing

Coherent Manipulation of Orbital Feshbach Molecules of Two-Electron Atoms

Authors: G. Cappellini, L. F. Livi, L. Franchi, D. Tusi, D. Benedicto Orenes, M. Inguscio, J. Catani, L. Fallani

Abstract: Ultracold molecules have experienced increasing attention in recent years. Compared to ultracold atoms, they possess several unique properties that make them perfect candidates for the implementation of new quantum-technological applications in several fields, from quantum simulation to quantum sensing and metrology. In particular, ultracold molecules of two-electron atoms (such as strontium or yt… ▽ More Ultracold molecules have experienced increasing attention in recent years. Compared to ultracold atoms, they possess several unique properties that make them perfect candidates for the implementation of new quantum-technological applications in several fields, from quantum simulation to quantum sensing and metrology. In particular, ultracold molecules of two-electron atoms (such as strontium or ytterbium) also inherit the peculiar properties of these atomic species, above all the possibility to access metastable electronic states via direct excitation on optical clock transitions with ultimate sensitivity and accuracy. In this paper we report on the production and coherent manipulation of molecular bound states of two fermionic $^{173}$Yb atoms in different electronic (orbital) states $^1$S$_0$ and $^3$P$_0$ in proximity of a scattering resonance involving atoms in different spin and electronic states, called orbital Feshbach resonance. We demonstrate that orbital molecules can be coherently photoassociated starting from a gas of ground-state atoms in a three-dimensional optical lattices by observing several photoassociation and photodissociation cycles. We also show the possibility to coherently control the molecular internal state by using Raman-assisted transfer to swap the nuclear spin of one of the atoms forming the molecule, thus demonstrating a powerful manipulation and detection tool of these molecular bound states. Finally, by exploiting this peculiar detection technique we provide first information on the lifetime of the molecular states in a many-body setting, paving the way towards future investigations of strongly interacting Fermi gases in a still unexplored regime. △ Less

Submitted 23 October, 2018; originally announced October 2018.

Comments: 11 pages, 8 figures

Journal ref: Phys. Rev. X 9, 011028 (2019)

A characterization of the Edge of Criticality in Binary Echo State Networks

Authors: Pietro Verzelli, Lorenzo Livi, Cesare Alippi

Abstract: Echo State Networks (ESNs) are simplified recurrent neural network models composed of a reservoir and a linear, trainable readout layer. The reservoir is tunable by some hyper-parameters that control the network behaviour. ESNs are known to be effective in solving tasks when configured on a region in (hyper-)parameter space called \emph{Edge of Criticality} (EoC), where the system is maximally sen… ▽ More Echo State Networks (ESNs) are simplified recurrent neural network models composed of a reservoir and a linear, trainable readout layer. The reservoir is tunable by some hyper-parameters that control the network behaviour. ESNs are known to be effective in solving tasks when configured on a region in (hyper-)parameter space called \emph{Edge of Criticality} (EoC), where the system is maximally sensitive to perturbations hence affecting its behaviour. In this paper, we propose binary ESNs, which are architecturally equivalent to standard ESNs but consider binary activation functions and binary recurrent weights. For these networks, we derive a closed-form expression for the EoC in the autonomous case and perform simulations in order to assess their behavior in the case of noisy neurons and in the presence of a signal. We propose a theoretical explanation for the fact that the variance of the input plays a major role in characterizing the EoC. △ Less

Submitted 3 October, 2018; originally announced October 2018.

Interpreting recurrent neural networks behaviour via excitable network attractors

Authors: Andrea Ceni, Peter Ashwin, Lorenzo Livi

Abstract: Introduction: Machine learning provides fundamental tools both for scientific research and for the development of technologies with significant impact on society. It provides methods that facilitate the discovery of regularities in data and that give predictions without explicit knowledge of the rules governing a system. However, a price is paid for exploiting such flexibility: machine learning me… ▽ More Introduction: Machine learning provides fundamental tools both for scientific research and for the development of technologies with significant impact on society. It provides methods that facilitate the discovery of regularities in data and that give predictions without explicit knowledge of the rules governing a system. However, a price is paid for exploiting such flexibility: machine learning methods are typically black-boxes where it is difficult to fully understand what the machine is doing or how it is operating. This poses constraints on the applicability and explainability of such methods. Methods: Our research aims to open the black-box of recurrent neural networks, an important family of neural networks used for processing sequential data. We propose a novel methodology that provides a mechanistic interpretation of behaviour when solving a computational task. Our methodology uses mathematical constructs called excitable network attractors, which are invariant sets in phase space composed of stable attractors and excitable connections between them. Results and Discussion: As the behaviour of recurrent neural networks depends both on training and on inputs to the system, we introduce an algorithm to extract network attractors directly from the trajectory of a neural network while solving tasks. Simulations conducted on a controlled benchmark task confirm the relevance of these attractors for interpreting the behaviour of recurrent neural networks, at least for tasks that involve learning a finite number of stable states and transitions between them. △ Less

Submitted 10 March, 2019; v1 submitted 27 July, 2018; originally announced July 2018.

Comments: revised version

The Deep Kernelized Autoencoder

Authors: Michael Kampffmeyer, Sigurd Løkse, Filippo M. Bianchi, Robert Jenssen, Lorenzo Livi

Abstract: Autoencoders learn data representations (codes) in such a way that the input is reproduced at the output of the network. However, it is not always clear what kind of properties of the input data need to be captured by the codes. Kernel machines have experienced great success by operating via inner-products in a theoretically well-defined reproducing kernel Hilbert space, hence capturing topologica… ▽ More Autoencoders learn data representations (codes) in such a way that the input is reproduced at the output of the network. However, it is not always clear what kind of properties of the input data need to be captured by the codes. Kernel machines have experienced great success by operating via inner-products in a theoretically well-defined reproducing kernel Hilbert space, hence capturing topological properties of input data. In this paper, we enhance the autoencoder's ability to learn effective data representations by aligning inner products between codes with respect to a kernel matrix. By doing so, the proposed kernelized autoencoder allows learning similarity-preserving embeddings of input data, where the notion of similarity is explicitly controlled by the user and encoded in a positive semi-definite kernel matrix. Experiments are performed for evaluating both reconstruction and kernel alignment performance in classification tasks and visualization of high-dimensional data. Additionally, we show that our method is capable to emulate kernel principal component analysis on a denoising task, obtaining competitive results at a much lower computational cost. △ Less

Submitted 23 July, 2018; v1 submitted 19 July, 2018; originally announced July 2018.

Comments: This work extends the preliminary (conference) version of this paper (arXiv:1702.02526), Applied Soft Computing, Elsevier, 2018

Change Point Methods on a Sequence of Graphs

Authors: Daniele Zambon, Cesare Alippi, Lorenzo Livi

Abstract: Given a finite sequence of graphs, e.g., coming from technological, biological, and social networks, the paper proposes a methodology to identify possible changes in stationarity in the stochastic process generating the graphs. In order to cover a large class of applications, we consider the general family of attributed graphs where both topology (number of vertexes and edge configuration) and rel… ▽ More Given a finite sequence of graphs, e.g., coming from technological, biological, and social networks, the paper proposes a methodology to identify possible changes in stationarity in the stochastic process generating the graphs. In order to cover a large class of applications, we consider the general family of attributed graphs where both topology (number of vertexes and edge configuration) and related attributes are allowed to change also in the stationary case. Novel Change Point Methods (CPMs) are proposed, that (i) map graphs into a vector domain; (ii) apply a suitable statistical test in the vector space; (iii) detect the change --if any-- according to a confidence level and provide an estimate for its time occurrence. Two specific multivariate CPMs have been designed: one that detects shifts in the distribution mean, the other addressing generic changes affecting the distribution. We ground our proposal with theoretical results showing how to relate the inference attained in the numerical vector space to the graph domain, and vice versa. We also show how to extend the methodology for handling multiple change points in the same sequence. Finally, the proposed CPMs have been validated on real data sets coming from epileptic-seizure detection problems and on labeled data sets for graph classification. Results show the effectiveness of what proposed in relevant application scenarios. △ Less

Submitted 5 October, 2018; v1 submitted 18 May, 2018; originally announced May 2018.

Journal ref: IEEE Transactions on Signal Processing ( Volume: 67, Issue: 24, Dec.15, 15 2019)

Change Detection in Graph Streams by Learning Graph Embeddings on Constant-Curvature Manifolds

Authors: Daniele Grattarola, Daniele Zambon, Cesare Alippi, Lorenzo Livi

Abstract: The space of graphs is often characterised by a non-trivial geometry, which complicates learning and inference in practical applications. A common approach is to use embedding techniques to represent graphs as points in a conventional Euclidean space, but non-Euclidean spaces have often been shown to be better suited for embedding graphs. Among these, constant-curvature Riemannian manifolds (CCMs)… ▽ More The space of graphs is often characterised by a non-trivial geometry, which complicates learning and inference in practical applications. A common approach is to use embedding techniques to represent graphs as points in a conventional Euclidean space, but non-Euclidean spaces have often been shown to be better suited for embedding graphs. Among these, constant-curvature Riemannian manifolds (CCMs) offer embedding spaces suitable for studying the statistical properties of a graph distribution, as they provide ways to easily compute metric geodesic distances. In this paper, we focus on the problem of detecting changes in stationarity in a stream of attributed graphs. To this end, we introduce a novel change detection framework based on neural networks and CCMs, that takes into account the non-Euclidean nature of graphs. Our contribution in this work is twofold. First, via a novel approach based on adversarial learning, we compute graph embeddings by training an autoencoder to represent graphs on CCMs. Second, we introduce two novel change detection tests operating on CCMs. We perform experiments on synthetic data, as well as two real-world application scenarios: the detection of epileptic seizures using functional connectivity brain networks, and the detection of hostility between two subjects, using human skeletal graphs. Results show that the proposed methods are able to detect even small changes in a graph-generating process, consistently outperforming approaches based on Euclidean embeddings. △ Less

Submitted 11 April, 2019; v1 submitted 16 May, 2018; originally announced May 2018.

Comments: 14 pages, 8 figures

Journal ref: IEEE Transactions on Neural Networks and Learning Systems (Volume: 31, Issue: 6, June 2020)

Learning representations for multivariate time series with missing data using Temporal Kernelized Autoencoders

Authors: Filippo Maria Bianchi, Lorenzo Livi, Karl Øyvind Mikalsen, Michael Kampffmeyer, Robert Jenssen

Abstract: Learning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical dimensionality reduction approaches are designed for vectorial data and cannot deal explicitly with missing values. In this work, we propose a novel autoencoder architecture based on… ▽ More Learning compressed representations of multivariate time series (MTS) facilitates data analysis in the presence of noise and redundant information, and for a large number of variates and time steps. However, classical dimensionality reduction approaches are designed for vectorial data and cannot deal explicitly with missing values. In this work, we propose a novel autoencoder architecture based on recurrent neural networks to generate compressed representations of MTS. The proposed model can process inputs characterized by variable lengths and it is specifically designed to handle missing data. Our autoencoder learns fixed-length vectorial representations, whose pairwise similarities are aligned to a kernel function that operates in input space and that handles missing values. This allows to learn good representations, even in the presence of a significant amount of missing data. To show the effectiveness of the proposed approach, we evaluate the quality of the learned representations in several classification tasks, including those involving medical data, and we compare to other methods for dimensionality reduction. Successively, we design two frameworks based on the proposed architecture: one for imputing missing data and another for one-class classification. Finally, we analyze under what circumstances an autoencoder with recurrent layers can learn better compressed representations of MTS than feed-forward architectures. △ Less

Submitted 16 July, 2019; v1 submitted 9 May, 2018; originally announced May 2018.

Anomaly and Change Detection in Graph Streams through Constant-Curvature Manifold Embeddings

Authors: Daniele Zambon, Lorenzo Livi, Cesare Alippi

Abstract: Map** complex input data into suitable lower dimensional manifolds is a common procedure in machine learning. This step is beneficial mainly for two reasons: (1) it reduces the data dimensionality and (2) it provides a new data representation possibly characterised by convenient geometric properties. Euclidean spaces are by far the most widely used embedding spaces, thanks to their well-understo… ▽ More Map** complex input data into suitable lower dimensional manifolds is a common procedure in machine learning. This step is beneficial mainly for two reasons: (1) it reduces the data dimensionality and (2) it provides a new data representation possibly characterised by convenient geometric properties. Euclidean spaces are by far the most widely used embedding spaces, thanks to their well-understood structure and large availability of consolidated inference methods. However, recent research demonstrated that many types of complex data (e.g., those represented as graphs) are actually better described by non-Euclidean geometries. Here, we investigate how embedding graphs on constant-curvature manifolds (hyper-spherical and hyperbolic manifolds) impacts on the ability to detect changes in sequences of attributed graphs. The proposed methodology consists in embedding graphs into a geometric space and perform change detection there by means of conventional methods for numerical streams. The curvature of the space is a parameter that we learn to reproduce the geometry of the original application-dependent graph space. Preliminary experimental results show the potential capability of representing graphs by means of curved manifold, in particular for change and anomaly detection problems. △ Less

Submitted 3 May, 2018; originally announced May 2018.

Comments: To be published in IEEE IJCNN 2018

Journal ref: 2018 International Joint Conference on Neural Networks (IJCNN)

Time series kernel similarities for predicting Paroxysmal Atrial Fibrillation from ECGs

Authors: Filippo Maria Bianchi, Lorenzo Livi, Alberto Ferrante, Jelena Milosevic, Miroslaw Malek

Abstract: We tackle the problem of classifying Electrocardiography (ECG) signals with the aim of predicting the onset of Paroxysmal Atrial Fibrillation (PAF). Atrial fibrillation is the most common type of arrhythmia, but in many cases PAF episodes are asymptomatic. Therefore, in order to help diagnosing PAF, it is important to design procedures for detecting and, more importantly, predicting PAF episodes.… ▽ More We tackle the problem of classifying Electrocardiography (ECG) signals with the aim of predicting the onset of Paroxysmal Atrial Fibrillation (PAF). Atrial fibrillation is the most common type of arrhythmia, but in many cases PAF episodes are asymptomatic. Therefore, in order to help diagnosing PAF, it is important to design procedures for detecting and, more importantly, predicting PAF episodes. We propose a method for predicting PAF events whose first step consists of a feature extraction procedure that represents each ECG as a multi-variate time series. Successively, we design a classification framework based on kernel similarities for multi-variate time series, capable of handling missing data. We consider different approaches to perform classification in the original space of the multi-variate time series and in an embedding space, defined by the kernel similarity measure. We achieve a classification accuracy comparable with state of the art methods, with the additional advantage of detecting the PAF onset up to 15 minutes in advance. △ Less

Submitted 4 April, 2018; v1 submitted 21 January, 2018; originally announced January 2018.

State-dependent interactions in ultracold $^{174}$Yb probed by optical clock spectroscopy

Authors: L. Franchi, L. F. Livi, G. Cappellini, G. Binella, M. Inguscio, J. Catani, L. Fallani

Abstract: We report on the measurement of the scattering properties of ultracold $^{174}$Yb bosons in a three-dimensional (3D) optical lattice. Site occupancy in an atomic Mott insulator is resolved with high-precision spectroscopy on an ultranarrow optical clock transition. Scattering lengths and loss rate coefficients for $^{174}$Yb atoms in different collisional channels involving the ground state $^1$S… ▽ More We report on the measurement of the scattering properties of ultracold $^{174}$Yb bosons in a three-dimensional (3D) optical lattice. Site occupancy in an atomic Mott insulator is resolved with high-precision spectroscopy on an ultranarrow optical clock transition. Scattering lengths and loss rate coefficients for $^{174}$Yb atoms in different collisional channels involving the ground state $^1$S$_0$ and the metastable $^3$P$_0$ are derived. These studies set important constraints for future experimental studies of two-electron atoms for quantum-technological applications. △ Less

Submitted 13 July, 2017; originally announced July 2017.

Comments: 14 pages, 6 figures

Concept Drift and Anomaly Detection in Graph Streams

Authors: Daniele Zambon, Cesare Alippi, Lorenzo Livi

Abstract: Graph representations offer powerful and intuitive ways to describe data in a multitude of application domains. Here, we consider stochastic processes generating graphs and propose a methodology for detecting changes in stationarity of such processes. The methodology is general and considers a process generating attributed graphs with a variable number of vertices/edges, without the need to assume… ▽ More Graph representations offer powerful and intuitive ways to describe data in a multitude of application domains. Here, we consider stochastic processes generating graphs and propose a methodology for detecting changes in stationarity of such processes. The methodology is general and considers a process generating attributed graphs with a variable number of vertices/edges, without the need to assume one-to-one correspondence between vertices at different time steps. The methodology acts by embedding every graph of the stream into a vector domain, where a conventional multivariate change detection procedure can be easily applied. We ground the soundness of our proposal by proving several theoretical results. In addition, we provide a specific implementation of the methodology and evaluate its effectiveness on several detection problems involving attributed graphs representing biological molecules and drawings. Experimental results are contrasted with respect to suitable baseline methods, demonstrating the effectiveness of our approach. △ Less

Submitted 9 February, 2018; v1 submitted 21 June, 2017; originally announced June 2017.

Journal ref: IEEE Transactions on Neural Networks and Learning Systems (Volume: 29, Issue: 11, Nov. 2018)

Right-side-stretched multifractal spectra indicate small-worldness in networks

Authors: Paweł Oświȩcimka, Lorenzo Livi, Stanisław Drożdż

Abstract: Complex network formalism allows to explain the behavior of systems composed by interacting units. Several prototypical network models have been proposed thus far. The small-world model has been introduced to mimic two important features observed in real-world systems: i) local clustering and ii) the possibility to move across a network by means of long-range links that significantly reduce the ch… ▽ More Complex network formalism allows to explain the behavior of systems composed by interacting units. Several prototypical network models have been proposed thus far. The small-world model has been introduced to mimic two important features observed in real-world systems: i) local clustering and ii) the possibility to move across a network by means of long-range links that significantly reduce the characteristic path length. A natural question would be whether there exist several "types" of small-world architectures, giving rise to a continuum of models with properties (partially) shared with other models belonging to different network families. Here, we take advantage of the interplay between network theory and time series analysis and propose to investigate small-world signatures in complex networks by analyzing multifractal characteristics of time series generated from such networks. In particular, we suggest that the degree of right-sided asymmetry of multifractal spectra is linked with the degree of small-worldness present in networks. This claim is supported by numerical simulations performed on several parametric models, including prototypical small-world networks, scale-free, fractal and also real-world networks describing protein molecules. Our results also indicate that right-sided asymmetry emerges with the presence of the following topological properties: low edge density, low average shortest path, and high clustering coefficient. △ Less

Submitted 4 October, 2017; v1 submitted 22 May, 2017; originally announced May 2017.

Comments: Final version: 37 pages, 47 references, 11 figures. Title changed before publication. Original title "Right-sided multifractal spectra indicate small-worldness in networks"

Journal ref: Commun Nonlinear Sci Numer Simulat 57 (2018) 231-245

Deep Kernelized Autoencoders

Authors: Michael Kampffmeyer, Sigurd Løkse, Filippo Maria Bianchi, Robert Jenssen, Lorenzo Livi

Abstract: In this paper we introduce the deep kernelized autoencoder, a neural network model that allows an explicit approximation of (i) the map** from an input space to an arbitrary, user-specified kernel space and (ii) the back-projection from such a kernel space to input space. The proposed method is based on traditional autoencoders and is trained through a new unsupervised loss function. During trai… ▽ More In this paper we introduce the deep kernelized autoencoder, a neural network model that allows an explicit approximation of (i) the map** from an input space to an arbitrary, user-specified kernel space and (ii) the back-projection from such a kernel space to input space. The proposed method is based on traditional autoencoders and is trained through a new unsupervised loss function. During training, we optimize both the reconstruction accuracy of input samples and the alignment between a kernel matrix given as prior and the inner products of the hidden representations computed by the autoencoder. Kernel alignment provides control over the hidden representation learned by the autoencoder. Experiments have been performed to evaluate both reconstruction and kernel alignment performance. Additionally, we applied our method to emulate kPCA on a denoising task obtaining promising results. △ Less

Submitted 8 February, 2017; originally announced February 2017.

Synthetic dimensions and spin-orbit coupling with an optical clock transition

Authors: L. F. Livi, G. Cappellini, M. Diem, L. Franchi, C. Clivati, M. Frittelli, F. Levi, D. Calonico, J. Catani, M. Inguscio, L. Fallani

Abstract: We demonstrate a novel way of synthesizing spin-orbit interactions in ultracold quantum gases, based on a single-photon optical clock transition coupling two long-lived electronic states of two-electron $^{173}$Yb atoms. By map** the electronic states onto effective sites along a synthetic "electronic" dimension, we have engineered synthetic fermionic ladders with tunable magnetic fluxes. We hav… ▽ More We demonstrate a novel way of synthesizing spin-orbit interactions in ultracold quantum gases, based on a single-photon optical clock transition coupling two long-lived electronic states of two-electron $^{173}$Yb atoms. By map** the electronic states onto effective sites along a synthetic "electronic" dimension, we have engineered synthetic fermionic ladders with tunable magnetic fluxes. We have detected the spin-orbit coupling with fiber-link-enhanced clock spectroscopy and directly measured the emergence of chiral edge currents, probing them as a function of the magnetic field flux. These results open new directions for the investigation of topological states of matter with ultracold atomic gases. △ Less

Submitted 18 September, 2016; v1 submitted 15 September, 2016; originally announced September 2016.

Comments: Minor changes with respect to v1 (we have corrected some typos, fixed the use of some mathematical symbols, added one reference)

Journal ref: Phys. Rev. Lett. 117, 220401 (2016)

Multiplex visibility graphs to investigate recurrent neural networks dynamics

Authors: Filippo Maria Bianchi, Lorenzo Livi, Cesare Alippi, Robert Jenssen

Abstract: A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning of such hyperparameters may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize the internal RNN dynamics. Through this insight, we are able to design a prin… ▽ More A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning of such hyperparameters may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize the internal RNN dynamics. Through this insight, we are able to design a principled unsupervised method to derive configurations with maximized performances, in terms of prediction error and memory capacity. In particular, we propose to model time series of neurons activations with the recently introduced horizontal visibility graphs, whose topological properties reflect important dynamical features of the underlying dynamic system. Successively, each graph becomes a layer of a larger structure, called multiplex. We show that topological properties of such a multiplex reflect important features of RNN dynamics and are used to guide the tuning procedure. To validate the proposed method, we consider a class of RNNs called echo state networks. We perform experiments and discuss results on several benchmarks and real-world dataset of call data records. △ Less

Submitted 20 January, 2017; v1 submitted 10 September, 2016; originally announced September 2016.

Multifractal cross-correlation effects in two-variable time series of complex network vertex observables

Authors: Paweł Oświȩcimka, Lorenzo Livi, Stanisław Drożdż

Abstract: We investigate the scaling of the cross-correlations calculated for two-variable time series containing vertex properties in the context of complex networks. Time series of such observables are obtained by means of stationary, unbiased random walks. We consider three vertex properties that provide, respectively, short, medium, and long-range information regarding the topological role of vertices i… ▽ More We investigate the scaling of the cross-correlations calculated for two-variable time series containing vertex properties in the context of complex networks. Time series of such observables are obtained by means of stationary, unbiased random walks. We consider three vertex properties that provide, respectively, short, medium, and long-range information regarding the topological role of vertices in a given network. In order to reveal the relation between these quantities, we applied the multifractal cross-correlation analysis technique, which provides information about the nonlinear effects in coupling of time series. We show that the considered network models are characterized by unique multifractal properties of the cross-correlation. In particular, it is possible to distinguish between Erdös-Rényi, Barabási-Albert, and Watts-Strogatz networks on the basis of fractal cross-correlation. Moreover, the analysis of protein contact networks reveals characteristics shared with both scale-free and small-world models. △ Less

Submitted 21 September, 2016; v1 submitted 12 April, 2016; originally announced April 2016.

Comments: Revised version. Previous title "Exploring complex networks by means of two-variable time series of vertex observables"

Journal ref: Phys. Rev. E 94, 042307 (2016)

One-class classifiers based on entropic spanning graphs

Authors: Lorenzo Livi, Cesare Alippi

Abstract: One-class classifiers offer valuable tools to assess the presence of outliers in data. In this paper, we propose a design methodology for one-class classifiers based on entropic spanning graphs. Our approach takes into account the possibility to process also non-numeric data by means of an embedding procedure. The spanning graph is learned on the embedded input data and the outcoming partition of… ▽ More One-class classifiers offer valuable tools to assess the presence of outliers in data. In this paper, we propose a design methodology for one-class classifiers based on entropic spanning graphs. Our approach takes into account the possibility to process also non-numeric data by means of an embedding procedure. The spanning graph is learned on the embedded input data and the outcoming partition of vertices defines the classifier. The final partition is derived by exploiting a criterion based on mutual information minimization. Here, we compute the mutual information by using a convenient formulation provided in terms of the $α$-Jensen difference. Once training is completed, in order to associate a confidence level with the classifier decision, a graph-based fuzzy model is constructed. The fuzzification process is based only on topological information of the vertices of the entropic spanning graph. As such, the proposed one-class classifier is suitable also for data characterized by complex geometric structures. We provide experiments on well-known benchmarks containing both feature vectors and labeled graphs. In addition, we apply the method to the protein solubility recognition problem by considering several representations for the input samples. Experimental results demonstrate the effectiveness and versatility of the proposed method with respect to other state-of-the-art approaches. △ Less

Submitted 12 August, 2016; v1 submitted 8 April, 2016; originally announced April 2016.

Comments: Extended and revised version of the paper "One-Class Classification Through Mutual Information Minimization" presented at the 2016 IEEE IJCNN, Vancouver, Canada

Determination of the edge of criticality in echo state networks through Fisher information maximization

Authors: Lorenzo Livi, Filippo Maria Bianchi, Cesare Alippi

Abstract: It is a widely accepted fact that the computational capability of recurrent neural networks is maximized on the so-called "edge of criticality". Once the network operates in this configuration, it performs efficiently on a specific application both in terms of (i) low prediction error and (ii) high short-term memory capacity. Since the behavior of recurrent networks is strongly influenced by the p… ▽ More It is a widely accepted fact that the computational capability of recurrent neural networks is maximized on the so-called "edge of criticality". Once the network operates in this configuration, it performs efficiently on a specific application both in terms of (i) low prediction error and (ii) high short-term memory capacity. Since the behavior of recurrent networks is strongly influenced by the particular input signal driving the dynamics, a universal, application-independent method for determining the edge of criticality is still missing. In this paper, we aim at addressing this issue by proposing a theoretically motivated, unsupervised method based on Fisher information for determining the edge of criticality in recurrent neural networks. It is proven that Fisher information is maximized for (finite-size) systems operating in such critical regions. However, Fisher information is notoriously difficult to compute and either requires the probability density function or the conditional dependence of the system states with respect to the model parameters. The paper takes advantage of a recently-developed non-parametric estimator of the Fisher information matrix and provides a method to determine the critical region of echo state networks, a particular class of recurrent networks. The considered control parameters, which indirectly affect the echo state network performance, are explored to identify those configurations lying on the edge of criticality and, as such, maximizing Fisher information and computational performance. Experimental results on benchmarks and real-world data demonstrate the effectiveness of the proposed method. △ Less

Submitted 2 September, 2016; v1 submitted 11 March, 2016; originally announced March 2016.

Investigating echo state networks dynamics by means of recurrence analysis

Authors: Filippo Maria Bianchi, Lorenzo Livi, Cesare Alippi

Abstract: In this paper, we elaborate over the well-known interpretability issue in echo state networks. The idea is to investigate the dynamics of reservoir neurons with time-series analysis techniques taken from research on complex systems. Notably, we analyze time-series of neuron activations with Recurrence Plots (RPs) and Recurrence Quantification Analysis (RQA), which permit to visualize and character… ▽ More In this paper, we elaborate over the well-known interpretability issue in echo state networks. The idea is to investigate the dynamics of reservoir neurons with time-series analysis techniques taken from research on complex systems. Notably, we analyze time-series of neuron activations with Recurrence Plots (RPs) and Recurrence Quantification Analysis (RQA), which permit to visualize and characterize high-dimensional dynamical systems. We show that this approach is useful in a number of ways. First, the two-dimensional representation offered by RPs provides a way for visualizing the high-dimensional dynamics of a reservoir. Our results suggest that, if the network is stable, reservoir and input denote similar line patterns in the respective RPs. Conversely, the more unstable the ESN, the more the RP of the reservoir presents instability patterns. As a second result, we show that the $\mathrm{L_{max}}$ measure is highly correlated with the well-established maximal local Lyapunov exponent. This suggests that complexity measures based on RP diagonal lines distribution provide a valuable tool to quantify the degree of network stability. Finally, our analysis shows that all RQA measures fluctuate on the proximity of the so-called edge of stability, where an ESN typically achieves maximum computational capability. We verify that the determination of the edge of stability provided by such RQA measures is more accurate than two well-known criteria based on the Jacobian matrix of the reservoir. Therefore, we claim that RPs and RQA-based analyses can be used as valuable tools to design an effective network given a specific problem. △ Less

Submitted 24 April, 2016; v1 submitted 26 January, 2016; originally announced January 2016.

Comments: Revised version. 24 pages; 12 figures

Measuring absolute frequencies beyond the GPS limit via long-haul optical frequency dissemination

Authors: C. Clivati, G. Cappellini, L. Livi, F. Poggiali, M. Siciliani de Cumis, M. Mancini, G. Pagano, M. Frittelli, A. Mura, G. A. Costanzo, F. Levi, D. Calonico, L. Fallani, J. Catani, M. Inguscio

Abstract: Global Positioning System (GPS) dissemination of frequency standards is ubiquitous at present, providing the most widespread time and frequency reference for the majority of industrial and research applications worldwide. On the other hand, the ultimate limits of the GPS presently curb further advances in high-precision, scientific and industrial applications relying on this dissemination scheme.… ▽ More Global Positioning System (GPS) dissemination of frequency standards is ubiquitous at present, providing the most widespread time and frequency reference for the majority of industrial and research applications worldwide. On the other hand, the ultimate limits of the GPS presently curb further advances in high-precision, scientific and industrial applications relying on this dissemination scheme. Here, we demonstrate that these limits can be reliably overcome even in laboratories without a local atomic clock by replacing the GPS with a 642-km-long optical fiber link to a remote primary caesium frequency standard. Through this configuration we stably address the $^1$S$_0$---$^3$P$_0$ clock transition in an ultracold gas of $^{173}$Yb, with a precision that exceeds the possibilities of a GPS-based measurement, dismissing the need for a local clock infrastructure to perform high-precision tasks beyond GPS limit. We also report an improvement of two orders of magnitude in the accuracy on the transition frequency reported in literature. △ Less

Submitted 26 November, 2015; originally announced November 2015.

Data-driven detrending of nonstationary fractal time series with echo state networks

Authors: Enrico Maiorino, Filippo Maria Bianchi, Lorenzo Livi, Antonello Rizzi, Alireza Sadeghian

Abstract: In this paper, we propose a novel data-driven approach for removing trends (detrending) from nonstationary, fractal and multifractal time series. We consider real-valued time series relative to measurements of an underlying dynamical system that evolves through time. We assume that such a dynamical process is predictable to a certain degree by means of a class of recurrent networks called Echo Sta… ▽ More In this paper, we propose a novel data-driven approach for removing trends (detrending) from nonstationary, fractal and multifractal time series. We consider real-valued time series relative to measurements of an underlying dynamical system that evolves through time. We assume that such a dynamical process is predictable to a certain degree by means of a class of recurrent networks called Echo State Network (ESN), which are capable to model a generic dynamical process. In order to isolate the superimposed (multi)fractal component of interest, we define a data-driven filter by leveraging on the ESN prediction capability to identify the trend component of a given input time series. Specifically, the (estimated) trend is removed from the original time series and the residual signal is analyzed with the multifractal detrended fluctuation analysis procedure to verify the correctness of the detrending procedure. In order to demonstrate the effectiveness of the proposed technique, we consider several synthetic time series consisting of different types of trends and fractal noise components with known characteristics. We also process a real-world dataset, the sunspot time series, which is well-known for its multifractal features and has recently gained attention in the complex systems field. Results demonstrate the validity and generality of the proposed detrending method based on ESNs. △ Less

Submitted 3 October, 2016; v1 submitted 24 October, 2015; originally announced October 2015.

Comments: Revised version

A strongly interacting gas of two-electron fermions at an orbital Feshbach resonance

Authors: G. Pagano, M. Mancini, G. Cappellini, L. Livi, C. Sias, J. Catani, M. Inguscio, L. Fallani

Abstract: We report on the experimental observation of a strongly interacting gas of ultracold two-electron fermions with orbital degree of freedom and magnetically tunable interactions. This realization has been enabled by the demonstration of a novel kind of Feshbach resonance occurring in the scattering of two 173Yb atoms in different nuclear and electronic states. The strongly interacting regime at reso… ▽ More We report on the experimental observation of a strongly interacting gas of ultracold two-electron fermions with orbital degree of freedom and magnetically tunable interactions. This realization has been enabled by the demonstration of a novel kind of Feshbach resonance occurring in the scattering of two 173Yb atoms in different nuclear and electronic states. The strongly interacting regime at resonance is evidenced by the observation of anisotropic hydrodynamic expansion of the two-orbital Fermi gas. These results pave the way towards the realization of new quantum states of matter with strongly correlated fermions with orbital degree of freedom. △ Less

Submitted 14 September, 2015; originally announced September 2015.

Comments: 5 pages, 4 figures

Journal ref: Phys. Rev. Lett. 115, 265301 (2015)

Discrimination and characterization of Parkinsonian rest tremors by analyzing long-term correlations and multifractal signatures

Authors: Lorenzo Livi, Alireza Sadeghian, Hamid Sadeghian

Abstract: In this paper, we analyze 48 signals of rest tremor velocity related to 12 distinct subjects affected by Parkinson's disease. The subjects belong to two different groups, formed by four and eight subjects with, respectively, high- and low-amplitude rest tremors. Each subject is tested in four settings, given by combining the use of deep brain stimulation and L-DOPA medication. We develop two main… ▽ More In this paper, we analyze 48 signals of rest tremor velocity related to 12 distinct subjects affected by Parkinson's disease. The subjects belong to two different groups, formed by four and eight subjects with, respectively, high- and low-amplitude rest tremors. Each subject is tested in four settings, given by combining the use of deep brain stimulation and L-DOPA medication. We develop two main feature-based representations of such signals, which are obtained by considering (i) the long-term correlations and multifractal properties, and (ii) the power spectra. The feature-based representations are initially utilized for the purpose of characterizing the subjects under different settings. In agreement with previous studies, we show that deep brain stimulation does not significantly characterize neither of the two groups, regardless of the adopted representation. On the other hand, the medication effect yields statistically significant differences in both high- and low-amplitude tremor groups. We successively test several different instances of the two feature-based representations of the signals in the setting of supervised classification and (nonlinear) feature transformation. We consider three different classification problems, involving the recognition of (i) the presence of medication, (ii) the use of deep brain stimulation, and (iii) the membership to the high- and low-amplitude tremor groups. Classification results show that the use of medication can be discriminated with higher accuracy, considering many of the feature-based representations. Notably, we show that the best results are obtained with a parsimonious, two-dimensional representation encoding the long-term correlations and multifractal character of the signals. △ Less

Submitted 15 May, 2015; v1 submitted 10 April, 2015; originally announced April 2015.

Comments: 10 pages, 41 references

On the long-term correlations and multifractal properties of electric arc furnace time series

Authors: Lorenzo Livi, Enrico Maiorino, Antonello Rizzi, Alireza Sadeghian

Abstract: In this paper, we study long-term correlations and multifractal properties elaborated from time series of three-phase current signals coming from an industrial electric arc furnace plant. Implicit sinusoidal trends are suitably detected by considering the scaling of the fluctuation functions. Time series are then filtered via a Fourier-based analysis, removing hence such strong periodicities. In t… ▽ More In this paper, we study long-term correlations and multifractal properties elaborated from time series of three-phase current signals coming from an industrial electric arc furnace plant. Implicit sinusoidal trends are suitably detected by considering the scaling of the fluctuation functions. Time series are then filtered via a Fourier-based analysis, removing hence such strong periodicities. In the filtered time series we detected long-term, positive correlations. The presence of positive correlations is in agreement with the typical V--I characteristic (hysteresis) of the electric arc furnace, providing thus a sound physical justification for the memory effects found in the current time series. The multifractal signature is strong enough in the filtered time series to be effectively classified as multifractal. △ Less

Submitted 15 June, 2015; v1 submitted 11 March, 2015; originally announced March 2015.

A generative model for protein contact networks

Authors: Lorenzo Livi, Enrico Maiorino, Alessandro Giuliani, Antonello Rizzi, Alireza Sadeghian

Abstract: In this paper we present a generative model for protein contact networks. The soundness of the proposed model is investigated by focusing primarily on mesoscopic properties elaborated from the spectra of the graph Laplacian. To complement the analysis, we study also classical topological descriptors, such as statistics of the shortest paths and the important feature of modularity. Our experiments… ▽ More In this paper we present a generative model for protein contact networks. The soundness of the proposed model is investigated by focusing primarily on mesoscopic properties elaborated from the spectra of the graph Laplacian. To complement the analysis, we study also classical topological descriptors, such as statistics of the shortest paths and the important feature of modularity. Our experiments show that the proposed model results in a considerable improvement with respect to two suitably chosen generative mechanisms, mimicking with better approximation real protein contact networks in terms of diffusion properties elaborated from the Laplacian spectra. However, as well as the other considered models, it does not reproduce with sufficient accuracy the shortest paths structure. To compensate this drawback, we designed a second step involving a targeted edge reconfiguration process. The ensemble of reconfigured networks denotes improvements that are statistically significant. As a byproduct of our study, we demonstrate that modularity, a well-known property of proteins, does not entirely explain the actual network architecture characterizing protein contact networks. In fact, we conclude that modularity, intended as a quantification of an underlying community structure, should be considered as an emergent property of the structural organization of proteins. Interestingly, such a property is suitably optimized in protein contact networks together with the feature of path efficiency. △ Less

Submitted 8 March, 2015; originally announced March 2015.

Comments: 18 pages, 67 references

Observation of chiral edge states with neutral fermions in synthetic Hall ribbons

Authors: M. Mancini, G. Pagano, G. Cappellini, L. Livi, M. Rider, J. Catani, C. Sias, P. Zoller, M. Inguscio, M. Dalmonte, L. Fallani

Abstract: Chiral edge states are a hallmark of quantum Hall physics. In electronic systems, they appear as a macroscopic consequence of the cyclotron orbits induced by a magnetic field, which are naturally truncated at the physical boundary of the sample. Here we report on the experimental realization of chiral edge states in a ribbon geometry with an ultracold gas of neutral fermions subjected to an artifi… ▽ More Chiral edge states are a hallmark of quantum Hall physics. In electronic systems, they appear as a macroscopic consequence of the cyclotron orbits induced by a magnetic field, which are naturally truncated at the physical boundary of the sample. Here we report on the experimental realization of chiral edge states in a ribbon geometry with an ultracold gas of neutral fermions subjected to an artificial gauge field. By imaging individual sites along a synthetic dimension, we detect the existence of the edge states, investigate the onset of chirality as a function of the bulk-edge coupling, and observe the edge-cyclotron orbits induced during a quench dynamics. The realization of fermionic chiral edge states is a fundamental achievement, which opens the door towards experiments including edge state interferometry and the study of non-Abelian anyons in atomic systems. △ Less

Submitted 9 February, 2015; originally announced February 2015.

Comments: 10 pages (6 + 4 supplementary material)

On the impact of topological properties of smart grids in power losses optimization problems

Authors: Francesca Possemato, Maurizio Paschero, Lorenzo Livi, Antonello Rizzi, Alireza Sadeghian

Abstract: Power losses reduction is one of the main targets for any electrical energy distribution company. In this paper, we face the problem of joint optimization of both topology and network parameters in a real smart grid. We consider a portion of the Italian electric distribution network managed by the ACEA Distribuzione S.p.A. located in Rome. We perform both the power factor correction (PFC) for tuni… ▽ More Power losses reduction is one of the main targets for any electrical energy distribution company. In this paper, we face the problem of joint optimization of both topology and network parameters in a real smart grid. We consider a portion of the Italian electric distribution network managed by the ACEA Distribuzione S.p.A. located in Rome. We perform both the power factor correction (PFC) for tuning the generators and the distributed feeder reconfiguration (DFR) to set the state of the breakers. This joint optimization problem is faced considering a suitable objective function and by adopting genetic algorithms as global optimization strategy. We analyze admissible network configurations, showing that some of these violate constraints on current and voltage at branches and nodes. Such violations depend only on pure topological properties of the configurations. We perform tests by feeding the simulation environment with real data concerning hourly samples of dissipated and generated active and reactive power values of the ACEA smart grid. Results show that removing the configurations violating the electrical constraints from the solution space leads to interesting improvements in terms of power loss reduction. To conclude, we provide also an electrical interpretation of the phenomenon using graph-based pattern analysis techniques. △ Less

Submitted 21 January, 2015; v1 submitted 19 January, 2015; originally announced January 2015.

Comments: 35 pages, 38 references

Building pattern recognition applications with the SPARE library

Authors: Lorenzo Livi, Guido Del Vescovo, Antonello Rizzi, Fabio Massimo Frattale Mascioli

Abstract: This paper presents the SPARE C++ library, an open source software tool conceived to build pattern recognition and soft computing systems. The library follows the requirement of the generality: most of the implemented algorithms are able to process user-defined input data types transparently, such as labeled graphs and sequences of objects, as well as standard numeric vectors. Here we present a hi… ▽ More This paper presents the SPARE C++ library, an open source software tool conceived to build pattern recognition and soft computing systems. The library follows the requirement of the generality: most of the implemented algorithms are able to process user-defined input data types transparently, such as labeled graphs and sequences of objects, as well as standard numeric vectors. Here we present a high-level picture of the SPARE library characteristics, focusing instead on the specific practical possibility of constructing pattern recognition systems for different input data types. In particular, as a proof of concept, we discuss two application instances involving clustering of real-valued multidimensional sequences and classification of labeled graphs. △ Less

Submitted 20 February, 2015; v1 submitted 20 October, 2014; originally announced October 2014.

Comments: Home page: https://sourceforge.net/p/libspare/home/Spare/

ACM Class: D.2.2

Multifractal Characterization of Protein Contact Networks

Authors: Enrico Maiorino, Lorenzo Livi, Alessandro Giuliani, Alireza Sadeghian, Antonello Rizzi

Abstract: The multifractal detrended fluctuation analysis of time series is able to reveal the presence of long-range correlations and, at the same time, to characterize the self-similarity of the series. The rich information derivable from the characteristic exponents and the multifractal spectrum can be further analyzed to discover important insights about the underlying dynamical process. In this paper,… ▽ More The multifractal detrended fluctuation analysis of time series is able to reveal the presence of long-range correlations and, at the same time, to characterize the self-similarity of the series. The rich information derivable from the characteristic exponents and the multifractal spectrum can be further analyzed to discover important insights about the underlying dynamical process. In this paper, we employ multifractal analysis techniques in the study of protein contact networks. To this end, initially a network is mapped to three different time series, each of which is generated by a stationary unbiased random walk. To capture the peculiarities of the networks at different levels, we accordingly consider three observables at each vertex: the degree, the clustering coefficient, and the closeness centrality. To compare the results with suitable references, we consider also instances of three well-known network models and two typical time series with pure monofractal and multifractal properties. The first result of notable interest is that time series associated to proteins contact networks exhibit long-range correlations (strong persistence), which are consistent with signals in-between the typical monofractal and multifractal behavior. Successively, a suitable embedding of the multifractal spectra allows to focus on ensemble properties, which in turn gives us the possibility to make further observations regarding the considered networks. In particular, we highlight the different role that small and large fluctuations of the considered observables play in the characterization of the network topology. △ Less

Submitted 9 October, 2014; v1 submitted 3 October, 2014; originally announced October 2014.

An Agent-Based Algorithm exploiting Multiple Local Dissimilarities for Clusters Mining and Knowledge Discovery

Authors: Filippo Maria Bianchi, Enrico Maiorino, Lorenzo Livi, Antonello Rizzi, Alireza Sadeghian

Abstract: We propose a multi-agent algorithm able to automatically discover relevant regularities in a given dataset, determining at the same time the set of configurations of the adopted parametric dissimilarity measure yielding compact and separated clusters. Each agent operates independently by performing a Markovian random walk on a suitable weighted graph representation of the input dataset. Such a wei… ▽ More We propose a multi-agent algorithm able to automatically discover relevant regularities in a given dataset, determining at the same time the set of configurations of the adopted parametric dissimilarity measure yielding compact and separated clusters. Each agent operates independently by performing a Markovian random walk on a suitable weighted graph representation of the input dataset. Such a weighted graph representation is induced by the specific parameter configuration of the dissimilarity measure adopted by the agent, which searches and takes decisions autonomously for one cluster at a time. Results show that the algorithm is able to discover parameter configurations that yield a consistent and interpretable collection of clusters. Moreover, we demonstrate that our algorithm shows comparable performances with other similar state-of-the-art algorithms when facing specific clustering problems. △ Less

Submitted 17 September, 2014; originally announced September 2014.

Analysis of heat kernel highlights the strongly modular and heat-preserving structure of proteins

Authors: Lorenzo Livi, Enrico Maiorino, Andrea Pinna, Alireza Sadeghian, Antonello Rizzi, Alessandro Giuliani

Abstract: In this paper, we study the structure and dynamical properties of protein contact networks with respect to other biological networks, together with simulated archetypal models acting as probes. We consider both classical topological descriptors, such as the modularity and statistics of the shortest paths, and different interpretations in terms of diffusion provided by the discrete heat kernel, whi… ▽ More In this paper, we study the structure and dynamical properties of protein contact networks with respect to other biological networks, together with simulated archetypal models acting as probes. We consider both classical topological descriptors, such as the modularity and statistics of the shortest paths, and different interpretations in terms of diffusion provided by the discrete heat kernel, which is elaborated from the normalized graph Laplacians. A principal component analysis shows high discrimination among the network types, either by considering the topological and heat kernel based vector characterizations. Furthermore, a canonical correlation analysis demonstrates the strong agreement among those two characterizations, providing thus an important justification in terms of interpretability for the heat kernel. Finally, and most importantly, the focused analysis of the heat kernel provides a way to yield insights on the fact that proteins have to satisfy specific structural design constraints that the other considered networks do not need to obey. Notably, the heat trace decay of an ensemble of varying-size proteins denotes subdiffusion, a peculiar property of proteins. △ Less

Submitted 16 March, 2015; v1 submitted 5 September, 2014; originally announced September 2014.

Designing labeled graph classifiers by exploiting the Rényi entropy of the dissimilarity representation

Authors: Lorenzo Livi

Abstract: Representing patterns as labeled graphs is becoming increasingly common in the broad field of computational intelligence. Accordingly, a wide repertoire of pattern recognition tools, such as classifiers and knowledge discovery procedures, are nowadays available and tested for various datasets of labeled graphs. However, the design of effective learning procedures operating in the space of labeled… ▽ More Representing patterns as labeled graphs is becoming increasingly common in the broad field of computational intelligence. Accordingly, a wide repertoire of pattern recognition tools, such as classifiers and knowledge discovery procedures, are nowadays available and tested for various datasets of labeled graphs. However, the design of effective learning procedures operating in the space of labeled graphs is still a challenging problem, especially from the computational complexity viewpoint. In this paper, we present a major improvement of a general-purpose classifier for graphs, which is conceived on an interplay between dissimilarity representation, clustering, information-theoretic techniques, and evolutionary optimization algorithms. The improvement focuses on a specific key subroutine devised to compress the input data. We prove different theorems which are fundamental to the setting of the parameters controlling such a compression operation. We demonstrate the effectiveness of the resulting classifier by benchmarking the developed variants on well-known datasets of labeled graphs, considering as distinct performance indicators the classification accuracy, computing time, and parsimony in terms of structural complexity of the synthesized classification models. The results show state-of-the-art standards in terms of test set accuracy and a considerable speed-up for what concerns the computing time. △ Less

Submitted 20 April, 2017; v1 submitted 22 August, 2014; originally announced August 2014.

Comments: Revised version

ACM Class: I.2.6; H.1.1; E.4