Showing 1–50 of 70 results for author: Micheli, A

Formal Analysis and Verification of Max-Plus Linear Systems

Authors: Muhammad Syifa'ul Mufid, Andrea Micheli, Alessandro Abate, Alessandro Cimatti

Abstract: Max-Plus Linear (MPL) systems are an algebraic formalism with practical applications in transportation networks, manufacturing and biological systems. In this paper, we investigate the problem of automatically analyzing the properties of MPL, taking into account both structural properties such as transient and cyclicity, and the open problem of user-defined temporal properties. We propose Time-Dif… ▽ More Max-Plus Linear (MPL) systems are an algebraic formalism with practical applications in transportation networks, manufacturing and biological systems. In this paper, we investigate the problem of automatically analyzing the properties of MPL, taking into account both structural properties such as transient and cyclicity, and the open problem of user-defined temporal properties. We propose Time-Difference LTL (TDLTL), a logic that encompasses the delays between the discrete time events governed by an MPL system, and characterize the problem of model checking TDLTL over MPL. We first consider a framework based on the verification of infinite-state transition systems, and propose an approach based on an encoding into model checking. Then, we leverage the specific features of MPL systems to devise a highly optimized, combinational approach based on Satisfiability Modulo Theory (SMT). We experimentally evaluate the features of the proposed approaches on a large set of benchmarks. The results show that the proposed approach substantially outperforms the state of the art competitors in expressiveness and effectiveness, and demonstrate the superiority of the combinational approach over the reduction to model checking. △ Less

Submitted 21 August, 2023; originally announced August 2023.

Comments: 28 pages (including appendixes)

Modeling Edge Features with Deep Bayesian Graph Networks

Authors: Daniele Atzeni, Federico Errica, Davide Bacciu, Alessio Micheli

Abstract: We propose an extension of the Contextual Graph Markov Model, a deep and probabilistic machine learning model for graphs, to model the distribution of edge features. Our approach is architectural, as we introduce an additional Bayesian network map** edge features into discrete states to be used by the original model. In doing so, we are also able to build richer graph representations even in the… ▽ More We propose an extension of the Contextual Graph Markov Model, a deep and probabilistic machine learning model for graphs, to model the distribution of edge features. Our approach is architectural, as we introduce an additional Bayesian network map** edge features into discrete states to be used by the original model. In doing so, we are also able to build richer graph representations even in the absence of edge features, which is confirmed by the performance improvements on standard graph classification benchmarks. Moreover, we successfully test our proposal in a graph regression scenario where edge features are of fundamental importance, and we show that the learned edge representation provides substantial performance improvements against the original model on three link prediction tasks. By kee** the computational complexity linear in the number of edges, the proposed model is amenable to large-scale graph processing. △ Less

Submitted 17 August, 2023; originally announced August 2023.

Comments: Releasing pre-print version to comply with TAILOR project requirements

Is Rewiring Actually Helpful in Graph Neural Networks?

Authors: Domenico Tortorella, Alessio Micheli

Abstract: Graph neural networks compute node representations by performing multiple message-passing steps that consist in local aggregations of node features. Having deep models that can leverage longer-range interactions between nodes is hindered by the issues of over-smoothing and over-squashing. In particular, the latter is attributed to the graph topology which guides the message-passing, causing a node… ▽ More Graph neural networks compute node representations by performing multiple message-passing steps that consist in local aggregations of node features. Having deep models that can leverage longer-range interactions between nodes is hindered by the issues of over-smoothing and over-squashing. In particular, the latter is attributed to the graph topology which guides the message-passing, causing a node representation to become insensitive to information contained at distant nodes. Many graph rewiring methods have been proposed to remedy or mitigate this problem. However, properly evaluating the benefits of these methods is made difficult by the coupling of over-squashing with other issues strictly related to model training, such as vanishing gradients. Therefore, we propose an evaluation setting based on message-passing models that do not require training to compute node and graph representations. We perform a systematic experimental comparison on real-world node and graph classification tasks, showing that rewiring the underlying graph rarely does confer a practical benefit for message-passing. △ Less

Submitted 31 May, 2023; originally announced May 2023.

Addressing Heterophily in Node Classification with Graph Echo State Networks

Authors: Alessio Micheli, Domenico Tortorella

Abstract: Node classification tasks on graphs are addressed via fully-trained deep message-passing models that learn a hierarchy of node representations via multiple aggregations of a node's neighbourhood. While effective on graphs that exhibit a high ratio of intra-class edges, this approach poses challenges in the opposite case, i.e. heterophily, where nodes belonging to the same class are usually further… ▽ More Node classification tasks on graphs are addressed via fully-trained deep message-passing models that learn a hierarchy of node representations via multiple aggregations of a node's neighbourhood. While effective on graphs that exhibit a high ratio of intra-class edges, this approach poses challenges in the opposite case, i.e. heterophily, where nodes belonging to the same class are usually further apart. In graphs with a high degree of heterophily, the smoothed representations based on close neighbours computed by convolutional models are no longer effective. So far, architectural variations in message-passing models to reduce excessive smoothing or rewiring the input graph to improve longer-range message passing have been proposed. In this paper, we address the challenges of heterophilic graphs with Graph Echo State Network (GESN) for node classification. GESN is a reservoir computing model for graphs, where node embeddings are recursively computed by an untrained message-passing function. Our experiments show that reservoir models are able to achieve better or comparable accuracy with respect to most fully trained deep models that implement ad hoc variations in the architectural bias or perform rewiring as a preprocessing step on the input graph, with an improvement in terms of efficiency/accuracy trade-off. Furthermore, our analysis shows that GESN is able to effectively encode the structural relationships of a graph node, by showing a correlation between iterations of the recursive embedding function and the distribution of shortest paths in a graph. △ Less

Submitted 3 July, 2023; v1 submitted 14 May, 2023; originally announced May 2023.

Comments: 15 pages, 10 figures. arXiv admin note: text overlap with arXiv:2212.06538

Journal ref: Neurocomputing, vol. 550, article 126506 (2023)

NeuroBench: A Framework for Benchmarking Neuromorphic Computing Algorithms and Systems

Authors: Jason Yik, Korneel Van den Berghe, Douwe den Blanken, Younes Bouhadjar, Maxime Fabre, Paul Hueber, Denis Kleyko, Noah Pacik-Nelson, Pao-Sheng Vincent Sun, Guangzhi Tang, Shenqi Wang, Biyan Zhou, Soikat Hasan Ahmed, George Vathakkattil Joseph, Benedetto Leto, Aurora Micheli, Anurag Kumar Mishra, Gregor Lenz, Tao Sun, Zergham Ahmed, Mahmoud Akl, Brian Anderson, Andreas G. Andreou, Chiara Bartolozzi, Arindam Basu , et al. (73 additional authors not shown)

Abstract: Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. Prior neu… ▽ More Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. Prior neuromorphic computing benchmark efforts have not seen widespread adoption due to a lack of inclusive, actionable, and iterative benchmark design and guidelines. To address these shortcomings, we present NeuroBench: a benchmark framework for neuromorphic computing algorithms and systems. NeuroBench is a collaboratively-designed effort from an open community of nearly 100 co-authors across over 50 institutions in industry and academia, aiming to provide a representative structure for standardizing the evaluation of neuromorphic approaches. The NeuroBench framework introduces a common set of tools and systematic methodology for inclusive benchmark measurement, delivering an objective reference framework for quantifying neuromorphic approaches in both hardware-independent (algorithm track) and hardware-dependent (system track) settings. In this article, we present initial performance baselines across various model architectures on the algorithm track and outline the system track benchmark tasks and guidelines. NeuroBench is intended to continually expand its benchmarks and features to foster and track the progress made by the research community. △ Less

Submitted 17 January, 2024; v1 submitted 10 April, 2023; originally announced April 2023.

Comments: Updated from whitepaper to full perspective article preprint

An Efficient Incremental Simple Temporal Network Data Structure for Temporal Planning

Authors: Andrea Micheli

Abstract: One popular technique to solve temporal planning problems consists in decoupling the causal decisions, demanding them to heuristic search, from temporal decisions, demanding them to a simple temporal network (STN) solver. In this architecture, one needs to check the consistency of a series of STNs that are related one another, therefore having methods to incrementally re-use previous computations… ▽ More One popular technique to solve temporal planning problems consists in decoupling the causal decisions, demanding them to heuristic search, from temporal decisions, demanding them to a simple temporal network (STN) solver. In this architecture, one needs to check the consistency of a series of STNs that are related one another, therefore having methods to incrementally re-use previous computations and that avoid expensive memory duplication is of paramount importance. In this paper, we describe in detail how STNs are used in temporal planning, we identify a clear interface to support this use-case and we present an efficient data-structure implementing this interface that is both time- and memory-efficient. We show that our data structure, called \deltastn, is superior to other state-of-the-art approaches on temporal planning sequences of problems. △ Less

Submitted 11 August, 2023; v1 submitted 14 December, 2022; originally announced December 2022.

Comments: V2: Fixed a typo in the algorithm pseudocode

Leave Graphs Alone: Addressing Over-Squashing without Rewiring

Authors: Domenico Tortorella, Alessio Micheli

Abstract: Recent works have investigated the role of graph bottlenecks in preventing long-range information propagation in message-passing graph neural networks, causing the so-called `over-squashing' phenomenon. As a remedy, graph rewiring mechanisms have been proposed as preprocessing steps. Graph Echo State Networks (GESNs) are a reservoir computing model for graphs, where node embeddings are recursively… ▽ More Recent works have investigated the role of graph bottlenecks in preventing long-range information propagation in message-passing graph neural networks, causing the so-called `over-squashing' phenomenon. As a remedy, graph rewiring mechanisms have been proposed as preprocessing steps. Graph Echo State Networks (GESNs) are a reservoir computing model for graphs, where node embeddings are recursively computed by an untrained message-passing function. In this paper, we show that GESNs can achieve a significantly better accuracy on six heterophilic node classification tasks without altering the graph connectivity, thus suggesting a different route for addressing the over-squashing problem. △ Less

Submitted 13 December, 2022; originally announced December 2022.

Comments: Extended Abstract. Presented at the First Learning on Graphs Conference (LoG 2022), Virtual Event, December 9-12, 2022

Comparing quantumness criteria

Authors: Jerome Martin, Amaury Micheli, Vincent Vennin

Abstract: Measuring the quantumness of a system can be done with a variety of methods. In this article we compare different criteria, namely quantum discord, Bell inequality violation and non-separability, for systems placed in a Gaussian state. When the state is pure, these criteria are equivalent, while we find that they do not necessarily coincide when decoherence takes place. Finally, we prove that thes… ▽ More Measuring the quantumness of a system can be done with a variety of methods. In this article we compare different criteria, namely quantum discord, Bell inequality violation and non-separability, for systems placed in a Gaussian state. When the state is pure, these criteria are equivalent, while we find that they do not necessarily coincide when decoherence takes place. Finally, we prove that these criteria are essentially controlled by the semi-minor axis of the ellipse representing the state's Wigner function in phase space. △ Less

Submitted 20 March, 2023; v1 submitted 18 November, 2022; originally announced November 2022.

Comments: 8 pages without appendix (total 19 pages), 5 figures. Matches published version in EPL

Quantum cosmological gravitational waves?

Authors: Amaury Micheli, Patrick Peter

Abstract: General relativity and its cosmological solution predicts the existence of tensor modes of perturbations evolving on top of our Friedman-Lemaître-Robertson-Walker expanding Universe. Being gauge invariant and not necessarily coupled to other quantum sources, they can be seen as representing pure gravity. Unambiguously showing they are indeed to be quantised would thus provide an unquestionable pro… ▽ More General relativity and its cosmological solution predicts the existence of tensor modes of perturbations evolving on top of our Friedman-Lemaître-Robertson-Walker expanding Universe. Being gauge invariant and not necessarily coupled to other quantum sources, they can be seen as representing pure gravity. Unambiguously showing they are indeed to be quantised would thus provide an unquestionable proof of the quantum nature of gravitation. This review will present a summary of the various theoretical issues that could lead to this conclusion. △ Less

Submitted 18 January, 2023; v1 submitted 31 October, 2022; originally announced November 2022.

Comments: Invited chapter for the Section "Perturbative Quantum Gravity" of the "Handbook of Quantum Gravity" (Eds. C. Bambi, L. Modesto and I.L. Shapiro, Springer Singapore, expected in 2023). New version matches the accepted version

Beyond Homophily with Graph Echo State Networks

Authors: Domenico Tortorella, Alessio Micheli

Abstract: Graph Echo State Networks (GESN) have already demonstrated their efficacy and efficiency in graph classification tasks. However, semi-supervised node classification brought out the problem of over-smoothing in end-to-end trained deep models, which causes a bias towards high homophily graphs. We evaluate for the first time GESN on node classification tasks with different degrees of homophily, analy… ▽ More Graph Echo State Networks (GESN) have already demonstrated their efficacy and efficiency in graph classification tasks. However, semi-supervised node classification brought out the problem of over-smoothing in end-to-end trained deep models, which causes a bias towards high homophily graphs. We evaluate for the first time GESN on node classification tasks with different degrees of homophily, analyzing also the impact of the reservoir radius. Our experiments show that reservoir models are able to achieve better or comparable accuracy with respect to fully trained deep models that implement ad hoc variations in the architectural bias, with a gain in terms of efficiency. △ Less

Submitted 27 October, 2022; originally announced October 2022.

Comments: Accepted for oral presentation at ESANN 2022

Journal ref: Proceedings of the 30th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2022), pp. 491-496

Fast and Slow Optimal Trading with Exogenous Information

Authors: Rama Cont, Alessandro Micheli, Eyal Neuman

Abstract: We consider a stochastic game between a slow institutional investor and a high-frequency trader who are trading a risky asset and their aggregated order-flow impacts the asset price. We model this system by means of two coupled stochastic control problems, in which the high-frequency trader exploits the available information on a price predicting signal more frequently, but is also subject to peri… ▽ More We consider a stochastic game between a slow institutional investor and a high-frequency trader who are trading a risky asset and their aggregated order-flow impacts the asset price. We model this system by means of two coupled stochastic control problems, in which the high-frequency trader exploits the available information on a price predicting signal more frequently, but is also subject to periodic "end of day" inventory constraints. We first derive the optimal strategy of the high-frequency trader given any admissible strategy of the institutional investor. Then, we solve the problem of the institutional investor given the optimal signal-adaptive strategy of the high-frequency trader, in terms of the resolvent of a Fredholm integral equation, thus establishing the unique multi-period Stackelberg equilibrium of the game. Our results provide an explicit solution to the game, which shows that the high-frequency trader can adopt either predatory or cooperative strategies in each period, depending on the tradeoff between the order-flow and the trading signal. We also show that the institutional investor's strategy is considerably more profitable when the order-flow of the high-frequency trader is taken into account in her trading strategy. △ Less

Submitted 23 June, 2023; v1 submitted 4 October, 2022; originally announced October 2022.

Comments: 66 pages, 6 figures

MSC Class: 49N70; 49N90; 93E20; 60H30

Phonon decay in 1D atomic Bose quasicondensates via Beliaev-Landau dam**

Authors: Amaury Micheli, Scott Robertson

Abstract: In a 1D Bose gas, there is no non-trivial scattering channel involving three Bogoliubov quasiparticles that conserves both energy and momentum. Nevertheless, we show that such 3-wave mixing processes (Beliaev and Landau dam**) account for their decay via interactions with thermal fluctuations. Within an appropriate time window where the Fermi Golden Rule is expected to apply, the occupation numb… ▽ More In a 1D Bose gas, there is no non-trivial scattering channel involving three Bogoliubov quasiparticles that conserves both energy and momentum. Nevertheless, we show that such 3-wave mixing processes (Beliaev and Landau dam**) account for their decay via interactions with thermal fluctuations. Within an appropriate time window where the Fermi Golden Rule is expected to apply, the occupation number of the initially occupied mode decays exponentially and the rate takes a simple analytic form. The result is shown to compare favorably with simulations based on the Truncated Wigner Approximation. It is also shown that the same processes slow down the exponential growth of phonons induced by a parametric oscillation. △ Less

Submitted 22 December, 2022; v1 submitted 31 May, 2022; originally announced May 2022.

Comments: 12 pages, 6 figures; Appendices: 16 pages, 8 figures; Version accepted for publication in Physical Review B

Brain-Computer Interfaces: Investigating the Transition from Visually Evoked to Purely Imagined Steady-State Potentials

Authors: Arturo Micheli, Davide Consoli, Adrien Merlini, Paolo Ricci, Francesco P. Andriulli

Abstract: Brain-Computer Interfaces (BCIs) based on Steady State Visually Evoked Potentials (SSVEPs) have proven effective and provide significant accuracy and information-transfer rates. This family of strategies, however, requires external devices that provide the frequency stimuli required by the technique. This limits the scenarios in which they can be applied, especially when compared to other BCI appr… ▽ More Brain-Computer Interfaces (BCIs) based on Steady State Visually Evoked Potentials (SSVEPs) have proven effective and provide significant accuracy and information-transfer rates. This family of strategies, however, requires external devices that provide the frequency stimuli required by the technique. This limits the scenarios in which they can be applied, especially when compared to other BCI approaches. In this work, we have investigated the possibility of obtaining frequency responses in the EEG output based on the pure visual imagination of SSVEP-eliciting stimuli. Our results show that not only that EEG signals present frequency-specific peaks related to the frequency the user is focusing on, but also that promising classification accuracy can be achieved, paving the way for a robust and reliable visual imagery BCI modality. △ Less

Submitted 25 April, 2022; originally announced April 2022.

Discord and Decoherence

Authors: Jerome Martin, Amaury Micheli, Vincent Vennin

Abstract: In quantum information theory, quantum discord has been proposed as a tool to characterise the presence of "quantum correlations" between the subparts of a given system. Whether a system behaves quantum-mechanically or classically is believed to be impacted by the phenomenon of decoherence, which originates from the unavoidable interaction between this system and an environment. Generically, decoh… ▽ More In quantum information theory, quantum discord has been proposed as a tool to characterise the presence of "quantum correlations" between the subparts of a given system. Whether a system behaves quantum-mechanically or classically is believed to be impacted by the phenomenon of decoherence, which originates from the unavoidable interaction between this system and an environment. Generically, decoherence is associated with a decrease of the state purity, i.e. a transition from a pure to a mixed state. In this paper, we investigate how quantum discord is modified by this quantum-to-classical transition. This study is carried out on systems described by quadratic Hamiltonians and Gaussian states, with generalised squeezing parameters. A generic parametrisation is also introduced to describe the way the system is partitioned into two subsystems. We find that the evolution of quantum discord in presence of an environment is a competition between the growth of the squeezing amplitude and the decrease of the state purity. In phase space, this corresponds to whether the semi-minor axis of the Wigner ellipse increases or decreases, which has a clear geometrical interpretation. Finally, these considerations are applied to primordial cosmological perturbations, thus allowing us to investigate how large-scale structures in our universe, which are believed to arise from quantum fluctuations, can exhibit classical properties. △ Less

Submitted 19 October, 2022; v1 submitted 9 December, 2021; originally announced December 2021.

Comments: 35 pages without appendices (total 57 pages), 10 figures, typo corrected in the labels of figure 3

Closed-Loop Nash Competition for Liquidity

Authors: Alessandro Micheli, Johannes Muhle-Karbe, Eyal Neuman

Abstract: We study a multi-player stochastic differential game, where agents interact through their joint price impact on an asset that they trade to exploit a common trading signal. In this context, we prove that a closed-loop Nash equilibrium exists if the price impact parameter is small enough. Compared to the corresponding open-loop Nash equilibrium, both the agents' optimal trading rates and their perf… ▽ More We study a multi-player stochastic differential game, where agents interact through their joint price impact on an asset that they trade to exploit a common trading signal. In this context, we prove that a closed-loop Nash equilibrium exists if the price impact parameter is small enough. Compared to the corresponding open-loop Nash equilibrium, both the agents' optimal trading rates and their performance move towards the central-planner solution, in that excessive trading due to lack of coordination is reduced. However, the size of this effect is modest for plausible parameter values. △ Less

Submitted 21 June, 2023; v1 submitted 6 December, 2021; originally announced December 2021.

Comments: 41 pages, 5 figures, supplementary appendix

MSC Class: 49N90; 91A25; 91G10; 93E20

Dynamic Graph Echo State Networks

Authors: Domenico Tortorella, Alessio Micheli

Abstract: Dynamic temporal graphs represent evolving relations between entities, e.g. interactions between social network users or infection spreading. We propose an extension of graph echo state networks for the efficient processing of dynamic temporal graphs, with a sufficient condition for their echo state property, and an experimental analysis of reservoir layout impact. Compared to temporal graph kerne… ▽ More Dynamic temporal graphs represent evolving relations between entities, e.g. interactions between social network users or infection spreading. We propose an extension of graph echo state networks for the efficient processing of dynamic temporal graphs, with a sufficient condition for their echo state property, and an experimental analysis of reservoir layout impact. Compared to temporal graph kernels that need to hold the entire history of vertex interactions, our model provides a vector encoding for the dynamic graph that is updated at each time-step without requiring training. Experiments show accuracy comparable to approximate temporal graph kernels on twelve dissemination process classification tasks. △ Less

Submitted 27 October, 2022; v1 submitted 16 October, 2021; originally announced October 2021.

Comments: Accepted for oral presentation at ESANN 2021

Journal ref: Proceedings of the 29th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2021), pp. 99-104

TEACHING -- Trustworthy autonomous cyber-physical applications through human-centred intelligence

Authors: Davide Bacciu, Siranush Akarmazyan, Eric Armengaud, Manlio Bacco, George Bravos, Calogero Calandra, Emanuele Carlini, Antonio Carta, Pietro Cassara, Massimo Coppola, Charalampos Davalas, Patrizio Dazzi, Maria Carmela Degennaro, Daniele Di Sarli, Jürgen Dobaj, Claudio Gallicchio, Sylvain Girbal, Alberto Gotta, Riccardo Groppo, Vincenzo Lomonaco, Georg Macher, Daniele Mazzei, Gabriele Mencagli, Dimitrios Michail, Alessio Micheli , et al. (10 additional authors not shown)

Abstract: This paper discusses the perspective of the H2020 TEACHING project on the next generation of autonomous applications running in a distributed and highly heterogeneous environment comprising both virtual and physical resources spanning the edge-cloud continuum. TEACHING puts forward a human-centred vision leveraging the physiological, emotional, and cognitive state of the users as a driver for the… ▽ More This paper discusses the perspective of the H2020 TEACHING project on the next generation of autonomous applications running in a distributed and highly heterogeneous environment comprising both virtual and physical resources spanning the edge-cloud continuum. TEACHING puts forward a human-centred vision leveraging the physiological, emotional, and cognitive state of the users as a driver for the adaptation and optimization of the autonomous applications. It does so by building a distributed, embedded and federated learning system complemented by methods and tools to enforce its dependability, security and privacy preservation. The paper discusses the main concepts of the TEACHING approach and singles out the main AI-related research challenges associated with it. Further, we provide a discussion of the design choices for the TEACHING system to tackle the aforementioned challenges △ Less

Submitted 14 July, 2021; originally announced July 2021.

Phase Transition Adaptation

Authors: Claudio Gallicchio, Alessio Micheli, Luca Silvestri

Abstract: Artificial Recurrent Neural Networks are a powerful information processing abstraction, and Reservoir Computing provides an efficient strategy to build robust implementations by projecting external inputs into high dimensional dynamical system trajectories. In this paper, we propose an extension of the original approach, a local unsupervised learning mechanism we call Phase Transition Adaptation,… ▽ More Artificial Recurrent Neural Networks are a powerful information processing abstraction, and Reservoir Computing provides an efficient strategy to build robust implementations by projecting external inputs into high dimensional dynamical system trajectories. In this paper, we propose an extension of the original approach, a local unsupervised learning mechanism we call Phase Transition Adaptation, designed to drive the system dynamics towards the `edge of stability'. Here, the complex behavior exhibited by the system elicits an enhancement in its overall computational capacity. We show experimentally that our approach consistently achieves its purpose over several datasets. △ Less

Submitted 20 April, 2021; originally announced April 2021.

Comments: Accepted at IJCNN 2021

Pyramidal Reservoir Graph Neural Network

Authors: Filippo Maria Bianchi, Claudio Gallicchio, Alessio Micheli

Abstract: We propose a deep Graph Neural Network (GNN) model that alternates two types of layers. The first type is inspired by Reservoir Computing (RC) and generates new vertex features by iterating a non-linear map until it converges to a fixed point. The second type of layer implements graph pooling operations, that gradually reduce the support graph and the vertex features, and further improve the compu… ▽ More We propose a deep Graph Neural Network (GNN) model that alternates two types of layers. The first type is inspired by Reservoir Computing (RC) and generates new vertex features by iterating a non-linear map until it converges to a fixed point. The second type of layer implements graph pooling operations, that gradually reduce the support graph and the vertex features, and further improve the computational efficiency of the RC-based GNN. The architecture is, therefore, pyramidal. In the last layer, the features of the remaining vertices are combined into a single vector, which represents the graph embedding. Through a mathematical derivation introduced in this paper, we show formally how graph pooling can reduce the computational complexity of the model and speed-up the convergence of the dynamical updates of the vertex features. Our proposed approach to the design of RC-based GNNs offers an advantageous and principled trade-off between accuracy and complexity, which we extensively demonstrate in experiments on a large set of graph datasets. △ Less

Submitted 10 April, 2021; originally announced April 2021.

Comments: this is a pre-print version of a paper submitted for journal publication

Graph Mixture Density Networks

Authors: Federico Errica, Davide Bacciu, Alessio Micheli

Abstract: We introduce the Graph Mixture Density Networks, a new family of machine learning models that can fit multimodal output distributions conditioned on graphs of arbitrary topology. By combining ideas from mixture models and graph representation learning, we address a broader class of challenging conditional density estimation problems that rely on structured data. In this respect, we evaluate our me… ▽ More We introduce the Graph Mixture Density Networks, a new family of machine learning models that can fit multimodal output distributions conditioned on graphs of arbitrary topology. By combining ideas from mixture models and graph representation learning, we address a broader class of challenging conditional density estimation problems that rely on structured data. In this respect, we evaluate our method on a new benchmark application that leverages random graphs for stochastic epidemic simulations. We show a significant improvement in the likelihood of epidemic outcomes when taking into account both multimodality and structure. The empirical analysis is complemented by two real-world regression tasks showing the effectiveness of our approach in modeling the output prediction uncertainty. Graph Mixture Density Networks open appealing research opportunities in the study of structure-dependent phenomena that exhibit non-trivial conditional output distributions. △ Less

Submitted 25 June, 2021; v1 submitted 5 December, 2020; originally announced December 2020.

Journal ref: Proceedings of the 38th International Conference on Machine Learning, PMLR 139 (2021)

Accelerating the identification of informative reduced representations of proteins with deep learning for graphs

Authors: Federico Errica, Marco Giulini, Davide Bacciu, Roberto Menichetti, Alessio Micheli, Raffaello Potestio

Abstract: The limits of molecular dynamics (MD) simulations of macromolecules are steadily pushed forward by the relentless developments of computer architectures and algorithms. This explosion in the number and extent (in size and time) of MD trajectories induces the need of automated and transferable methods to rationalise the raw data and make quantitative sense out of them. Recently, an algorithmic appr… ▽ More The limits of molecular dynamics (MD) simulations of macromolecules are steadily pushed forward by the relentless developments of computer architectures and algorithms. This explosion in the number and extent (in size and time) of MD trajectories induces the need of automated and transferable methods to rationalise the raw data and make quantitative sense out of them. Recently, an algorithmic approach was developed by some of us to identify the subset of a protein's atoms, or map**, that enables the most informative description of it. This method relies on the computation, for a given reduced representation, of the associated map** entropy, that is, a measure of the information loss due to the simplification. Albeit relatively straightforward, this calculation can be time consuming. Here, we describe the implementation of a deep learning approach aimed at accelerating the calculation of the map** entropy. The method relies on deep graph networks, which provide extreme flexibility in the input format. We show that deep graph networks are accurate and remarkably efficient, with a speedup factor as large as $10^5$ with respect to the algorithmic computation of the map** entropy. Applications of this method, which entails a great potential in the study of biomolecules when used to reconstruct its map** entropy landscape, reach much farther than this, being the scheme easily transferable to the computation of arbitrary functions of a molecule's structure. △ Less

Submitted 14 July, 2020; originally announced July 2020.

Computation of the Transient in Max-Plus Linear Systems via SMT-Solving

Authors: Alessandro Abate, Alessandro Cimatti, Andrea Micheli, Muhammad Syifa'ul Mufid

Abstract: This paper proposes a new approach, grounded in Satisfiability Modulo Theories (SMT), to study the transient of a Max-Plus Linear (MPL) system, that is the number of steps leading to its periodic regime. Differently from state-of-the-art techniques, our approach allows the analysis of periodic behaviors for subsets of initial states, as well as the characterization of sets of initial states exhibi… ▽ More This paper proposes a new approach, grounded in Satisfiability Modulo Theories (SMT), to study the transient of a Max-Plus Linear (MPL) system, that is the number of steps leading to its periodic regime. Differently from state-of-the-art techniques, our approach allows the analysis of periodic behaviors for subsets of initial states, as well as the characterization of sets of initial states exhibiting the same specific periodic behavior and transient. Our experiments show that the proposed technique dramatically outperforms state-of-the-art methods based on max-plus algebra computations for systems of large dimensions. △ Less

Submitted 7 July, 2020; v1 submitted 1 July, 2020; originally announced July 2020.

Comments: The paper consists of 22 pages (including references and Appendix). It is accepted in FORMATS 2020 First revision

Evidence of Crowding on Russell 3000 Reconstitution Events

Authors: Alessandro Micheli, Eyal Neuman

Abstract: We develop a methodology which replicates in great accuracy the FTSE Russell indexes reconstitutions, including the quarterly rebalancings due to new initial public offerings (IPOs). While using only data available in the CRSP US Stock database for our index reconstruction, we demonstrate the accuracy of this methodology by comparing it to the original Russell US indexes for the time period betwee… ▽ More We develop a methodology which replicates in great accuracy the FTSE Russell indexes reconstitutions, including the quarterly rebalancings due to new initial public offerings (IPOs). While using only data available in the CRSP US Stock database for our index reconstruction, we demonstrate the accuracy of this methodology by comparing it to the original Russell US indexes for the time period between 1989 to 2019. A python package that generates the replicated indexes is also provided. As an application, we use our index reconstruction protocol to compute the permanent and temporary price impact on the Russell 3000 annual additions and deletions, and on the quarterly additions of new IPOs . We find that the index portfolios following the Russell 3000 index and rebalanced on an annual basis are overall more crowded than those following the index on a quarterly basis. This phenomenon implies that transaction costs of indexing strategies could be significantly reduced by buying new IPOs additions in proximity to quarterly rebalance dates. △ Less

Submitted 22 September, 2022; v1 submitted 12 June, 2020; originally announced June 2020.

Comments: 35 pages, 9 figures

Ring Reservoir Neural Networks for Graphs

Authors: Claudio Gallicchio, Alessio Micheli

Abstract: Machine Learning for graphs is nowadays a research topic of consolidated relevance. Common approaches in the field typically resort to complex deep neural network architectures and demanding training algorithms, highlighting the need for more efficient solutions. The class of Reservoir Computing (RC) models can play an important role in this context, enabling to develop fruitful graph embeddings t… ▽ More Machine Learning for graphs is nowadays a research topic of consolidated relevance. Common approaches in the field typically resort to complex deep neural network architectures and demanding training algorithms, highlighting the need for more efficient solutions. The class of Reservoir Computing (RC) models can play an important role in this context, enabling to develop fruitful graph embeddings through untrained recursive architectures. In this paper, we study progressive simplifications to the design strategy of RC neural networks for graphs. Our core proposal is based on sha** the organization of the hidden neurons to follow a ring topology. Experimental results on graph classification tasks indicate that ring-reservoirs architectures enable particularly effective network configurations, showing consistent advantages in terms of predictive performance. △ Less

Submitted 11 May, 2020; originally announced May 2020.

Comments: Accepted for IJCNN/WCCI 2020

Higgs self-coupling measurements using deep learning in the $b\bar{b}b\bar{b}$ final state

Authors: Jacob Amacker, William Balunas, Lydia Beresford, Daniela Bortoletto, James Frost, Cigdem Issever, Jesse Liu, James McKee, Alessandro Micheli, Santiago Paredes Saenz, Michael Spannowsky, Beojan Stanislaus

Abstract: Measuring the Higgs trilinear self-coupling $λ_{hhh}$ is experimentally demanding but fundamental for understanding the shape of the Higgs potential. We present a comprehensive analysis strategy for the HL-LHC using di-Higgs events in the four $b$-quark channel ($hh \to 4b$), extending current methods in several directions. We perform deep learning to suppress the formidable multijet background wi… ▽ More Measuring the Higgs trilinear self-coupling $λ_{hhh}$ is experimentally demanding but fundamental for understanding the shape of the Higgs potential. We present a comprehensive analysis strategy for the HL-LHC using di-Higgs events in the four $b$-quark channel ($hh \to 4b$), extending current methods in several directions. We perform deep learning to suppress the formidable multijet background with dedicated optimisation for BSM $λ_{hhh}$ scenarios. We compare the $λ_{hhh}$ constraining power of events using different multiplicities of large radius jets with a two-prong structure that reconstruct boosted $h \to bb$ decays. We show that current uncertainties in the SM top Yukawa coupling $y_t$ can modify $λ_{hhh}$ constraints by $\sim 20\%$. For SM $y_t$, we find prospects of $-0.8 < λ_{hhh} / λ_{hhh}^\text{SM} < 6.6$ at 68% CL under simplified assumptions for 3000~fb$^{-1}$ of HL-LHC data. Our results provide a careful assessment of di-Higgs identification and machine learning techniques for all-hadronic measurements of the Higgs self-coupling and sharpens the requirements for future improvement. △ Less

Submitted 12 October, 2020; v1 submitted 8 April, 2020; originally announced April 2020.

Comments: 36 pages, 15 figures + bibliography and appendices

Report number: IPPP/20/11

Journal ref: JHEP 12 (2020) 115

Robust Plan Execution with Unexpected Observations

Authors: Oscar Lima, Michael Cashmore, Daniele Magazzeni, Andrea Micheli, Rodrigo Ventura

Abstract: In order to ensure the robust actuation of a plan, execution must be adaptable to unexpected situations in the world and to exogenous events. This is critical in domains in which committing to a wrong ordering of actions can cause the plan failure, even when all the actions succeed. We propose an approach to the execution of a task plan that permits some adaptability to unexpected observations of… ▽ More In order to ensure the robust actuation of a plan, execution must be adaptable to unexpected situations in the world and to exogenous events. This is critical in domains in which committing to a wrong ordering of actions can cause the plan failure, even when all the actions succeed. We propose an approach to the execution of a task plan that permits some adaptability to unexpected observations of the state while maintaining the validity of the plan through online reasoning. Our approach computes an adaptable, partially-ordered plan from a given totally-ordered plan. The partially-ordered plan is adaptable in that it can exploit beneficial differences between the world and what was expected. The approach is general in that it can be used with any task planner that produces either a totally or a partially-ordered plan. We propose a plan execution algorithm that computes online the complete set of valid totally-ordered plans described by an adaptable partially-ordered plan together with the probability of success for each of them. This set is then used to choose the next action to execute. △ Less

Submitted 20 March, 2020; originally announced March 2020.

Comments: Preprint, not submitted to any conference yet

ACM Class: I.2.9

A Deep Generative Model for Fragment-Based Molecule Generation

Authors: Marco Podda, Davide Bacciu, Alessio Micheli

Abstract: Molecule generation is a challenging open problem in cheminformatics. Currently, deep generative approaches addressing the challenge belong to two broad categories, differing in how molecules are represented. One approach encodes molecular graphs as strings of text, and learns their corresponding character-based language model. Another, more expressive, approach operates directly on the molecular… ▽ More Molecule generation is a challenging open problem in cheminformatics. Currently, deep generative approaches addressing the challenge belong to two broad categories, differing in how molecules are represented. One approach encodes molecular graphs as strings of text, and learns their corresponding character-based language model. Another, more expressive, approach operates directly on the molecular graph. In this work, we address two limitations of the former: generation of invalid and duplicate molecules. To improve validity rates, we develop a language model for small molecular substructures called fragments, loosely inspired by the well-known paradigm of Fragment-Based Drug Design. In other words, we generate molecules fragment by fragment, instead of atom by atom. To improve uniqueness rates, we present a frequency-based masking strategy that helps generate molecules with infrequent fragments. We show experimentally that our model largely outperforms other language model-based competitors, reaching state-of-the-art performances typical of graph-based approaches. Moreover, generated molecules display molecular properties similar to those in the training sample, even in absence of explicit task-specific supervision. △ Less

Submitted 28 February, 2020; originally announced February 2020.

Journal ref: PMLR 108:2240-2250 (2020)

Machine learning approaches for identifying prey handling activity in otariid pinnipeds

Authors: Rita Pucci, Alessio Micheli, Stefano Chessa, Jane Hunter

Abstract: Systems developed in wearable devices with sensors onboard are widely used to collect data of humans and animals activities with the perspective of an on-board automatic classification of data. An interesting application of these systems is to support animals' behaviour monitoring gathered by sensors' data analysis. This is a challenging area and in particular with fixed memories capabilities beca… ▽ More Systems developed in wearable devices with sensors onboard are widely used to collect data of humans and animals activities with the perspective of an on-board automatic classification of data. An interesting application of these systems is to support animals' behaviour monitoring gathered by sensors' data analysis. This is a challenging area and in particular with fixed memories capabilities because the devices should be able to operate autonomously for long periods before being retrieved by human operators, and being able to classify activities onboard can significantly improve their autonomy. In this paper, we focus on the identification of prey handling activity in seals (when the animal start attaching and biting the prey), which is one of the main movement that identifies a successful foraging activity. Data taken into consideration are streams of 3D accelerometers and depth sensors values collected by devices attached directly on seals. To analyse these data, we propose an automatic model based on Machine Learning (ML) algorithms. In particular, we compare the performance (in terms of accuracy and F1score) of three ML algorithms: Input Delay Neural Networks, Support Vector Machines, and Echo State Networks. We attend to the final aim of develo** an automatic classifier on-board. For this purpose, in this paper, the comparison is performed concerning the performance obtained by each ML approach developed and its memory footprint. In the end, we highlight the advantage of using an ML algorithm, in terms of feasibility in wild animals' monitoring. △ Less

Submitted 10 February, 2020; originally announced February 2020.

Edge-based sequential graph generation with recurrent neural networks

Authors: Davide Bacciu, Alessio Micheli, Marco Podda

Abstract: Graph generation with Machine Learning is an open problem with applications in various research fields. In this work, we propose to cast the generative process of a graph into a sequential one, relying on a node ordering procedure. We use this sequential process to design a novel generative model composed of two recurrent neural networks that learn to predict the edges of graphs: the first network… ▽ More Graph generation with Machine Learning is an open problem with applications in various research fields. In this work, we propose to cast the generative process of a graph into a sequential one, relying on a node ordering procedure. We use this sequential process to design a novel generative model composed of two recurrent neural networks that learn to predict the edges of graphs: the first network generates one endpoint of each edge, while the second network generates the other endpoint conditioned on the state of the first. We test our approach extensively on five different datasets, comparing with two well-known baselines coming from graph literature, and two recurrent approaches, one of which holds state of the art performances. Evaluation is conducted considering quantitative and qualitative characteristics of the generated samples. Results show that our approach is able to yield novel, and unique graphs originating from very different distributions, while retaining structural properties very similar to those in the training sample. Under the proposed evaluation framework, our approach is able to reach performances comparable to the current state of the art on the graph generation task. △ Less

Submitted 31 January, 2020; originally announced February 2020.

Theoretically Expressive and Edge-aware Graph Learning

Authors: Federico Errica, Davide Bacciu, Alessio Micheli

Abstract: We propose a new Graph Neural Network that combines recent advancements in the field. We give theoretical contributions by proving that the model is strictly more general than the Graph Isomorphism Network and the Gated Graph Neural Network, as it can approximate the same functions and deal with arbitrary edge values. Then, we show how a single node information can flow through the graph unchanged… ▽ More We propose a new Graph Neural Network that combines recent advancements in the field. We give theoretical contributions by proving that the model is strictly more general than the Graph Isomorphism Network and the Gated Graph Neural Network, as it can approximate the same functions and deal with arbitrary edge values. Then, we show how a single node information can flow through the graph unchanged. △ Less

Submitted 24 January, 2020; originally announced January 2020.

A Gentle Introduction to Deep Learning for Graphs

Authors: Davide Bacciu, Federico Errica, Alessio Micheli, Marco Podda

Abstract: The adaptive processing of graph data is a long-standing research topic which has been lately consolidated as a theme of major interest in the deep learning community. The snap increase in the amount and breadth of related research has come at the price of little systematization of knowledge and attention to earlier literature. This work is designed as a tutorial introduction to the field of deep… ▽ More The adaptive processing of graph data is a long-standing research topic which has been lately consolidated as a theme of major interest in the deep learning community. The snap increase in the amount and breadth of related research has come at the price of little systematization of knowledge and attention to earlier literature. This work is designed as a tutorial introduction to the field of deep learning for graphs. It favours a consistent and progressive introduction of the main concepts and architectural aspects over an exposition of the most recent literature, for which the reader is referred to available surveys. The paper takes a top-down view to the problem, introducing a generalized formulation of graph representation learning based on a local and iterative approach to structured information processing. It introduces the basic building blocks that can be combined to design novel and effective neural models for graphs. The methodological exposition is complemented by a discussion of interesting research challenges and applications in the field. △ Less

Submitted 15 June, 2020; v1 submitted 29 December, 2019; originally announced December 2019.

A Fair Comparison of Graph Neural Networks for Graph Classification

Authors: Federico Errica, Marco Podda, Davide Bacciu, Alessio Micheli

Abstract: Experimental reproducibility and replicability are critical topics in machine learning. Authors have often raised concerns about their lack in scientific publications to improve the quality of the field. Recently, the graph representation learning field has attracted the attention of a wide research community, which resulted in a large stream of works. As such, several Graph Neural Network models… ▽ More Experimental reproducibility and replicability are critical topics in machine learning. Authors have often raised concerns about their lack in scientific publications to improve the quality of the field. Recently, the graph representation learning field has attracted the attention of a wide research community, which resulted in a large stream of works. As such, several Graph Neural Network models have been developed to effectively tackle graph classification. However, experimental procedures often lack rigorousness and are hardly reproducible. Motivated by this, we provide an overview of common practices that should be avoided to fairly compare with the state of the art. To counter this troubling trend, we ran more than 47000 experiments in a controlled and uniform framework to re-evaluate five popular models across nine common benchmarks. Moreover, by comparing GNNs with structure-agnostic baselines we provide convincing evidence that, on some datasets, structural information has not been exploited yet. We believe that this work can contribute to the development of the graph learning field, by providing a much needed grounding for rigorous evaluations of graph classification models. △ Less

Submitted 17 February, 2022; v1 submitted 20 December, 2019; originally announced December 2019.

Comments: Extended version of the paper published at the International Conference on Learning Representations (ICLR), 2020. Additional results are shown in the appendix

Regions of the type C Catalan arrangement

Authors: Anne Micheli, Vu Nguyen Dinh

Abstract: In this paper, we give a bijection between rooted labeled ordered forests with a selected subset of their leaves and the regions of the type $C$ Catalan arrangement in $\R^n$. We thus obtain a bijective proof of the well-known enumeration formula of these regions ${2^n}n! \binom{2n}{n}$. In this paper, we give a bijection between rooted labeled ordered forests with a selected subset of their leaves and the regions of the type $C$ Catalan arrangement in $\R^n$. We thus obtain a bijective proof of the well-known enumeration formula of these regions ${2^n}n! \binom{2n}{n}$. △ Less

Submitted 20 April, 2020; v1 submitted 20 December, 2019; originally announced December 2019.

Comments: 10 pages, 3 figures

Fast and Deep Graph Neural Networks

Authors: Claudio Gallicchio, Alessio Micheli

Abstract: We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse net… ▽ More We address the efficiency issue for the construction of a deep graph neural network (GNN). The approach exploits the idea of representing each input graph as a fixed point of a dynamical system (implemented through a recurrent neural network), and leverages a deep architectural organization of the recurrent units. Efficiency is gained by many aspects, including the use of small and very sparse networks, where the weights of the recurrent units are left untrained under the stability condition introduced in this work. This can be viewed as a way to study the intrinsic power of the architecture of a deep GNN, and also to provide insights for the set-up of more complex fully-trained models. Through experimental results, we show that even without training of the recurrent connections, the architecture of small deep GNN is surprisingly able to achieve or improve the state-of-the-art performance on a significant set of tasks in the field of graphs classification. △ Less

Submitted 20 November, 2019; originally announced November 2019.

Comments: Pre-print of 'Fast and Deep Graph Neural Networks', accepted for AAAI 2020. This document includes the Supplementary Material

Towards Efficient Anytime Computation and Execution of Decoupled Robustness Envelopes for Temporal Plans

Authors: Michael Cashmore, Alessandro Cimatti, Daniele Magazzeni, Andrea Micheli, Parisa Zehtabi

Abstract: One of the major limitations for the employment of model-based planning and scheduling in practical applications is the need of costly re-planning when an incongruence between the observed reality and the formal model is encountered during execution. Robustness Envelopes characterize the set of possible contingencies that a plan is able to address without re-planning, but their exact computation i… ▽ More One of the major limitations for the employment of model-based planning and scheduling in practical applications is the need of costly re-planning when an incongruence between the observed reality and the formal model is encountered during execution. Robustness Envelopes characterize the set of possible contingencies that a plan is able to address without re-planning, but their exact computation is extremely expensive; furthermore, general robustness envelopes are not amenable for efficient execution. In this paper, we present a novel, anytime algorithm to approximate Robustness Envelopes, making them scalable and executable. This is proven by an experimental analysis showing the efficiency of the algorithm, and by a concrete case study where the execution of robustness envelopes significantly reduces the number of re-plannings. △ Less

Submitted 17 November, 2019; originally announced November 2019.

Comments: 8 pages, 5 figures

Temporal Planning with Intermediate Conditions and Effects

Authors: Alessandro Valentini, Andrea Micheli, Alessandro Cimatti

Abstract: Automated temporal planning is the technology of choice when controlling systems that can execute more actions in parallel and when temporal constraints, such as deadlines, are needed in the model. One limitation of several action-based planning systems is that actions are modeled as intervals having conditions and effects only at the extremes and as invariants, but no conditions nor effects can b… ▽ More Automated temporal planning is the technology of choice when controlling systems that can execute more actions in parallel and when temporal constraints, such as deadlines, are needed in the model. One limitation of several action-based planning systems is that actions are modeled as intervals having conditions and effects only at the extremes and as invariants, but no conditions nor effects can be specified at arbitrary points or sub-intervals. In this paper, we address this limitation by providing an effective heuristic-search technique for temporal planning, allowing the definition of actions with conditions and effects at any arbitrary time within the action duration. We experimentally demonstrate that our approach is far better than standard encodings in PDDL 2.1 and is competitive with other approaches that can (directly or indirectly) represent intermediate action conditions or effects. △ Less

Submitted 25 September, 2019; originally announced September 2019.

Reservoir Topology in Deep Echo State Networks

Authors: Claudio Gallicchio, Alessio Micheli

Abstract: Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms… ▽ More Deep Echo State Networks (DeepESNs) recently extended the applicability of Reservoir Computing (RC) methods towards the field of deep learning. In this paper we study the impact of constrained reservoir topologies in the architectural design of deep reservoirs, through numerical experiments on several RC benchmarks. The major outcome of our investigation is to show the remarkable effect, in terms of predictive performance gain, achieved by the synergy between a deep reservoir construction and a structured organization of the recurrent units in each layer. Our results also indicate that a particularly advantageous architectural setting is obtained in correspondence of DeepESNs where reservoir units are structured according to a permutation recurrent matrix. △ Less

Submitted 24 September, 2019; originally announced September 2019.

Comments: Preprint of the paper published in the proceedings of ICANN 2019

Embeddings and Representation Learning for Structured Data

Authors: Benjamin Paaßen, Claudio Gallicchio, Alessio Micheli, Alessandro Sperduti

Abstract: Performing machine learning on structured data is complicated by the fact that such data does not have vectorial form. Therefore, multiple approaches have emerged to construct vectorial representations of structured data, from kernel and distance approaches to recurrent, recursive, and convolutional neural networks. Recent years have seen heightened attention in this demanding field of research an… ▽ More Performing machine learning on structured data is complicated by the fact that such data does not have vectorial form. Therefore, multiple approaches have emerged to construct vectorial representations of structured data, from kernel and distance approaches to recurrent, recursive, and convolutional neural networks. Recent years have seen heightened attention in this demanding field of research and several new approaches have emerged, such as metric learning on structured data, graph convolutional neural networks, and recurrent decoder networks for structured data. In this contribution, we provide an high-level overview of the state-of-the-art in representation learning and embeddings for structured data across a wide range of machine learning fields. △ Less

Submitted 15 May, 2019; originally announced May 2019.

Comments: Oral presentation at the 27th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2019) in Bruges, Belgium, on April 24th, 2019

Journal ref: Proc. ESANN (2019), 85-94

Richness of Deep Echo State Network Dynamics

Authors: Claudio Gallicchio, Alessio Micheli

Abstract: Reservoir Computing (RC) is a popular methodology for the efficient design of Recurrent Neural Networks (RNNs). Recently, the advantages of the RC approach have been extended to the context of multi-layered RNNs, with the introduction of the Deep Echo State Network (DeepESN) model. In this paper, we study the quality of state dynamics in progressively higher layers of DeepESNs, using tools from th… ▽ More Reservoir Computing (RC) is a popular methodology for the efficient design of Recurrent Neural Networks (RNNs). Recently, the advantages of the RC approach have been extended to the context of multi-layered RNNs, with the introduction of the Deep Echo State Network (DeepESN) model. In this paper, we study the quality of state dynamics in progressively higher layers of DeepESNs, using tools from the areas of information theory and numerical analysis. Our experimental results on RC benchmark datasets reveal the fundamental role played by the strength of inter-reservoir connections to increasingly enrich the representations developed in higher layers. Our analysis also gives interesting insights into the possibility of effective exploitation of training algorithms based on stochastic gradient descent in the RC field. △ Less

Submitted 24 September, 2019; v1 submitted 12 March, 2019; originally announced March 2019.

Comments: Preprint of the paper accepted at IWANN 2019

Comparison between DeepESNs and gated RNNs on multivariate time-series prediction

Authors: Claudio Gallicchio, Alessio Micheli, Luca Pedrelli

Abstract: We propose an experimental comparison between Deep Echo State Networks (DeepESNs) and gated Recurrent Neural Networks (RNNs) on multivariate time-series prediction tasks. In particular, we compare reservoir and fully-trained RNNs able to represent signals featured by multiple time-scales dynamics. The analysis is performed in terms of efficiency and prediction accuracy on 4 polyphonic music tasks.… ▽ More We propose an experimental comparison between Deep Echo State Networks (DeepESNs) and gated Recurrent Neural Networks (RNNs) on multivariate time-series prediction tasks. In particular, we compare reservoir and fully-trained RNNs able to represent signals featured by multiple time-scales dynamics. The analysis is performed in terms of efficiency and prediction accuracy on 4 polyphonic music tasks. Our results show that DeepESN is able to outperform ESN in terms of prediction accuracy and efficiency. Whereas, between fully-trained approaches, Gated Recurrent Units (GRU) outperforms Long Short-Term Memory (LSTM) and simple RNN models in most cases. Overall, DeepESN turned out to be extremely more efficient than others RNN approaches and the best solution in terms of prediction accuracy on 3 out of 4 tasks. △ Less

Submitted 20 November, 2019; v1 submitted 30 December, 2018; originally announced December 2018.

Comments: Preprint version of Claudio Gallicchio, Alessio Micheli and Luca Pedrelli (2019) Comparison between DeepESNs and gated RNNs on multivariate time-series prediction. In: ESANN 2019 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 24-26 April 2019, i6doc.com publ., ISBN 978-287-587-065-0

Tree Edit Distance Learning via Adaptive Symbol Embeddings

Authors: Benjamin Paaßen, Claudio Gallicchio, Alessio Micheli, Barbara Hammer

Abstract: Metric learning has the aim to improve classification accuracy by learning a distance measure which brings data points from the same class closer together and pushes data points from different classes further apart. Recent research has demonstrated that metric learning approaches can also be applied to trees, such as molecular structures, abstract syntax trees of computer programs, or syntax trees… ▽ More Metric learning has the aim to improve classification accuracy by learning a distance measure which brings data points from the same class closer together and pushes data points from different classes further apart. Recent research has demonstrated that metric learning approaches can also be applied to trees, such as molecular structures, abstract syntax trees of computer programs, or syntax trees of natural language, by learning the cost function of an edit distance, i.e. the costs of replacing, deleting, or inserting nodes in a tree. However, learning such costs directly may yield an edit distance which violates metric axioms, is challenging to interpret, and may not generalize well. In this contribution, we propose a novel metric learning approach for trees which we call embedding edit distance learning (BEDL) and which learns an edit distance indirectly by embedding the tree nodes as vectors, such that the Euclidean distance between those vectors supports class discrimination. We learn such embeddings by reducing the distance to prototypical trees from the same class and increasing the distance to prototypical trees from different classes. In our experiments, we show that BEDL improves upon the state-of-the-art in metric learning for trees on six benchmark data sets, ranging from computer science over biomedical data to a natural-language processing data set containing over 300,000 nodes. △ Less

Submitted 16 July, 2018; v1 submitted 13 June, 2018; originally announced June 2018.

Comments: Paper at the International Conference of Machine Learning (2018), 2018-07-10 to 2018-07-15 in Stockholm, Sweden

Journal ref: Proceedings of Machine Learning Research 80 (2018) 3973-3982

Contextual Graph Markov Model: A Deep and Generative Approach to Graph Processing

Authors: Davide Bacciu, Federico Errica, Alessio Micheli

Abstract: We introduce the Contextual Graph Markov Model, an approach combining ideas from generative models and neural networks for the processing of graph data. It founds on a constructive methodology to build a deep architecture comprising layers of probabilistic models that learn to encode the structured information in an incremental fashion. Context is diffused in an efficient and scalable way across t… ▽ More We introduce the Contextual Graph Markov Model, an approach combining ideas from generative models and neural networks for the processing of graph data. It founds on a constructive methodology to build a deep architecture comprising layers of probabilistic models that learn to encode the structured information in an incremental fashion. Context is diffused in an efficient and scalable way across the graph vertexes and edges. The resulting graph encoding is used in combination with discriminative models to address structure classification benchmarks. △ Less

Submitted 25 November, 2019; v1 submitted 27 May, 2018; originally announced May 2018.

Journal ref: Proceedings of the 35th International Conference on Machine Learning, PMLR 80 (2018) 294-303

Deep Echo State Networks for Diagnosis of Parkinson's Disease

Authors: Claudio Gallicchio, Alessio Micheli, Luca Pedrelli

Abstract: In this paper, we introduce a novel approach for diagnosis of Parkinson's Disease (PD) based on deep Echo State Networks (ESNs). The identification of PD is performed by analyzing the whole time-series collected from a tablet device during the sketching of spiral tests, without the need for feature extraction and data preprocessing. We evaluated the proposed approach on a public dataset of spiral… ▽ More In this paper, we introduce a novel approach for diagnosis of Parkinson's Disease (PD) based on deep Echo State Networks (ESNs). The identification of PD is performed by analyzing the whole time-series collected from a tablet device during the sketching of spiral tests, without the need for feature extraction and data preprocessing. We evaluated the proposed approach on a public dataset of spiral tests. The results of experimental analysis show that DeepESNs perform significantly better than shallow ESN model. Overall, the proposed approach obtains state-of-the-art results in the identification of PD on this kind of temporal data. △ Less

Submitted 19 February, 2018; originally announced February 2018.

Comments: This is a pre-print of the paper submitted to the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2018

Deep Echo State Network (DeepESN): A Brief Survey

Authors: Claudio Gallicchio, Alessio Micheli

Abstract: The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of DeepESNs allowed to… ▽ More The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of DeepESNs allowed to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers, i.e. on the bias of depth in RNNs architectural design. In this paper, we summarize the advancements in the development, analysis and applications of DeepESNs. △ Less

Submitted 25 September, 2020; v1 submitted 12 December, 2017; originally announced December 2017.

Hierarchical Temporal Representation in Linear Reservoir Computing

Authors: Claudio Gallicchio, Alessio Micheli, Luca Pedrelli

Abstract: Recently, studies on deep Reservoir Computing (RC) highlighted the role of layering in deep recurrent neural networks (RNNs). In this paper, the use of linear recurrent units allows us to bring more evidence on the intrinsic hierarchical temporal representation in deep RNNs through frequency analysis applied to the state signals. The potentiality of our approach is assessed on the class of Multipl… ▽ More Recently, studies on deep Reservoir Computing (RC) highlighted the role of layering in deep recurrent neural networks (RNNs). In this paper, the use of linear recurrent units allows us to bring more evidence on the intrinsic hierarchical temporal representation in deep RNNs through frequency analysis applied to the state signals. The potentiality of our approach is assessed on the class of Multiple Superimposed Oscillator tasks. Furthermore, our investigation provides useful insights to open a discussion on the main aspects that characterize the deep learning framework in the temporal domain. △ Less

Submitted 10 July, 2017; v1 submitted 16 May, 2017; originally announced May 2017.

Comments: This is a pre-print of the paper submitted to the 27th Italian Workshop on Neural Networks, WIRN 2017

The xSAP Safety Analysis Platform

Authors: Benjamin Bittner, Marco Bozzano, Roberto Cavada, Alessandro Cimatti, Marco Gario, Alberto Griggio, Cristian Mattarei, Andrea Micheli, Gianni Zampedri

Abstract: This paper describes the xSAP safety analysis platform. xSAP provides several model-based safety analysis features for finite- and infinite-state synchronous transition systems. In particular, it supports library-based definition of fault modes, an automatic model extension facility, generation of safety analysis artifacts such as Dynamic Fault Trees (DFTs) and Failure Mode and Effects Analysis (F… ▽ More This paper describes the xSAP safety analysis platform. xSAP provides several model-based safety analysis features for finite- and infinite-state synchronous transition systems. In particular, it supports library-based definition of fault modes, an automatic model extension facility, generation of safety analysis artifacts such as Dynamic Fault Trees (DFTs) and Failure Mode and Effects Analysis (FMEA) tables. Moreover, it supports probabilistic evaluation of Fault Trees, failure propagation analysis using Timed Failure Propagation Graphs (TFPGs), and Common Cause Analysis (CCA). xSAP has been used in several industrial projects as verification back-end, and is currently being evaluated in a joint R&D Project involving FBK and The Boeing Company. △ Less

Submitted 29 April, 2015; v1 submitted 28 April, 2015; originally announced April 2015.

Bilayer superfluidity of fermionic polar molecules: many body effects

Authors: M. A. Baranov, A. Micheli, S. Ronen, P. Zoller

Abstract: We study the BCS superfluid transition in a single-component fermionic gas of dipolar particles loaded in a tight bilayer trap, with the electric dipole moments polarized perpendicular to the layers. Based on the detailed analysis of the interlayer scattering, we calculate the critical temperature of the interlayer superfluid pairing transition when the layer separation is both smaller (dilute reg… ▽ More We study the BCS superfluid transition in a single-component fermionic gas of dipolar particles loaded in a tight bilayer trap, with the electric dipole moments polarized perpendicular to the layers. Based on the detailed analysis of the interlayer scattering, we calculate the critical temperature of the interlayer superfluid pairing transition when the layer separation is both smaller (dilute regime) and of the order or larger (dense regime) than the mean interparticle separation in each layer. Our calculations go beyond the standard BCS approach and include the many-body contributions resulting in the mass renormalization, as well as additional contributions to the pairing interaction. We find that the many-body effects have a pronounced effect on the critical temperature, and can either decrease (in the very dilute limit) or increase (in the dense and moderately dilute limits) the transition temperature as compared to the BCS approach. △ Less

Submitted 27 December, 2010; originally announced December 2010.

Comments: 23 pages, 10 figures, final approval from S. Ronen was not received due to his no-response

A superfluid-droplet crystal and a free-space supersolid in a dipole-blockaded gas

Authors: F. Cinti, P. Jain, M. Boninsegni, A. Micheli, P. Zoller, G. Pupillo

Abstract: A novel supersolid phase is predicted for an ensemble of Rydberg atoms in the dipole-blockade regime, interacting via a repulsive dipolar potential "softened" at short distances. Using exact numerical techniques, we study the low temperature phase diagram of this system, and observe an intriguing phase consisting of a crystal of mesoscopic superfluid droplets. At low temperature, phase coherence t… ▽ More A novel supersolid phase is predicted for an ensemble of Rydberg atoms in the dipole-blockade regime, interacting via a repulsive dipolar potential "softened" at short distances. Using exact numerical techniques, we study the low temperature phase diagram of this system, and observe an intriguing phase consisting of a crystal of mesoscopic superfluid droplets. At low temperature, phase coherence throughout the whole system, and the ensuing bulk superfluidity, are established through tunnelling of identical particles between neighbouring droplets. △ Less

Submitted 21 September, 2010; v1 submitted 13 May, 2010; originally announced May 2010.

Comments: 4 pages, 4 figures

Journal ref: DOI: Phys. Rev. Lett. 105, 135301 (2010)

Universal rates for reactive ultracold polar molecules in reduced dimensions

Authors: Andrea Micheli, Zbigniew Idziaszek, Guido Pupillo, Mikhail A. Baranov, Peter Zoller, Paul S. Julienne

Abstract: Analytic expressions describe universal elastic and reactive rates of quasi-two-dimensional and quasi-one-dimensional collisions of highly reactive ultracold molecules interacting by a van der Waals potential. Exact and approximate calculations for the example species of KRb show that stability and evaporative cooling can be realized for spin-polarized fermions at moderate dipole and trap** str… ▽ More Analytic expressions describe universal elastic and reactive rates of quasi-two-dimensional and quasi-one-dimensional collisions of highly reactive ultracold molecules interacting by a van der Waals potential. Exact and approximate calculations for the example species of KRb show that stability and evaporative cooling can be realized for spin-polarized fermions at moderate dipole and trap** strength, whereas bosons or unlike fermions require significantly higher dipole or trap** strengths. △ Less

Submitted 29 April, 2010; originally announced April 2010.

Comments: 4 pages, 3 figures

Journal ref: Phys. Rev. Lett. 105, 073202 (2010)

Dynamical crystal creation with polar molecules or Rydberg atoms in optical lattices

Authors: J. Schachenmayer, I. Lesanovsky, A. Micheli, A. J. Daley

Abstract: We investigate the dynamical formation of crystalline states with systems of polar molecules or Rydberg atoms loaded into a deep optical lattice. External fields in these systems can be used to couple the atoms or molecules between two internal states: one that is weakly interacting and one that exhibits a strong dipole-dipole interaction. By appropriate time variation of the external fields, we s… ▽ More We investigate the dynamical formation of crystalline states with systems of polar molecules or Rydberg atoms loaded into a deep optical lattice. External fields in these systems can be used to couple the atoms or molecules between two internal states: one that is weakly interacting and one that exhibits a strong dipole-dipole interaction. By appropriate time variation of the external fields, we show that it is possible to produce crystalline states of the strongly interacting states with high filling fractions chosen via the parameters of the coupling. We study the coherent dynamics of this process in one dimension (1D) using a modified form of the time-evolving block decimation (TEBD) algorithm, and obtain crystalline states for system sizes and parameters corresponding to realistic experimental configurations. For polar molecules these crystalline states will be long-lived, assisting in a characterization of the state via the measurement of correlation functions. We also show that as the coupling strength increases in the model, the crystalline order is broken. This is characterized in 1D by a change in density-density correlation functions, which decay to a constant in the crystalline regime, but show different regions of exponential and algebraic decay for larger coupling strengths. △ Less

Submitted 17 January, 2011; v1 submitted 30 March, 2010; originally announced March 2010.

Comments: 15 pages, 13 figures

Journal ref: New J. Phys. 12 103044 (2010)