Showing 1–7 of 7 results for author: Kisil, I

HOTTBOX: Higher Order Tensor ToolBOX

Authors: Ilya Kisil, Giuseppe G. Calvi, Bruno S. Dees, Danilo P. Mandic

Abstract: HOTTBOX is a Python library for exploratory analysis and visualisation of multi-dimensional arrays of data, also known as tensors. The library includes methods ranging from standard multi-way operations and data manipulation through to multi-linear algebra based tensor decompositions. HOTTBOX also comprises sophisticated algorithms for generalised multi-linear classification and data fusion, such… ▽ More HOTTBOX is a Python library for exploratory analysis and visualisation of multi-dimensional arrays of data, also known as tensors. The library includes methods ranging from standard multi-way operations and data manipulation through to multi-linear algebra based tensor decompositions. HOTTBOX also comprises sophisticated algorithms for generalised multi-linear classification and data fusion, such as Support Tensor Machine (STM) and Tensor Ensemble Learning (TEL). For user convenience, HOTTBOX offers a unifying API which establishes a self-sufficient ecosystem for various forms of efficient representation of multi-way data and the corresponding decomposition and association algorithms. Particular emphasis is placed on scalability and interactive visualisation, to support multidisciplinary data analysis communities working on big data and tensors. HOTTBOX also provides means for integration with other popular data science libraries for visualisation and data manipulation. The source code, examples and documentation ca be found at https://github.com/hottbox/hottbox. △ Less

Submitted 30 November, 2021; originally announced November 2021.

Reducing Computational Complexity of Tensor Contractions via Tensor-Train Networks

Authors: Ilya Kisil, Giuseppe G. Calvi, Kriton Konstantinidis, Yao Lei Xu, Danilo P. Mandic

Abstract: There is a significant expansion in both volume and range of applications along with the concomitant increase in the variety of data sources. These ever-expanding trends have highlighted the necessity for more versatile analysis tools that offer greater opportunities for algorithmic developments and computationally faster operations than the standard flat-view matrix approach. Tensors, or multi-wa… ▽ More There is a significant expansion in both volume and range of applications along with the concomitant increase in the variety of data sources. These ever-expanding trends have highlighted the necessity for more versatile analysis tools that offer greater opportunities for algorithmic developments and computationally faster operations than the standard flat-view matrix approach. Tensors, or multi-way arrays, provide such an algebraic framework which is naturally suited to data of such large volume, diversity, and veracity. Indeed, the associated tensor decompositions have demonstrated their potential in breaking the Curse of Dimensionality associated with traditional matrix methods, where a necessary exponential increase in data volume leads to adverse or even intractable consequences on computational complexity. A key tool underpinning multi-linear manipulation of tensors and tensor networks is the standard Tensor Contraction Product (TCP). However, depending on the dimensionality of the underlying tensors, the TCP also comes at the price of high computational complexity in tensor manipulation. In this work, we resort to diagrammatic tensor network manipulation to calculate such products in an efficient and computationally tractable manner, by making use of Tensor Train decomposition (TTD). This has rendered the underlying concepts easy to perceive, thereby enhancing intuition of the associated underlying operations, while preserving mathematical rigour. In addition to bypassing the cumbersome mathematical multi-linear expressions, the proposed Tensor Train Contraction Product model is shown to accelerate significantly the underlying computational operations, as it is independent of tensor order and linear in the tensor dimension, as opposed to performing the full computations through the standard approach (exponential in tensor order). △ Less

Submitted 8 September, 2021; v1 submitted 1 September, 2021; originally announced September 2021.

An Example-Driven Introduction to Data Analytics on Graphs

Authors: Ljubisa Stankovic, Danilo Mandic, Milos Dakovic, Ilya Kisil, Ervin Sejdic, Anthony G. Constantinides

Abstract: Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advant… ▽ More Graphs are irregular structures which naturally account for data integrity, however, traditional approaches have been established outside Signal Processing, and largely focus on analyzing the underlying graphs rather than signals on graphs. Given the rapidly increasing availability of multisensor and multinode measurements, likely recorded on irregular or ad-hoc grids, it would be extremely advantageous to analyze such structured data as graph signals and thus benefit from the ability of graphs to incorporate spatial awareness of the sensing locations, sensor importance, and local versus global sensor association. The aim of this lecture note is therefore to establish a common language between graph signals, defined on irregular signal domains, and some of the most fundamental paradigms in DSP, such as spectral analysis of multichannel signals, system transfer function, digital filter design, parameter estimation, and optimal filter design. This is achieved through a physically meaningful and intuitive real-world example of geographically distributed multisensor temperature estimation. A similar spatial multisensor arrangement is already widely used in Signal Processing curricula to introduce minimum variance estimators and Kalman filters \cite{HM}, and by adopting this framework we facilitate a seamless integration of graph theory into the curriculum of existing DSP courses. By bridging the gap between standard approaches and graph signal processing, we also show that standard methods can be thought of as special cases of their graph counterparts, evaluated on linear graphs. It is hoped that our approach would not only help to demystify graph theoretic approaches in education and research but it would also empower practitioners to explore a whole host of otherwise prohibitive modern applications. △ Less

Submitted 12 May, 2019; v1 submitted 26 March, 2019; originally announced March 2019.

Comments: 10 pages, 3 figures, 5 boxes

An Intuitive Derivation of the Coherence Index Relation in Compressive Sensing

Authors: Ljubisa Stankovic, Danilo Mandic, Milos Dakovic, Ilya Kisil

Abstract: The existence and uniqueness conditions are a prerequisite for reliable reconstruction of sparse signals from reduced sets of measurements within the Compressive Sensing (CS) paradigm. However, despite their underpinning role for practical applications, existing uniqueness relations are either computationally prohibitive to implement (Restricted Isometry Property), or involve mathematical tools th… ▽ More The existence and uniqueness conditions are a prerequisite for reliable reconstruction of sparse signals from reduced sets of measurements within the Compressive Sensing (CS) paradigm. However, despite their underpinning role for practical applications, existing uniqueness relations are either computationally prohibitive to implement (Restricted Isometry Property), or involve mathematical tools that are beyond the standard background of engineering graduates (Coherence Index). This can introduce conceptual and computational obstacles in the development of engineering intuition, the design of suboptimal practical solutions, or understanding of limitations. To this end, we introduce a simple but rigorous derivation of the coherence index condition, based on standard linear algebra, with the aim to empower signal processing practitioners with intuition in the design and ease in implementation of CS systems. Given that the coherence index is one of very few CS metrics that admits mathematically tractable and computationally feasible calculation, it is our hope that this work will help bridge the gap between the theory and applications of compressive sensing. △ Less

Submitted 26 March, 2019; originally announced March 2019.

Comments: 8 pages, 3 figures, 3 boxes

Tensor Ensemble Learning for Multidimensional Data

Authors: Ilia Kisil, Ahmad Moniri, Danilo P. Mandic

Abstract: In big data applications, classical ensemble learning is typically infeasible on the raw input data and dimensionality reduction techniques are necessary. To this end, novel framework that generalises classic flat-view ensemble learning to multidimensional tensor-valued data is introduced. This is achieved by virtue of tensor decompositions, whereby the proposed method, referred to as tensor ensem… ▽ More In big data applications, classical ensemble learning is typically infeasible on the raw input data and dimensionality reduction techniques are necessary. To this end, novel framework that generalises classic flat-view ensemble learning to multidimensional tensor-valued data is introduced. This is achieved by virtue of tensor decompositions, whereby the proposed method, referred to as tensor ensemble learning (TEL), decomposes every input data sample into multiple factors which allows for a flexibility in the choice of multiple learning algorithms in order to improve test performance. The TEL framework is shown to naturally compress multidimensional data in order to take advantage of the inherent multi-way data structure and exploit the benefit of ensemble learning. The proposed framework is verified through the application of Higher Order Singular Value Decomposition (HOSVD) to the ETH-80 dataset and is shown to outperform the classical ensemble learning approach of bootstrap aggregating. △ Less

Submitted 17 December, 2018; originally announced December 2018.

The sum of tensor networks

Authors: Giuseppe G. Calvi, Ilia Kisil, Danilo P. Mandic

Abstract: Tensor networks (TNs) have been gaining interest as multiway data analysis tools owing to their ability to tackle the curse of dimensionality and to represent tensors as smaller-scale interconnections of their intrinsic features. However, despite the obvious advantages, the current treatment of TNs as stand-alone entities does not take full benefit of their underlying structure and the associated… ▽ More Tensor networks (TNs) have been gaining interest as multiway data analysis tools owing to their ability to tackle the curse of dimensionality and to represent tensors as smaller-scale interconnections of their intrinsic features. However, despite the obvious advantages, the current treatment of TNs as stand-alone entities does not take full benefit of their underlying structure and the associated feature localization. To this end, embarking upon the analogy with a feature fusion, we propose a rigorous framework for the combination of TNs, focusing on their summation as the natural way for their combination. This allows for feature combination for any number of tensors, as long as their TN representation topologies are isomorphic. The benefits of the proposed framework are demonstrated on the classification of several groups of partially related images, where it outperforms standard machine learning algorithms. △ Less

Submitted 2 November, 2017; originally announced November 2017.

Tensor Valued Common and Individual Feature Extraction: Multi-dimensional Perspective

Authors: Ilia Kisil, Giuseppe G. Calvi, Danilo P. Mandic

Abstract: A novel method for common and individual feature analysis from exceedingly large-scale data is proposed, in order to ensure the tractability of both the computation and storage and thus mitigate the curse of dimensionality, a major bottleneck in modern data science. This is achieved by making use of the inherent redundancy in so-called multi-block data structures, which represent multiple observat… ▽ More A novel method for common and individual feature analysis from exceedingly large-scale data is proposed, in order to ensure the tractability of both the computation and storage and thus mitigate the curse of dimensionality, a major bottleneck in modern data science. This is achieved by making use of the inherent redundancy in so-called multi-block data structures, which represent multiple observations of the same phenomenon taken at different times, angles or recording conditions. Upon providing an intrinsic link between the properties of the outer vector product and extracted features in tensor decompositions (TDs), the proposed common and individual information extraction from multi-block data is performed through imposing physical meaning to otherwise unconstrained factorisation approaches. This is shown to dramatically reduce the dimensionality of search spaces for subsequent classification procedures and to yield greatly enhanced accuracy. Simulations on a multi-class classification task of large-scale extraction of individual features from a collection of partially related real-world images demonstrate the advantages of the "blessing of dimensionality" associated with TDs. △ Less

Submitted 1 November, 2017; originally announced November 2017.