-
Algorithm-agnostic low-rank approximation of operator monotone matrix functions
Authors:
David Persson,
Raphael A. Meyer,
Christopher Musco
Abstract:
Low-rank approximation of a matrix function, $f(A)$, is an important task in computational mathematics. Most methods require direct access to $f(A)$, which is often considerably more expensive than accessing $A$. Persson and Kressner (SIMAX 2023) avoid this issue for symmetric positive semidefinite matrices by proposing funNyström, which first constructs a Nyström approximation to $A$ using subspa…
▽ More
Low-rank approximation of a matrix function, $f(A)$, is an important task in computational mathematics. Most methods require direct access to $f(A)$, which is often considerably more expensive than accessing $A$. Persson and Kressner (SIMAX 2023) avoid this issue for symmetric positive semidefinite matrices by proposing funNyström, which first constructs a Nyström approximation to $A$ using subspace iteration, and then uses the approximation to directly obtain a low-rank approximation for $f(A)$. They prove that the method yields a near-optimal approximation whenever $f$ is a continuous operator monotone function with $f(0) = 0$.
We significantly generalize the results of Persson and Kressner beyond subspace iteration. We show that if $\widehat{A}$ is a near-optimal low-rank Nyström approximation to $A$ then $f(\widehat{A})$ is a near-optimal low-rank approximation to $f(A)$, independently of how $\widehat{A}$ is computed. Further, we show sufficient conditions for a basis $Q$ to produce a near-optimal Nyström approximation $\widehat{A} = AQ(Q^T AQ)^{\dagger} Q^T A$. We use these results to establish that many common low-rank approximation methods produce near-optimal Nyström approximations to $A$ and therefore to $f(A)$.
△ Less
Submitted 23 November, 2023;
originally announced November 2023.
-
HEAL-SWIN: A Vision Transformer On The Sphere
Authors:
Oscar Carlsson,
Jan E. Gerken,
Hampus Linander,
Heiner Spieß,
Fredrik Ohlsson,
Christoffer Petersson,
Daniel Persson
Abstract:
High-resolution wide-angle fisheye images are becoming more and more important for robotics applications such as autonomous driving. However, using ordinary convolutional neural networks or vision transformers on this data is problematic due to projection and distortion losses introduced when projecting to a rectangular grid on the plane. We introduce the HEAL-SWIN transformer, which combines the…
▽ More
High-resolution wide-angle fisheye images are becoming more and more important for robotics applications such as autonomous driving. However, using ordinary convolutional neural networks or vision transformers on this data is problematic due to projection and distortion losses introduced when projecting to a rectangular grid on the plane. We introduce the HEAL-SWIN transformer, which combines the highly uniform Hierarchical Equal Area iso-Latitude Pixelation (HEALPix) grid used in astrophysics and cosmology with the Hierarchical Shifted-Window (SWIN) transformer to yield an efficient and flexible model capable of training on high-resolution, distortion-free spherical data. In HEAL-SWIN, the nested structure of the HEALPix grid is used to perform the patching and windowing operations of the SWIN transformer, enabling the network to process spherical representations with minimal computational overhead. We demonstrate the superior performance of our model on both synthetic and real automotive datasets, as well as a selection of other image datasets, for semantic segmentation, depth regression and classification tasks. Our code is publicly available at https://github.com/JanEGerken/HEAL-SWIN.
△ Less
Submitted 8 May, 2024; v1 submitted 14 July, 2023;
originally announced July 2023.
-
Large Scale Passenger Detection with Smartphone/Bus Implicit Interaction and Multisensory Unsupervised Cause-effect Learning
Authors:
Valentino Servizi,
Dan R. Persson,
Francisco C. Pereira,
Hannah Villadsen,
Per Bækgaard,
Jeppe Rich,
Otto A. Nielsen
Abstract:
Intelligent Transportation Systems (ITS) underpin the concept of Mobility as a Service (MaaS), which requires universal and seamless users' access across multiple public and private transportation systems while allowing operators' proportional revenue sharing. Current user sensing technologies such as Walk-in/Walk-out (WIWO) and Check-in/Check-out (CICO) have limited scalability for large-scale de…
▽ More
Intelligent Transportation Systems (ITS) underpin the concept of Mobility as a Service (MaaS), which requires universal and seamless users' access across multiple public and private transportation systems while allowing operators' proportional revenue sharing. Current user sensing technologies such as Walk-in/Walk-out (WIWO) and Check-in/Check-out (CICO) have limited scalability for large-scale deployments. These limitations prevent ITS from supporting analysis, optimization, calculation of revenue sharing, and control of MaaS comfort, safety, and efficiency. We focus on the concept of implicit Be-in/Be-out (BIBO) smartphone-sensing and classification.
To close the gap and enhance smartphones towards MaaS, we developed a proprietary smartphone-sensing platform collecting contemporary Bluetooth Low Energy (BLE) signals from BLE devices installed on buses and Global Positioning System (GPS) locations of both buses and smartphones. To enable the training of a model based on GPS features against the BLE pseudo-label, we propose the Cause-Effect Multitask Wasserstein Autoencoder (CEMWA). CEMWA combines and extends several frameworks around Wasserstein autoencoders and neural networks. As a dimensionality reduction tool, CEMWA obtains an auto-validated representation of a latent space describing users' smartphones within the transport system. This representation allows BIBO clustering via DBSCAN.
We perform an ablation study of CEMWA's alternative architectures and benchmark against the best available supervised methods. We analyze performance's sensitivity to label quality. Under the naïve assumption of accurate ground truth, XGBoost outperforms CEMWA. Although XGBoost and Random Forest prove to be tolerant to label noise, CEMWA is agnostic to label noise by design and provides the best performance with an 88\% F1 score.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
"Is not the truth the truth?": Analyzing the Impact of User Validations for Bus In/Out Detection in Smartphone-based Surveys
Authors:
Valentino Servizi.,
Dan R. Persson,
Francisco C. Pereira,
Hannah Villadsen,
Per Bækgaard,
Inon Peled,
Otto A. Nielsen
Abstract:
Passenger flow allows the study of users' behavior through the public network and assists in designing new facilities and services. This flow is observed through interactions between passengers and infrastructure. For this task, Bluetooth technology and smartphones represent the ideal solution. The latter component allows users' identification, authentication, and billing, while the former allows…
▽ More
Passenger flow allows the study of users' behavior through the public network and assists in designing new facilities and services. This flow is observed through interactions between passengers and infrastructure. For this task, Bluetooth technology and smartphones represent the ideal solution. The latter component allows users' identification, authentication, and billing, while the former allows short-range implicit interactions, device-to-device. To assess the potential of such a use case, we need to verify how robust Bluetooth signal and related machine learning (ML) classifiers are against the noise of realistic contexts. Therefore, we model binary passenger states with respect to a public vehicle, where one can either be-in or be-out (BIBO). The BIBO label identifies a fundamental building block of continuously-valued passenger flow. This paper describes the Human-Computer interaction experimental setting in a semi-controlled environment, which involves: two autonomous vehicles operating on two routes, serving three bus stops and eighteen users, as well as a proprietary smartphone-Bluetooth sensing platform. The resulting dataset includes multiple sensors' measurements of the same event and two ground-truth levels, the first being validation by participants, the second by three video-cameras surveilling buses and track. We performed a Monte-Carlo simulation of labels-flip to emulate human errors in the labeling process, as is known to happen in smartphone surveys; next we used such flipped labels for supervised training of ML classifiers. The impact of errors on model performance bias can be large. Results show ML tolerance to label flips caused by human or machine errors up to 30%.
△ Less
Submitted 24 February, 2022;
originally announced February 2022.
-
Equivariance versus Augmentation for Spherical Images
Authors:
Jan E. Gerken,
Oscar Carlsson,
Hampus Linander,
Fredrik Ohlsson,
Christoffer Petersson,
Daniel Persson
Abstract:
We analyze the role of rotational equivariance in convolutional neural networks (CNNs) applied to spherical images. We compare the performance of the group equivariant networks known as S2CNNs and standard non-equivariant CNNs trained with an increasing amount of data augmentation. The chosen architectures can be considered baseline references for the respective design paradigms. Our models are tr…
▽ More
We analyze the role of rotational equivariance in convolutional neural networks (CNNs) applied to spherical images. We compare the performance of the group equivariant networks known as S2CNNs and standard non-equivariant CNNs trained with an increasing amount of data augmentation. The chosen architectures can be considered baseline references for the respective design paradigms. Our models are trained and evaluated on single or multiple items from the MNIST or FashionMNIST dataset projected onto the sphere. For the task of image classification, which is inherently rotationally invariant, we find that by considerably increasing the amount of data augmentation and the size of the networks, it is possible for the standard CNNs to reach at least the same performance as the equivariant network. In contrast, for the inherently equivariant task of semantic segmentation, the non-equivariant networks are consistently outperformed by the equivariant networks with significantly fewer parameters. We also analyze and compare the inference latency and training times of the different networks, enabling detailed tradeoff considerations between equivariant architectures and data augmentation for practical problems. The equivariant spherical networks used in the experiments are available at https://github.com/JanEGerken/sem_seg_s2cnn .
△ Less
Submitted 12 July, 2022; v1 submitted 8 February, 2022;
originally announced February 2022.
-
Geometric Deep Learning and Equivariant Neural Networks
Authors:
Jan E. Gerken,
Jimmy Aronsson,
Oscar Carlsson,
Hampus Linander,
Fredrik Ohlsson,
Christoffer Petersson,
Daniel Persson
Abstract:
We survey the mathematical foundations of geometric deep learning, focusing on group equivariant and gauge equivariant neural networks. We develop gauge equivariant convolutional neural networks on arbitrary manifolds $\mathcal{M}$ using principal bundles with structure group $K$ and equivariant maps between sections of associated vector bundles. We also discuss group equivariant neural networks f…
▽ More
We survey the mathematical foundations of geometric deep learning, focusing on group equivariant and gauge equivariant neural networks. We develop gauge equivariant convolutional neural networks on arbitrary manifolds $\mathcal{M}$ using principal bundles with structure group $K$ and equivariant maps between sections of associated vector bundles. We also discuss group equivariant neural networks for homogeneous spaces $\mathcal{M}=G/K$, which are instead equivariant with respect to the global symmetry $G$ on $\mathcal{M}$. Group equivariant layers can be interpreted as intertwiners between induced representations of $G$, and we show their relation to gauge equivariant convolutional layers. We analyze several applications of this formalism, including semantic segmentation and object detection networks. We also discuss the case of spherical networks in great detail, corresponding to the case $\mathcal{M}=S^2=\mathrm{SO}(3)/\mathrm{SO}(2)$. Here we emphasize the use of Fourier analysis involving Wigner matrices, spherical harmonics and Clebsch-Gordan coefficients for $G=\mathrm{SO}(3)$, illustrating the power of representation theory for deep learning.
△ Less
Submitted 28 May, 2021;
originally announced May 2021.
-
Massive MIMO with 1-bit ADC
Authors:
Chiara Risi,
Daniel Persson,
Erik G. Larsson
Abstract:
We investigate massive multiple-input-multiple output (MIMO) uplink systems with 1-bit analog-to-digital converters (ADCs) on each receiver antenna. Receivers that rely on 1-bit ADC do not need energy-consuming interfaces such as automatic gain control (AGC). This decreases both ADC building and operational costs. Our design is based on maximal ratio combining (MRC), zero-forcing (ZF), and least s…
▽ More
We investigate massive multiple-input-multiple output (MIMO) uplink systems with 1-bit analog-to-digital converters (ADCs) on each receiver antenna. Receivers that rely on 1-bit ADC do not need energy-consuming interfaces such as automatic gain control (AGC). This decreases both ADC building and operational costs. Our design is based on maximal ratio combining (MRC), zero-forcing (ZF), and least squares (LS) detection, taking into account the effects of the 1-bit ADC on channel estimation. Through numerical results, we show good performance of the system in terms of mutual information and symbol error rate (SER). Furthermore, we provide an analytical approach to calculate the mutual information and SER of the MRC receiver. The analytical approach reduces complexity in the sense that a symbol and channel noise vectors Monte Carlo simulation is avoided.
△ Less
Submitted 30 April, 2014;
originally announced April 2014.
-
Scaling up MIMO: Opportunities and Challenges with Very Large Arrays
Authors:
Fredrik Rusek,
Daniel Persson,
Buon Kiong Lau,
Erik G. Larsson,
Thomas L. Marzetta,
Ove Edfors,
Fredrik Tufvesson
Abstract:
This paper surveys recent advances in the area of very large MIMO systems.
With very large MIMO, we think of systems that use antenna arrays with an order of magnitude more elements than in systems being built today, say a hundred antennas or more. Very large MIMO entails an unprecedented number of antennas simultaneously serving a much smaller number of terminals. The disparity in number emerge…
▽ More
This paper surveys recent advances in the area of very large MIMO systems.
With very large MIMO, we think of systems that use antenna arrays with an order of magnitude more elements than in systems being built today, say a hundred antennas or more. Very large MIMO entails an unprecedented number of antennas simultaneously serving a much smaller number of terminals. The disparity in number emerges as a desirable operating condition and a practical one as well. The number of terminals that can be simultaneously served is limited, not by the number of antennas, but rather by our inability to acquire channel-state information for an unlimited number of terminals. Larger numbers of terminals can always be accommodated by combining very large MIMO technology with conventional time- and frequency-division multiplexing via OFDM. Very large MIMO arrays is a new research field both in communication theory, propagation, and electronics and represents a paradigm shift in the way of thinking both with regards to theory, systems and implementation. The ultimate vision of very large MIMO systems is that the antenna array would consist of small active antenna units, plugged into an (optical) fieldbus.
△ Less
Submitted 16 January, 2012;
originally announced January 2012.
