Showing 1–50 of 66 results for author: Ostergaard, J

Higher-order Common Information

Authors: Jan Østergaard

Abstract: We present a new notion $R_\ell$ of higher-order common information, which quantifies the information that $\ell\geq 2$ arbitrarily distributed random variables have in common. We provide analytical lower bounds on $R_3$ and $R_4$ for jointly Gaussian distributed sources and provide computable lower bounds for $R_\ell$ for any $\ell$ and any sources. We also provide a practical method to estimate… ▽ More We present a new notion $R_\ell$ of higher-order common information, which quantifies the information that $\ell\geq 2$ arbitrarily distributed random variables have in common. We provide analytical lower bounds on $R_3$ and $R_4$ for jointly Gaussian distributed sources and provide computable lower bounds for $R_\ell$ for any $\ell$ and any sources. We also provide a practical method to estimate the lower bounds on, e.g., real-world time-series data. As an example, we consider EEG data acquired in a setup with competing acoustic stimuli. We demonstrate that $R_3$ has descriptive properties that is not in $R_2$. Moreover, we observe a linear relationship between the amount of common information $R_3$ communicated from the acoustic stimuli and to the brain and the corresponding cortical activity in terms of neural tracking of the envelopes of the stimuli. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Comments: Submitted to IEEE Transactions on Information Theory

Directed Redundancy in Time Series

Authors: Jan Østergaard

Abstract: We quantify the average amount of redundant information that is transferred from a subset of relevant random source processes to a target process. To identify the relevant source processes, we consider those that are connected to the target process and in addition share a certain proportion of the total information causally provided to the target. Even if the relevant processes have no directed in… ▽ More We quantify the average amount of redundant information that is transferred from a subset of relevant random source processes to a target process. To identify the relevant source processes, we consider those that are connected to the target process and in addition share a certain proportion of the total information causally provided to the target. Even if the relevant processes have no directed information exchange between them, they can still causally provide redundant information to the target. This makes it difficult to identify the relevant processes. To solve this issue, we propose the existence of a hidden redundancy process that governs the shared information among the relevant processes. We bound the redundancy by the minimal average directed redundancy from the relevant processes to the target, from the hidden redundancy process to the target, and from the hidden redundancy process to the relevant processes. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: Accepted to be presented at the IEEE International Symposium on Information Theory 2024

Deep low-latency joint speech transmission and enhancement over a gaussian channel

Authors: Mohammad Bokaei, Jesper Jensen, Simon Doclo, Jan Østergaard

Abstract: Ensuring intelligible speech communication for hearing assistive devices in low-latency scenarios presents significant challenges in terms of speech enhancement, coding and transmission. In this paper, we propose novel solutions for low-latency joint speech transmission and enhancement, leveraging deep neural networks (DNNs). Our approach integrates two state-of-the-art DNN architectures for low-l… ▽ More Ensuring intelligible speech communication for hearing assistive devices in low-latency scenarios presents significant challenges in terms of speech enhancement, coding and transmission. In this paper, we propose novel solutions for low-latency joint speech transmission and enhancement, leveraging deep neural networks (DNNs). Our approach integrates two state-of-the-art DNN architectures for low-latency speech enhancement and low-latency analog joint source-channel-based transmission, creating a combined low-latency system and jointly training both systems in an end-to-end approach. Due to the computational demands of the enhancement system, this order is suitable when high computational power is unavailable in the decoder, like hearing assistive devices. The proposed system enables the configuration of total latency, achieving high performance even at latencies as low as 3 ms, which is typically challenging to attain. The simulation results provide compelling evidence that a joint enhancement and transmission system is superior to a simple concatenation system in diverse settings, encompassing various wireless channel conditions, latencies, and background noise scenarios. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions

Authors: Holger Severin Bovbjerg, Jesper Jensen, Jan Østergaard, Zheng-Hua Tan

Abstract: In this paper, we propose the use of self-supervised pretraining on a large unlabelled data set to improve the performance of a personalized voice activity detection (VAD) model in adverse conditions. We pretrain a long short-term memory (LSTM)-encoder using the autoregressive predictive coding (APC) framework and fine-tune it for personalized VAD. We also propose a denoising variant of APC, with… ▽ More In this paper, we propose the use of self-supervised pretraining on a large unlabelled data set to improve the performance of a personalized voice activity detection (VAD) model in adverse conditions. We pretrain a long short-term memory (LSTM)-encoder using the autoregressive predictive coding (APC) framework and fine-tune it for personalized VAD. We also propose a denoising variant of APC, with the goal of improving the robustness of personalized VAD. The trained models are systematically evaluated on both clean speech and speech contaminated by various types of noise at different SNR-levels and compared to a purely supervised model. Our experiments show that self-supervised pretraining not only improves performance in clean conditions, but also yields models which are more robust to adverse conditions compared to purely supervised learning. △ Less

Submitted 23 January, 2024; v1 submitted 27 December, 2023; originally announced December 2023.

Comments: To be published at ICASSP2024, 14th of April 2024, Seoul, South Korea. Copyright (c) 2023 IEEE. 5 pages, 2, figures, 5 tables

MSC Class: 68T10 ACM Class: I.2.6

Improving Uniquely Decodable Codes in Binary Adder Channels

Authors: József Balogh, The Nguyen, Patric R. J. Ostergard, Ethan Patrick White, Michael Wigal

Abstract: We present a general method to modify existing uniquely decodable codes in the $T$-user binary adder channel. If at least one of the original constituent codes does not have average weight exactly half of the dimension, then our method produces a new set of constituent codes in a higher dimension, with a strictly higher rate. Using our method we improve the highest known rate for the $T$-user bina… ▽ More We present a general method to modify existing uniquely decodable codes in the $T$-user binary adder channel. If at least one of the original constituent codes does not have average weight exactly half of the dimension, then our method produces a new set of constituent codes in a higher dimension, with a strictly higher rate. Using our method we improve the highest known rate for the $T$-user binary adder channel for all $T \geq 2$. This information theory problem is equivalent to co-Sidon problems initiated by Lindstr{ö}m in the 1960s, and also the multi-set union-free problem. Our results improve the known lower bounds in these settings as well. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Comments: 8 pages

MSC Class: 05D40; 05C65; 05D05; 94A40; 05B10

Investigating the Design Space of Diffusion Models for Speech Enhancement

Authors: Philippe Gonzalez, Zheng-Hua Tan, Jan Østergaard, Jesper Jensen, Tommy Sonne Alstrøm, Tobias May

Abstract: Diffusion models are a new class of generative models that have shown outstanding performance in image generation literature. As a consequence, studies have attempted to apply diffusion models to other tasks, such as speech enhancement. A popular approach in adapting diffusion models to speech enhancement consists in modelling a progressive transformation between the clean and noisy speech signals… ▽ More Diffusion models are a new class of generative models that have shown outstanding performance in image generation literature. As a consequence, studies have attempted to apply diffusion models to other tasks, such as speech enhancement. A popular approach in adapting diffusion models to speech enhancement consists in modelling a progressive transformation between the clean and noisy speech signals. However, one popular diffusion model framework previously laid in image generation literature did not account for such a transformation towards the system input, which prevents from relating the existing diffusion-based speech enhancement systems with the aforementioned diffusion model framework. To address this, we extend this framework to account for the progressive transformation between the clean and noisy speech signals. This allows us to apply recent developments from image generation literature, and to systematically investigate design aspects of diffusion models that remain largely unexplored for speech enhancement, such as the neural network preconditioning, the training loss weighting, the stochastic differential equation (SDE), or the amount of stochasticity injected in the reverse process. We show that the performance of previous diffusion-based speech enhancement systems cannot be attributed to the progressive transformation between the clean and noisy speech signals. Moreover, we show that a proper choice of preconditioning, training loss weighting, SDE and sampler allows to outperform a popular diffusion-based speech enhancement system in terms of perceptual metrics while using fewer sampling steps, thus reducing the computational cost by a factor of four. △ Less

Submitted 7 December, 2023; originally announced December 2023.

Diffusion-Based Speech Enhancement in Matched and Mismatched Conditions Using a Heun-Based Sampler

Authors: Philippe Gonzalez, Zheng-Hua Tan, Jan Østergaard, Jesper Jensen, Tommy Sonne Alstrøm, Tobias May

Abstract: Diffusion models are a new class of generative models that have recently been applied to speech enhancement successfully. Previous works have demonstrated their superior performance in mismatched conditions compared to state-of-the art discriminative models. However, this was investigated with a single database for training and another one for testing, which makes the results highly dependent on t… ▽ More Diffusion models are a new class of generative models that have recently been applied to speech enhancement successfully. Previous works have demonstrated their superior performance in mismatched conditions compared to state-of-the art discriminative models. However, this was investigated with a single database for training and another one for testing, which makes the results highly dependent on the particular databases. Moreover, recent developments from the image generation literature remain largely unexplored for speech enhancement. These include several design aspects of diffusion models, such as the noise schedule or the reverse sampler. In this work, we systematically assess the generalization performance of a diffusion-based speech enhancement model by using multiple speech, noise and binaural room impulse response (BRIR) databases to simulate mismatched acoustic conditions. We also experiment with a noise schedule and a sampler that have not been applied to speech enhancement before. We show that the proposed system substantially benefits from using multiple databases for training, and achieves superior performance compared to state-of-the-art discriminative models in both matched and mismatched conditions. We also show that a Heun-based sampler achieves superior performance at a smaller computational cost compared to a sampler commonly used for speech enhancement. △ Less

Submitted 16 January, 2024; v1 submitted 5 December, 2023; originally announced December 2023.

Comments: Accepted to ICASSP 2024

Head Orientation Estimation with Distributed Microphones Using Speech Radiation Patterns

Authors: Kaspar Müller, Bilgesu Çakmak, Paul Didier, Simon Doclo, Jan Østergaard, Tobias Wolff

Abstract: Determining the head orientation of a talker is not only beneficial for various speech signal processing applications, such as source localization or speech enhancement, but also facilitates intuitive voice control and interaction with smart environments or modern car assistants. Most approaches for head orientation estimation are based on visual cues. However, this requires camera systems which o… ▽ More Determining the head orientation of a talker is not only beneficial for various speech signal processing applications, such as source localization or speech enhancement, but also facilitates intuitive voice control and interaction with smart environments or modern car assistants. Most approaches for head orientation estimation are based on visual cues. However, this requires camera systems which often are not available. We present an approach which purely uses audio signals captured with only a few distributed microphones around the talker. Specifically, we propose a novel method that directly incorporates measured or modeled speech radiation patterns to infer the talker's orientation during active speech periods based on a cosine similarity measure. Moreover, an automatic gain adjustment technique is proposed for uncalibrated, irregular microphone setups, such as ad-hoc sensor networks. In experiments with signals recorded in both anechoic and reverberant environments, the proposed method outperforms state-of-the-art approaches, using either measured or modeled speech radiation patterns. △ Less

Submitted 4 December, 2023; originally announced December 2023.

Comments: 6 pages, submitted to 57th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, 2023

Joint Minimum Processing Beamforming and Near-end Listening Enhancement

Authors: Andreas J. Fuglsig, Jesper Jensen, Zheng-Hua Tan, Lars S. Bertelsen, Jens Christian Lindof, Jan Østergaard

Abstract: We consider speech enhancement for signals picked up in one noisy environment that must be rendered to a listener in another noisy environment. For both far-end noise reduction and near-end listening enhancement, it has been shown that excessive focus on noise suppression or intelligibility maximization may lead to excessive speech distortions and quality degradations in favorable noise conditions… ▽ More We consider speech enhancement for signals picked up in one noisy environment that must be rendered to a listener in another noisy environment. For both far-end noise reduction and near-end listening enhancement, it has been shown that excessive focus on noise suppression or intelligibility maximization may lead to excessive speech distortions and quality degradations in favorable noise conditions, where intelligibility is already at ceiling level. Recently [1,2] propose to remedy this with a minimum processing framework that either reduces noise or enhances listening a minimum amount given that a certain intelligibility criterion is still satisfied Additionally, it has been shown that joint consideration of both environments improves speech enhancement performance. In this paper, we formulate a joint far- and near-end minimum processing framework, that improves intelligibility while limiting speech distortions in favorable noise conditions. We provide closed-form solutions to specific boundary scenarios and investigate performance for the general case using numerical optimization. We also show concatenating existing minimum processing far- and near-end enhancement methods preserves the effects of the initial methods. Results show that the joint optimization can further improve performance compared to the concatenated approach. △ Less

Submitted 5 February, 2024; v1 submitted 20 September, 2023; originally announced September 2023.

Comments: Accepted at IEEE ICASSP 2024 Workshop on Hands-free Speech Communication and Microphone Arrays (HSCMA) 2024

Synergy and Redundancy Dominated Effects in Time Series via Transfer Entropy Decompositions

Authors: Jan Østergaard, Payam Boubakani

Abstract: We present a new decomposition of transfer entropy to characterize the degree of synergy- and redundancy-dominated influence a time series has upon the interaction between other time series. We prove the existence of a class of time series, where the early past of the conditioning time series yields a synergistic effect upon the interaction, whereas the late past has a redundancy-dominated effect.… ▽ More We present a new decomposition of transfer entropy to characterize the degree of synergy- and redundancy-dominated influence a time series has upon the interaction between other time series. We prove the existence of a class of time series, where the early past of the conditioning time series yields a synergistic effect upon the interaction, whereas the late past has a redundancy-dominated effect. In general, different parts of the past can have different effects. Our information theoretic quantities are easy to compute in practice, and we demonstrate their usage on real-world brain data. △ Less

Submitted 6 May, 2024; v1 submitted 12 December, 2022; originally announced December 2022.

Comments: Accepted to be presented at the NeurIT: Information theory in neuroscience and neuroengineering workshop. In connection with ISIT 2024

Minimum Processing Near-end Listening Enhancement

Authors: Andreas Jonas Fuglsig, Jesper Jensen, Zheng-Hua Tan, Lars Søndergaard Bertelsen, Jens Christian Lindof, Jan Østergaard

Abstract: The intelligibility and quality of speech from a mobile phone or public announcement system are often affected by background noise in the listening environment. By pre-processing the speech signal it is possible to improve the speech intelligibility and quality -- this is known as near-end listening enhancement (NLE). Although, existing NLE techniques are able to greatly increase intelligibility i… ▽ More The intelligibility and quality of speech from a mobile phone or public announcement system are often affected by background noise in the listening environment. By pre-processing the speech signal it is possible to improve the speech intelligibility and quality -- this is known as near-end listening enhancement (NLE). Although, existing NLE techniques are able to greatly increase intelligibility in harsh noise environments, in favorable noise conditions the intelligibility of speech reaches a ceiling where it cannot be further enhanced. Actually, the focus of existing methods solely on improving the intelligibility causes unnecessary processing of the speech signal and leads to speech distortions and quality degradations. In this paper, we provide a new rationale for NLE, where the target speech is minimally processed in terms of a processing penalty, provided that a certain performance constraint, e.g., intelligibility, is satisfied. We present a closed-form solution for the case where the performance criterion is an intelligibility estimator based on the approximated speech intelligibility index and the processing penalty is the mean-square error between the processed and the clean speech. This produces an NLE method that adapts to changing noise conditions via a simple gain rule by limiting the processing to the minimum necessary to achieve a desired intelligibility, while at the same time focusing on quality in favorable noise situations by minimizing the amount of speech distortions. Through simulation studies, we show the proposed method attains speech quality on par or better than existing methods in both objective measurements and subjective listening tests, whilst still sustaining objective speech intelligibility performance on par with existing methods. △ Less

Submitted 30 May, 2023; v1 submitted 31 October, 2022; originally announced October 2022.

Model-based estimation of in-car-communication feedback applied to speech zone detection

Authors: Kaspar Müller, Simon Doclo, Jan Østergaard, Tobias Wolff

Abstract: Modern cars provide versatile tools to enhance speech communication. While an in-car communication (ICC) system aims at enhancing communication between the passengers by playing back desired speech via loudspeakers in the car, these loudspeaker signals may disturb a speech enhancement system required for hands-free telephony and automatic speech recognition. In this paper, we focus on speech zone… ▽ More Modern cars provide versatile tools to enhance speech communication. While an in-car communication (ICC) system aims at enhancing communication between the passengers by playing back desired speech via loudspeakers in the car, these loudspeaker signals may disturb a speech enhancement system required for hands-free telephony and automatic speech recognition. In this paper, we focus on speech zone detection, i.e. detecting which passenger in the car is speaking, which is a crucial component of the speech enhancement system. We propose a model-based feedback estimation method to improve robustness of speech zone detection against ICC feedback. Specifically, since the zone detection system typically does not have access to the ICC loudspeaker signals, the proposed method estimates the feedback signal from the observed microphone signals based on a free-field propagation model between the loudspeakers and the microphones as well as the ICC gain. We propose an efficient recursive implementation in the short-time Fourier transform domain using convolutive transfer functions. A realistic simulation study indicates that the proposed method allows to increase the ICC gain by about 6dB while still achieving robust speech zone detection results. △ Less

Submitted 7 October, 2022; originally announced October 2022.

Comments: 5 pages, submitted to International Workshop on Acoustic Signal Enhancement (IWAENC), Bamberg, Germany, 2022

Stabilizing Error Correction Codes for Controlling LTI Systems over Erasure Channels

Authors: Jan Østergaard

Abstract: We propose (k,k') stabilizing codes, which is a type of delayless error correction codes that are useful for control over networks with erasures. For each input symbol, k output symbols are generated by the stabilizing code. Receiving any k' of these outputs guarantees stability. Thus, the system to be stabilized is taken into account in the design of the erasure codes. Our focus is on LTI systems… ▽ More We propose (k,k') stabilizing codes, which is a type of delayless error correction codes that are useful for control over networks with erasures. For each input symbol, k output symbols are generated by the stabilizing code. Receiving any k' of these outputs guarantees stability. Thus, the system to be stabilized is taken into account in the design of the erasure codes. Our focus is on LTI systems, and we construct codes based on independent encodings and multiple descriptions. The theoretical efficiency and performance of the codes are assessed, and their practical performances are demonstrated in a simulation study. There is a significant gain over other delayless codes such as repetition codes. △ Less

Submitted 14 January, 2022; originally announced January 2022.

Comments: Accepted and presented at the IEEE 60th Conference on Decision and Control (CDC). arXiv admin note: substantial text overlap with arXiv:2112.11717

Stabilizing Error Correction Codes for Control over Erasure Channels

Authors: Jan Østergaard

Abstract: We propose $(k,k')$ stabilizing codes, which is a type of delayless error correction codes that are useful for control over networks with erasures. For each input symbol, $k$ output symbols are generated by the stabilizing code. Receiving at least $k'$ of these outputs guarantees stability. Thus, both the system to be stabilized and the channel is taken into account in the design of the erasure co… ▽ More We propose $(k,k')$ stabilizing codes, which is a type of delayless error correction codes that are useful for control over networks with erasures. For each input symbol, $k$ output symbols are generated by the stabilizing code. Receiving at least $k'$ of these outputs guarantees stability. Thus, both the system to be stabilized and the channel is taken into account in the design of the erasure codes. Receiving more than $k'$ outputs further improves the performance of the system. In the case of i.i.d. erasures, we further demonstrate that the erasure code can be constructed such that stability is achieved if on average at least $k'$ output symbols are received. Our focus is on LTI systems, and we construct codes based on independent encodings and multiple descriptions. Stability is assessed via Markov jump linear system theory. The theoretical efficiency and performance of the codes are assessed, and their practical performances are demonstrated in a simulation study. There is a significant gain over other delayless codes such as repetition codes. △ Less

Submitted 22 December, 2021; originally announced December 2021.

Comments: 14 pages. This is a slightly extended preprint (which includes additional simulations) of a manuscript submitted for review in the IEEE Transactions on Control of Network Systems

Joint Far- and Near-End Speech Intelligibility Enhancement based on the Approximated Speech Intelligibility Index

Authors: Andreas Jonas Fuglsig, Jan Østergaard, Jesper Jensen, Lars Søndergaard Bertelsen, Peter Mariager, Zheng-Hua Tan

Abstract: This paper considers speech enhancement of signals picked up in one noisy environment which must be presented to a listener in another noisy environment. Recently, it has been shown that an optimal solution to this problem requires the consideration of the noise sources in both environments jointly. However, the existing optimal mutual information based method requires a complicated system model t… ▽ More This paper considers speech enhancement of signals picked up in one noisy environment which must be presented to a listener in another noisy environment. Recently, it has been shown that an optimal solution to this problem requires the consideration of the noise sources in both environments jointly. However, the existing optimal mutual information based method requires a complicated system model that includes natural speech variations, and relies on approximations and assumptions of the underlying signal distributions. In this paper, we propose to use a simpler signal model and optimize speech intelligibility based on the Approximated Speech Intelligibility Index (ASII). We derive a closed-form solution to the joint far- and near-end speech enhancement problem that is independent of the marginal distribution of signal coefficients, and that achieves similar performance to existing work. In addition, we do not need to model or optimize for natural speech variations. △ Less

Submitted 15 November, 2021; originally announced November 2021.

Switching 3-edge-colorings of cubic graphs

Authors: Jan Goedgebeur, Patric R. J. Östergård

Abstract: The chromatic index of a cubic graph is either 3 or 4. Edge-Kempe switching, which can be used to transform edge-colorings, is here considered for 3-edge-colorings of cubic graphs. Computational results for edge-Kempe switching of cubic graphs up to order 30 and bipartite cubic graphs up to order 36 are tabulated. Families of cubic graphs of orders $4n+2$ and $4n+4$ with $2^n$ edge-Kempe equivalen… ▽ More The chromatic index of a cubic graph is either 3 or 4. Edge-Kempe switching, which can be used to transform edge-colorings, is here considered for 3-edge-colorings of cubic graphs. Computational results for edge-Kempe switching of cubic graphs up to order 30 and bipartite cubic graphs up to order 36 are tabulated. Families of cubic graphs of orders $4n+2$ and $4n+4$ with $2^n$ edge-Kempe equivalence classes are presented; it is conjectured that there are no cubic graphs with more edge-Kempe equivalence classes. New families of nonplanar bipartite cubic graphs with exactly one edge-Kempe equivalence class are also obtained. Edge-Kempe switching is further connected to cycle switching of Steiner triple systems, for which an improvement of the established classification algorithm is presented. △ Less

Submitted 4 May, 2021; originally announced May 2021.

Comments: 20 pages

Directed Data-Processing Inequalities for Systems with Feedback

Authors: Milan S. Derpich, Jan Østergaard

Abstract: We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal map**s, but are allowed to be non-linear, stochastic and time varying. These blocks can for example represent source encoders, decoders or even communication channels. Moreover, the involved s… ▽ More We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal map**s, but are allowed to be non-linear, stochastic and time varying. These blocks can for example represent source encoders, decoders or even communication channels. Moreover, the involved signals can be arbitrarily distributed. Our first main result relates mutual and directed informations and can be interpreted as a law of conservation of information flow. Our second main result is a pair of data-processing inequalities (one the conditional version of the other) between nested pairs of random sequences entirely within the closed loop. Our third main result is introducing and characterizing the notion of in-the-loop (ITL) transmission rate for channel coding scenarios in which the messages are internal to the loop. Interestingly, in this case the conventional notions of transmission rate associated with the entropy of the messages and of channel capacity based on maximizing the mutual information between the messages and the output turn out to be inadequate. Instead, as we show, the ITL transmission rate is the unique notion of rate for which a channel code attains zero error probability if and only if such ITL rate does not exceed the corresponding directed information rate from messages to decoded messages. We apply our data-processing inequalities to show that the supremum of achievable (in the usual channel coding sense) ITL transmission rates is upper bounded by the supremum of the directed information rate across the communication channel. Moreover, we present an example in which this upper bound is attained. Finally, ... △ Less

Submitted 24 March, 2021; originally announced March 2021.

Comments: Submitted to Entropy. arXiv admin note: substantial text overlap with arXiv:1301.6427

Comments on "A Framework for Control System Design Subject to Average Data-Rate Constraints"

Authors: Milan S. Derpich, Jan Østergaard

Abstract: Theorem~ 4.1 in the 2011 paper "A Framework for Control System Design Subject to Average Data-Rate Constraints" allows one to lower bound average operational data rates in feedback loops (including the situation in which encoder and decoder have side information). Unfortunately, its proof is invalid. In this note we first state the theorem and explain why its proof is flawed, and then provide a… ▽ More Theorem~ 4.1 in the 2011 paper "A Framework for Control System Design Subject to Average Data-Rate Constraints" allows one to lower bound average operational data rates in feedback loops (including the situation in which encoder and decoder have side information). Unfortunately, its proof is invalid. In this note we first state the theorem and explain why its proof is flawed, and then provide a correct proof under weaker assumptions. △ Less

Submitted 23 March, 2021; originally announced March 2021.

Comments: Submitted to IEEE Transactions on Automatic Control

Deep Sound Field Reconstruction in Real Rooms: Introducing the ISOBEL Sound Field Dataset

Authors: Miklas Strøm Kristoffersen, Martin Bo Møller, Pablo Martínez-Nuevo, Jan Østergaard

Abstract: Knowledge of loudspeaker responses are useful in a number of applications, where a sound system is located inside a room that alters the listening experience depending on position within the room. Acquisition of sound fields for sound sources located in reverberant rooms can be achieved through labor intensive measurements of impulse response functions covering the room, or alternatively by means… ▽ More Knowledge of loudspeaker responses are useful in a number of applications, where a sound system is located inside a room that alters the listening experience depending on position within the room. Acquisition of sound fields for sound sources located in reverberant rooms can be achieved through labor intensive measurements of impulse response functions covering the room, or alternatively by means of reconstruction methods which can potentially require significantly fewer measurements. This paper extends evaluations of sound field reconstruction at low frequencies by introducing a dataset with measurements from four real rooms. The ISOBEL Sound Field dataset is publicly available, and aims to bridge the gap between synthetic and real-world sound fields in rectangular rooms. Moreover, the paper advances on a recent deep learning-based method for sound field reconstruction using a very low number of microphones, and proposes an approach for modeling both magnitude and phase response in a U-Net-like neural network architecture. The complex-valued sound field reconstruction demonstrates that the estimated room transfer functions are of high enough accuracy to allow for personalized sound zones with contrast ratios comparable to ideal room transfer functions using 15 microphones below 150 Hz. △ Less

Submitted 12 February, 2021; originally announced February 2021.

Estimating Conditional Transfer Entropy in Time Series using Mutual Information and Non-linear Prediction

Authors: Payam Shahsavari Baboukani, Carina Graversen, Emina Alickovic, Jan Østergaard

Abstract: We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of new information and improvement of the prediction accuracy provided by the variables. Then, using a greedy approach, the most informative subsets are selected… ▽ More We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of new information and improvement of the prediction accuracy provided by the variables. Then, using a greedy approach, the most informative subsets are selected in an iterative way. The algorithm terminates, when the highest ranked variable is not able to significantly improve the accuracy of the prediction as compared to that obtained using the existing selected subsets. In a simulation study, we compare our estimator to existing state-of-the-art methods at different data lengths and directed dependencies strengths. It is demonstrated that the proposed estimator has a significantly higher accuracy than that of existing methods, especially for the difficult case, where the data is highly correlated and coupled. Moreover, we show its false detection of directed dependencies due to instantaneous couplings effect is lower than that of existing measures. We also show applicability of the proposed estimator on real intracranial electroencephalography data. △ Less

Submitted 27 November, 2020; originally announced November 2020.

Comments: 22 pages, 9 figures

An Orthogonality Principle for Select-Maximum Estimation of Exponential Variables

Authors: Uri Erez, Jan Østergaard, Ram Zamir

Abstract: It was recently proposed to encode the one-sided exponential source X via K parallel channels, Y1, ..., YK , such that the error signals X - Yi, i = 1,...,K, are one-sided exponential and mutually independent given X. Moreover, it was shown that the optimal estimator \hat{Y} of the source X with respect to the one-sided error criterion, is simply given by the maximum of the outputs, i.e., \hat{Y}… ▽ More It was recently proposed to encode the one-sided exponential source X via K parallel channels, Y1, ..., YK , such that the error signals X - Yi, i = 1,...,K, are one-sided exponential and mutually independent given X. Moreover, it was shown that the optimal estimator \hat{Y} of the source X with respect to the one-sided error criterion, is simply given by the maximum of the outputs, i.e., \hat{Y} = max{Y1,..., YK}. In this paper, we show that the distribution of the resulting estimation error X - \hat{Y} , is equivalent to that of the optimum noise in the backward test-channel of the one-sided exponential source, i.e., it is one-sided exponentially distributed and statistically independent of the joint output Y1,...,YK. △ Less

Submitted 3 February, 2021; v1 submitted 5 November, 2020; originally announced November 2020.

Comments: 5 pages. Submitted to ISIT

Incremental Refinements and Multiple Descriptions with Feedback

Authors: Jan Østergaard, Uri Erez, Ram Zamir

Abstract: It is well known that independent (separate) encoding of K correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem, where the sources are repetitions of the same source, but each description must be individually good. We observe that under mild conditions about the source an… ▽ More It is well known that independent (separate) encoding of K correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem, where the sources are repetitions of the same source, but each description must be individually good. We observe that under mild conditions about the source and distortion measure, the rate ratio Rindependent(K)/Rjoint goes to one in the limit of small rate/high distortion. Moreover, we consider the excess rate with respect to the rate-distortion function, Rindependent(K, M) - R(D), in M rounds of K independent encodings with a final distortion level D. We provide two examples - a Gaussian source with mean-squared error and an exponential source with one-sided error - for which the excess rate vanishes in the limit as the number of rounds M goes to infinity, for any fixed D and K. This result has an interesting interpretation for a multi-round variant of the multiple descriptions problem, where after each round the encoder gets a (block) feedback regarding which of the descriptions arrived: In the limit as the number of rounds M goes to infinity (i.e., many incremental rounds), the total rate of received descriptions approaches the rate-distortion function. We provide theoretical and experimental evidence showing that this phenomenon is in fact more general than in the two examples above. △ Less

Submitted 20 May, 2022; v1 submitted 5 November, 2020; originally announced November 2020.

Comments: 62 pages. Accepted in the IEEE Transactions on Information Theory

Zero-Delay Rate Distortion via Filtering for Vector-Valued Gaussian Sources

Authors: Photios A. Stavrou, Jan Ostergaard, Charalambos D. Charalambous

Abstract: We deal with zero-delay source coding of a vector-valued Gauss-Markov source subject to a mean-squared error (MSE) fidelity criterion characterized by the operational zero-delay vector-valued Gaussian rate distortion function (RDF). We address this problem by considering the nonanticipative RDF (NRDF) which is a lower bound to the causal optimal performance theoretically attainable (OPTA) function… ▽ More We deal with zero-delay source coding of a vector-valued Gauss-Markov source subject to a mean-squared error (MSE) fidelity criterion characterized by the operational zero-delay vector-valued Gaussian rate distortion function (RDF). We address this problem by considering the nonanticipative RDF (NRDF) which is a lower bound to the causal optimal performance theoretically attainable (OPTA) function and operational zero-delay RDF. We recall the realization that corresponds to the optimal "test-channel" of the Gaussian NRDF, when considering a vector Gauss-Markov source subject to a MSE distortion in the finite time horizon. Then, we introduce sufficient conditions to show existence of solution for this problem in the infinite time horizon. For the asymptotic regime, we use the asymptotic characterization of the Gaussian NRDF to provide a new equivalent realization scheme with feedback which is characterized by a resource allocation (reverse-waterfilling) problem across the dimension of the vector source. We leverage the new realization to derive a predictive coding scheme via lattice quantization with subtractive dither and joint memoryless entropy coding. This coding scheme offers an upper bound to the operational zero-delay vector-valued Gaussian RDF. When we use scalar quantization, then for "r" active dimensions of the vector Gauss-Markov source the gap between the obtained lower and theoretical upper bounds is less than or equal to 0.254r + 1 bits/vector. We further show that it is possible when we use vector quantization, and assume infinite dimensional Gauss-Markov sources to make the previous gap to be negligible, i.e., Gaussian NRDF approximates the operational zero-delay Gaussian RDF. We also extend our results to vector-valued Gaussian sources of any finite memory under mild conditions. Our theoretical framework is demonstrated with illustrative numerical experiments. △ Less

Submitted 29 September, 2018; originally announced October 2018.

Comments: 32 pages, 9 figures, published in IEEE Journal of Selected Topics in Signal Processing

Information Loss in the Human Auditory System

Authors: Mohsen Zareian Jahromi, Adel Zahedi, Jesper Jensen, Jan Østergaard

Abstract: From the eardrum to the auditory cortex, where acoustic stimuli are decoded, there are several stages of auditory processing and transmission where information may potentially get lost. In this paper, we aim at quantifying the information loss in the human auditory system by using information theoretic tools. To do so, we consider a speech communication model, where words are uttered and sent th… ▽ More From the eardrum to the auditory cortex, where acoustic stimuli are decoded, there are several stages of auditory processing and transmission where information may potentially get lost. In this paper, we aim at quantifying the information loss in the human auditory system by using information theoretic tools. To do so, we consider a speech communication model, where words are uttered and sent through a noisy channel, and then received and processed by a human listener. We define a notion of information loss that is related to the human word recognition rate. To assess the word recognition rate of humans, we conduct a closed-vocabulary intelligibility test. We derive upper and lower bounds on the information loss. Simulations reveal that the bounds are tight and we observe that the information loss in the human auditory system increases as the signal to noise ratio (SNR) decreases. Our framework also allows us to study whether humans are optimal in terms of speech perception in a noisy environment. Towards that end, we derive optimal classifiers and compare the human and machine performance in terms of information loss and word recognition rate. We observe a higher information loss and lower word recognition rate for humans compared to the optimal classifiers. In fact, depending on the SNR, the machine classifier may outperform humans by as much as 8 dB. This implies that for the speech-in-stationary-noise setup considered here, the human auditory system is sub-optimal for recognizing noisy words. △ Less

Submitted 2 May, 2018; originally announced May 2018.

Stationarity in the Realizations of the Causal Rate-Distortion Function for One-Sided Stationary Sources

Authors: Milan S. Derpich, Marco A. Guerrero, Jan Østergaard

Abstract: This paper derives novel results on the characterization of the the causal information rate-distortion function (IRDF) $R_{c}^{it}(D)$ for arbitrarily-distributed one-sided stationary $κ$-th order Markov source x(1),x(2),.... It is first shown that Gorbunov and Pinsker's results on the stationarity of the realizations to the causal IRDF (stated for two-sided stationary sources) do not apply to the… ▽ More This paper derives novel results on the characterization of the the causal information rate-distortion function (IRDF) $R_{c}^{it}(D)$ for arbitrarily-distributed one-sided stationary $κ$-th order Markov source x(1),x(2),.... It is first shown that Gorbunov and Pinsker's results on the stationarity of the realizations to the causal IRDF (stated for two-sided stationary sources) do not apply to the commonly used family of asymptotic average single-letter (AASL) distortion criteria. Moreover, we show that, in general, a reconstruction sequence cannot be both jointly stationary with a one-sided stationary source sequence and causally related to it. This implies that, in general, the causal IRDF for one-sided stationary sources cannot be realized by a stationary distribution. However, we prove that for an arbitrarily distributed one-sided stationary source and a large class of distortion criteria (including AASL), the search for $R_{c}^{it}(D)$ can be restricted to distributions which yield the output sequence y(1), y(2),... jointly stationary with the source after $κ$ samples. Finally, we improve the definition of the stationary causal IRDF $\overline{R}_{c}^{it}(D)$ previously introduced by Derpich and Østergaard for two-sided Markovian stationary sources and show that $\overline{R}_{c}^{it}(D)$ for a two-sided source ...,x(-1),x(0),x(1),... equals $R_{c}^{it}(D)$ for the associated one-sided source x(1), x(2),.... This implies that, for the Gaussian quadratic case, the practical zero-delay encoder-decoder pairs proposed by Derpich and Østergaard for approaching $R_{c}^{it}(D)$ achieve an operational data rate which exceeds $R_{c}^{it}(D)$ by less than $1+0.5 \log_2(2 πe /12) \simeq 1.254$ bits per sample. △ Less

Submitted 10 April, 2018; originally announced April 2018.

Comments: Slightly improved version of a manuscript submitted to IEEE Transactions on Information Theory

MSC Class: 94A34 (Primary); 94A29 (Secondary)

LCL problems on grids

Authors: Sebastian Brandt, Juho Hirvonen, Janne H. Korhonen, Tuomo Lempiäinen, Patric R. J. Östergård, Christopher Purcell, Joel Rybicki, Jukka Suomela, Przemysław Uznański

Abstract: LCLs or locally checkable labelling problems (e.g. maximal independent set, maximal matching, and vertex colouring) in the LOCAL model of computation are very well-understood in cycles (toroidal 1-dimensional grids): every problem has a complexity of $O(1)$, $Θ(\log^* n)$, or $Θ(n)$, and the design of optimal algorithms can be fully automated. This work develops the complexity theory of LCL prob… ▽ More LCLs or locally checkable labelling problems (e.g. maximal independent set, maximal matching, and vertex colouring) in the LOCAL model of computation are very well-understood in cycles (toroidal 1-dimensional grids): every problem has a complexity of $O(1)$, $Θ(\log^* n)$, or $Θ(n)$, and the design of optimal algorithms can be fully automated. This work develops the complexity theory of LCL problems for toroidal 2-dimensional grids. The complexity classes are the same as in the 1-dimensional case: $O(1)$, $Θ(\log^* n)$, and $Θ(n)$. However, given an LCL problem it is undecidable whether its complexity is $Θ(\log^* n)$ or $Θ(n)$ in 2-dimensional grids. Nevertheless, if we correctly guess that the complexity of a problem is $Θ(\log^* n)$, we can completely automate the design of optimal algorithms. For any problem we can find an algorithm that is of a normal form $A' \circ S_k$, where $A'$ is a finite function, $S_k$ is an algorithm for finding a maximal independent set in $k$th power of the grid, and $k$ is a constant. Finally, partially with the help of automated design tools, we classify the complexity of several concrete LCL problems related to colourings and orientations. △ Less

Submitted 24 May, 2017; v1 submitted 17 February, 2017; originally announced February 2017.

An Upper Bound to Zero-Delay Rate Distortion via Kalman Filtering for Vector Gaussian Sources

Authors: Photios A. Stavrou, Jan Ostergaard, Charalambos D. Charalambous, Milan Derpich

Abstract: We deal with zero-delay source coding of a vector Gaussian autoregressive (AR) source subject to an average mean squared error (MSE) fidelity criterion. Toward this end, we consider the nonanticipative rate distortion function (NRDF) which is a lower bound to the causal and zero-delay rate distortion function (RDF). We use the realization scheme with feedback proposed in [1] to model the correspon… ▽ More We deal with zero-delay source coding of a vector Gaussian autoregressive (AR) source subject to an average mean squared error (MSE) fidelity criterion. Toward this end, we consider the nonanticipative rate distortion function (NRDF) which is a lower bound to the causal and zero-delay rate distortion function (RDF). We use the realization scheme with feedback proposed in [1] to model the corresponding optimal "test-channel" of the NRDF, when considering vector Gaussian AR(1) sources subject to an average MSE distortion. We give conditions on the vector Gaussian AR(1) source to ensure asymptotic stationarity of the realization scheme (bounded performance). Then, we encode the vector innovations due to Kalman filtering via lattice quantization with subtractive dither and memoryless entropy coding. This coding scheme provides a tight upper bound to the zero-delay Gaussian RDF. We extend this result to vector Gaussian AR sources of any finite order. Further, we show that for infinite dimensional vector Gaussian AR sources of any finite order, the NRDF coincides with the zero-delay RDF. Our theoretical framework is corroborated with a simulation example. △ Less

Submitted 21 August, 2017; v1 submitted 23 January, 2017; originally announced January 2017.

Comments: 7 pages, 6 figures, accepted for publication in IEEE Information Theory Workshop (ITW)

Delay Minimization in Real-time Communications with Joint Buffering and Coding

Authors: Jesper H. Sørensen, Petar Popovski, Jan Østergaard

Abstract: We present a closed-form expression for the minimal delay that is achievable in a setting that combines a buffer and an erasure code, used to mitigate the packet delay variance. The erasure code is modeled according to the recent information-theoretic results on finite block length codes. Evaluations reveal that accurate knowledge of the network parameters is essential for optimal operation. Moreo… ▽ More We present a closed-form expression for the minimal delay that is achievable in a setting that combines a buffer and an erasure code, used to mitigate the packet delay variance. The erasure code is modeled according to the recent information-theoretic results on finite block length codes. Evaluations reveal that accurate knowledge of the network parameters is essential for optimal operation. Moreover, it is shown that, when the network packet delay variance is large, the buffer delay becomes negligible. Therefore, in this case the delay budget should be spent mainly on the erasure code. △ Less

Submitted 14 October, 2016; originally announced October 2016.

The chromatic number of the square of the 8-cube

Authors: Janne I. Kokkala, Patric R. J. Östergård

Abstract: A cube-like graph is a Cayley graph for the elementary abelian group of order $2^n$. In studies of the chromatic number of cube-like graphs, the $k$th power of the $n$-dimensional hypercube, $Q_n^k$, is frequently considered. This coloring problem can be considered in the framework of coding theory, as the graph $Q_n^k$ can be constructed with one vertex for each binary word of length $n$ and edge… ▽ More A cube-like graph is a Cayley graph for the elementary abelian group of order $2^n$. In studies of the chromatic number of cube-like graphs, the $k$th power of the $n$-dimensional hypercube, $Q_n^k$, is frequently considered. This coloring problem can be considered in the framework of coding theory, as the graph $Q_n^k$ can be constructed with one vertex for each binary word of length $n$ and edges between vertices exactly when the Hamming distance between the corresponding words is at most $k$. Consequently, a proper coloring of $Q_n^k$ corresponds to a partition of the $n$-dimensional binary Hamming space into codes with minimum distance at least $k+1$. The smallest open case, the chromatic number of $Q_8^2$, is here settled by finding a 13-coloring. Such 13-colorings with specific symmetries are further classified. △ Less

Submitted 6 July, 2016; originally announced July 2016.

Comments: 15 pages. Preliminary results of this research were reported in arXiv:1509.06913

MSC Class: 05C15 (Primary) 94B25 (Secondary)

On the Construction of Jointly Superregular Lower Triangular Toeplitz Matrices

Authors: Jonas Hansen, Jan Østergaard, Johnny Kudahl, John H. Madsen

Abstract: Superregular matrices have the property that all of their submatrices, which can be full rank are so. Lower triangular superregular matrices are useful for e.g., maximum distance separable convolutional codes as well as for (sequential) network codes. In this work, we provide an explicit design for all superregular lower triangular Toeplitz matrices in GF(2^p) for the case of matrices with dimensi… ▽ More Superregular matrices have the property that all of their submatrices, which can be full rank are so. Lower triangular superregular matrices are useful for e.g., maximum distance separable convolutional codes as well as for (sequential) network codes. In this work, we provide an explicit design for all superregular lower triangular Toeplitz matrices in GF(2^p) for the case of matrices with dimensions less than or equal to 5 x 5. For higher dimensional matrices, we present a greedy algorithm that finds a solution provided the field size is sufficiently high. We also introduce the notions of jointly superregular and product preserving jointly superregular matrices, and extend our explicit constructions of superregular matrices to these cases. Jointly superregular matrices are necessary to achieve optimal decoding capabilities for the case of codes with a rate lower than 1/2 , and the product preserving property is necessary for optimal decoding capabilities in network recoding. △ Less

Submitted 27 April, 2016; v1 submitted 26 April, 2016; originally announced April 2016.

Comments: Accepted for publication in 2016 IEEE International Symposium on Information Theory (ISIT)

Constructing error-correcting binary codes using transitive permutation groups

Authors: Antti Laaksonen, Patric R. J. Östergård

Abstract: Let $A_2(n,d)$ be the maximum size of a binary code of length $n$ and minimum distance $d$. In this paper we present the following new lower bounds: $A_2(18,4) \ge 5632$, $A_2(21,4) \ge 40960$, $A_2(22,4) \ge 81920$, $A_2(23,4) \ge 163840$, $A_2(24,4) \ge 327680$, $A_2(24,10) \ge 136$, and $A_2(25,6) \ge 17920$. The new lower bounds are a result of a systematic computer search over transitive perm… ▽ More Let $A_2(n,d)$ be the maximum size of a binary code of length $n$ and minimum distance $d$. In this paper we present the following new lower bounds: $A_2(18,4) \ge 5632$, $A_2(21,4) \ge 40960$, $A_2(22,4) \ge 81920$, $A_2(23,4) \ge 163840$, $A_2(24,4) \ge 327680$, $A_2(24,10) \ge 136$, and $A_2(25,6) \ge 17920$. The new lower bounds are a result of a systematic computer search over transitive permutation groups. △ Less

Submitted 18 July, 2016; v1 submitted 20 April, 2016; originally announced April 2016.

The Entropy Gain of Linear Time-Invariant Filters and Some of its Implications

Authors: Milan S. Derpich, Matías Müller, Jan Østergaard

Abstract: We study the increase in per-sample differential entropy rate of random sequences and processes after being passed through a non minimum-phase (NMP) discrete-time, linear time-invariant (LTI) filter G. For such filters and random processes, it has long been established that this entropy gain, Gain(G), equals the integral of log|G(exp(jw))|. It is also known that, if the first sample of the impulse… ▽ More We study the increase in per-sample differential entropy rate of random sequences and processes after being passed through a non minimum-phase (NMP) discrete-time, linear time-invariant (LTI) filter G. For such filters and random processes, it has long been established that this entropy gain, Gain(G), equals the integral of log|G(exp(jw))|. It is also known that, if the first sample of the impulse response of G has unit-magnitude, then this integral equals the sum of the logarithm of the magnitudes of the non-minimum phase zeros of G, say B(G). In this note, we begin by showing that existing time-domain proofs of these results, which consider finite length-n sequences and then let n tend to infinity, have neglected significant mathematical terms and, therefore, are inaccurate. We discuss some of the implications of this oversight when considering random processes. We then present a rigorous time-domain analysis of the entropy gain of LTI filters for random processes. In particular, we show that the entropy gain between equal-length input and output sequences is upper bounded by B(G) and arises if and only if there exists an output additive disturbance with finite differential entropy (no matter how small) or a random initial state. Instead, when comparing the input differential entropy to that of the entire (longer) output of G, the entropy gain equals B(G) without the need for additional exogenous random signals. We illustrate some of the consequences of these results by presenting their implications in three different problems. Specifically: a simple derivation of the rate-distortion function for Gaussian non-stationary sources, conditions for equality in an information inequality of importance in networked control problems, and an observation on the capacity of auto-regressive Gaussian channels with feedback. △ Less

Submitted 11 December, 2015; originally announced December 2015.

Comments: Submitted to IEEE Transactions on Information Theory. This version of the manuscript has a shorter abstract (due to arxiv restrictions)

MSC Class: 94A17; 94A12; 93E20 ACM Class: H.1.1

A coloring of the square of the 8-cube with 13 colors

Authors: Janne I. Kokkala, Patric R. J. Östergård

Abstract: Let $χ_{\bar{k}}(n)$ be the number of colors required to color the $n$-dimensional hypercube such that no two vertices with the same color are at a distance at most $k$. In other words, $χ_{\bar{k}}(n)$ is the minimum number of binary codes with minimum distance at least $k+1$ required to partition the $n$-dimensional Hamming space. By giving an explicit coloring, it is shown that… ▽ More Let $χ_{\bar{k}}(n)$ be the number of colors required to color the $n$-dimensional hypercube such that no two vertices with the same color are at a distance at most $k$. In other words, $χ_{\bar{k}}(n)$ is the minimum number of binary codes with minimum distance at least $k+1$ required to partition the $n$-dimensional Hamming space. By giving an explicit coloring, it is shown that $χ_{\bar{2}}(8)=13$. △ Less

Submitted 23 September, 2015; originally announced September 2015.

Comments: 3 pages

Further Results on the Classification of MDS Codes

Authors: Janne I. Kokkala, Patric R. J. Östergård

Abstract: A $q$-ary maximum distance separable (MDS) code $C$ with length $n$, dimension $k$ over an alphabet $\mathcal{A}$ of size $q$ is a set of $q^k$ codewords that are elements of $\mathcal{A}^n$, such that the Hamming distance between two distinct codewords in $C$ is at least $n-k+1$. Sets of mutually orthogonal Latin squares of orders $q\leq 9$, corresponding to two-dimensional \mbox{$q$-}ary MDS cod… ▽ More A $q$-ary maximum distance separable (MDS) code $C$ with length $n$, dimension $k$ over an alphabet $\mathcal{A}$ of size $q$ is a set of $q^k$ codewords that are elements of $\mathcal{A}^n$, such that the Hamming distance between two distinct codewords in $C$ is at least $n-k+1$. Sets of mutually orthogonal Latin squares of orders $q\leq 9$, corresponding to two-dimensional \mbox{$q$-}ary MDS codes, and $q$-ary one-error-correcting MDS codes for $q\leq 8$ have been classified in earlier studies. These results are used here to complete the classification of all $7$-ary and $8$-ary MDS codes with $d\geq 3$ using a computer search. △ Less

Submitted 27 April, 2015; originally announced April 2015.

New Lower Bounds for the Shannon Capacity of Odd Cycles

Authors: K. Ashik Mathew, Patric R. J. Östergård

Abstract: The Shannon capacity of a graph $G$ is defined as $c(G)=\sup_{d\geq 1}(α(G^d))^{\frac{1}{d}},$ where $α(G)$ is the independence number of $G$. The Shannon capacity of the cycle $C_5$ on $5$ vertices was determined by Lovász in 1979, but the Shannon capacity of a cycle $C_p$ for general odd $p$ remains one of the most notorious open problems in information theory. By prescribing stabilizers for the… ▽ More The Shannon capacity of a graph $G$ is defined as $c(G)=\sup_{d\geq 1}(α(G^d))^{\frac{1}{d}},$ where $α(G)$ is the independence number of $G$. The Shannon capacity of the cycle $C_5$ on $5$ vertices was determined by Lovász in 1979, but the Shannon capacity of a cycle $C_p$ for general odd $p$ remains one of the most notorious open problems in information theory. By prescribing stabilizers for the independent sets in $C_p^d$ and using stochastic search methods, we show that $α(C_7^5)\geq 350$, $α(C_{11}^4)\geq 748$, $α(C_{13}^4)\geq 1534$ and $α(C_{15}^3)\geq 381$. This leads to improved lower bounds on the Shannon capacity of $C_7$ and $C_{15}$: $c(C_7)\geq 350^{\frac{1}{5}}> 3.2271$ and $c(C_{15})\geq 381^{\frac{1}{3}}> 7.2495$. △ Less

Submitted 7 April, 2015; originally announced April 2015.

Source Coding in Networks with Covariance Distortion Constraints

Authors: Adel Zahedi, Jan Østergaard, Søren Holdt Jensen, Patrick A. Naylor, Søren Bech

Abstract: We consider a source coding problem with a network scenario in mind, and formulate it as a remote vector Gaussian Wyner-Ziv problem under covariance matrix distortions. We define a notion of minimum for two positive-definite matrices based on which we derive an explicit formula for the rate-distortion function (RDF). We then study the special cases and applications of this result. We show that two… ▽ More We consider a source coding problem with a network scenario in mind, and formulate it as a remote vector Gaussian Wyner-Ziv problem under covariance matrix distortions. We define a notion of minimum for two positive-definite matrices based on which we derive an explicit formula for the rate-distortion function (RDF). We then study the special cases and applications of this result. We show that two well-studied source coding problems, i.e. remote vector Gaussian Wyner-Ziv problems with mean-squared error and mutual information constraints are in fact special cases of our results. Finally, we apply our results to a joint source coding and denoising problem. We consider a network with a centralized topology and a given weighted sum-rate constraint, where the received signals at the center are to be fused to maximize the output SNR while enforcing no linear distortion. We show that one can design the distortion matrices at the nodes in order to maximize the output SNR at the fusion center. We thereby bridge between denoising and source coding within this setup. △ Less

Submitted 27 September, 2016; v1 submitted 5 April, 2015; originally announced April 2015.

On the Classification of MDS Codes

Authors: Janne I. Kokkala, Denis S. Krotov, Patric R. J. Östergård

Abstract: A $q$-ary code of length $n$, size $M$, and minimum distance $d$ is called an $(n,M,d)_q$ code. An $(n,q^{k},n-k+1)_q$ code is called a maximum distance separable (MDS) code. In this work, some MDS codes over small alphabets are classified. It is shown that every $(k+d-1,q^k,d)_q$ code with $k\geq 3$, $d \geq 3$, $q \in \{5,7\}$ is equivalent to a linear code with the same parameters. This implies… ▽ More A $q$-ary code of length $n$, size $M$, and minimum distance $d$ is called an $(n,M,d)_q$ code. An $(n,q^{k},n-k+1)_q$ code is called a maximum distance separable (MDS) code. In this work, some MDS codes over small alphabets are classified. It is shown that every $(k+d-1,q^k,d)_q$ code with $k\geq 3$, $d \geq 3$, $q \in \{5,7\}$ is equivalent to a linear code with the same parameters. This implies that the $(6,5^4,3)_5$ code and the $(n,7^{n-2},3)_7$ MDS codes for $n\in\{6,7,8\}$ are unique. The classification of one-error-correcting $8$-ary MDS codes is also finished; there are $14$, $8$, $4$, and $4$ equivalence classes of $(n,8^{n-2},3)_8$ codes for $n=6,7,8,9$, respectively. One of the equivalence classes of perfect $(9,8^7,3)_8$ codes corresponds to the Hamming code and the other three are nonlinear codes for which there exists no previously known construction. △ Less

Submitted 21 November, 2014; originally announced November 2014.

Comments: Submitted to IEEE transactions on Information Theory; presented in part at the 4th International Castle Meeting in Coding Theory and Applications, Palmela, Portugal, September 2014

Journal ref: IEEE Trans. Inf. Theory 61(12) 2015, 6485-6492

Non-existence of a ternary constant weight $(16, 5, 15; 2048)$ diameter perfect code

Authors: Denis S. Krotov, Patric R. J. Östergård, Olli Pottonen

Abstract: Ternary constant weight codes of length $n=2^m$, weight $n-1$, cardinality $2^n$ and distance $5$ are known to exist for every $m$ for which there exists an APN permutation of order $2^m$, that is, at least for all odd $m \geq 3$ and for $m=6$. We show the non-existence of such codes for $m=4$ and prove that any codes with the parameters above are diameter perfect. Ternary constant weight codes of length $n=2^m$, weight $n-1$, cardinality $2^n$ and distance $5$ are known to exist for every $m$ for which there exists an APN permutation of order $2^m$, that is, at least for all odd $m \geq 3$ and for $m=6$. We show the non-existence of such codes for $m=4$ and prove that any codes with the parameters above are diameter perfect. △ Less

Submitted 29 August, 2014; originally announced August 2014.

Comments: 9 pages. Submitted for publication

Journal ref: Adv. Math. Commun. 10(2) 2016, 393-399

A Characterization of the Minimal Average Data Rate that Guarantees a Given Closed-Loop Performance Level

Authors: Eduardo~I. Silva, Milan S. Derpich, Jan Ostergaard, Marco A. Encina

Abstract: This paper studies networked control systems closed over noiseless digital channels. By focusing on noisy LTI plants with scalar-valued control inputs and sensor outputs, we derive an absolute lower bound on the minimal average data rate that allows one to achieve a prescribed level of stationary performance under Gaussianity assumptions. We also present a simple coding scheme that allows one to a… ▽ More This paper studies networked control systems closed over noiseless digital channels. By focusing on noisy LTI plants with scalar-valued control inputs and sensor outputs, we derive an absolute lower bound on the minimal average data rate that allows one to achieve a prescribed level of stationary performance under Gaussianity assumptions. We also present a simple coding scheme that allows one to achieve average data rates that are at most 1.254 bits away from the derived lower bound, while satisfying the performance constraint. Our results are given in terms of the solution to a stationary signal-to-noise ratio minimization problem and builds upon a recently proposed framework to deal with average data rate constraints in feedback systems. A numerical example is presented to illustrate our findings. △ Less

Submitted 1 July, 2014; originally announced July 2014.

Comments: Submitted to IEEE Transactions on Automatic Control on December 26, 2013

Distributed Remote Vector Gaussian Source Coding with Covariance Distortion Constraints

Authors: Adel Zahedi, Jan Ostergaard, Soren Holdt Jensen, Patrick Naylor, Soren Bech

Abstract: In this paper, we consider a distributed remote source coding problem, where a sequence of observations of source vectors is available at the encoder. The problem is to specify the optimal rate for encoding the observations subject to a covariance matrix distortion constraint and in the presence of side information at the decoder. For this problem, we derive lower and upper bounds on the rate-dist… ▽ More In this paper, we consider a distributed remote source coding problem, where a sequence of observations of source vectors is available at the encoder. The problem is to specify the optimal rate for encoding the observations subject to a covariance matrix distortion constraint and in the presence of side information at the decoder. For this problem, we derive lower and upper bounds on the rate-distortion function (RDF) for the Gaussian case, which in general do not coincide. We then provide some cases, where the RDF can be derived exactly. We also show that previous results on specific instances of this problem can be generalized using our results. We finally show that if the distortion measure is the mean squared error, or if it is replaced by a certain mutual information constraint, the optimal rate can be derived from our main result. △ Less

Submitted 4 June, 2014; v1 submitted 23 January, 2014; originally announced January 2014.

Comments: This is the final version accepted at ISIT'14

Distributed Remote Vector Gaussian Source Coding for Wireless Acoustic Sensor Networks

Authors: Adel Zahedi, Jan Ostergaard, Soren Holdt Jensen, Patrick Naylor, Soren Bech

Abstract: In this paper, we consider the problem of remote vector Gaussian source coding for a wireless acoustic sensor network. Each node receives messages from multiple nodes in the network and decodes these messages using its own measurement of the sound field as side information. The node's measurement and the estimates of the source resulting from decoding the received messages are then jointly encoded… ▽ More In this paper, we consider the problem of remote vector Gaussian source coding for a wireless acoustic sensor network. Each node receives messages from multiple nodes in the network and decodes these messages using its own measurement of the sound field as side information. The node's measurement and the estimates of the source resulting from decoding the received messages are then jointly encoded and transmitted to a neighboring node in the network. We show that for this distributed source coding scenario, one can encode a so-called conditional sufficient statistic of the sources instead of jointly encoding multiple sources. We focus on the case where node measurements are in form of noisy linearly mixed combinations of the sources and the acoustic channel mixing matrices are invertible. For this problem, we derive the rate-distortion function for vector Gaussian sources and under covariance distortion constraints. △ Less

Submitted 16 January, 2014; originally announced January 2014.

Comments: 10 pages, to be presented at the IEEE DCC'14

Sampling versus Random Binning for Multiple Descriptions of a Bandlimited Source

Authors: Adam Mashiach, Jan Ostergaard, Ram Zamir

Abstract: Random binning is an efficient, yet complex, coding technique for the symmetric L-description source coding problem. We propose an alternative approach, that uses the quantized samples of a bandlimited source as "descriptions". By the Nyquist condition, the source can be reconstructed if enough samples are received. We examine a coding scheme that combines sampling and noise-shaped quantization fo… ▽ More Random binning is an efficient, yet complex, coding technique for the symmetric L-description source coding problem. We propose an alternative approach, that uses the quantized samples of a bandlimited source as "descriptions". By the Nyquist condition, the source can be reconstructed if enough samples are received. We examine a coding scheme that combines sampling and noise-shaped quantization for a scenario in which only K < L descriptions or all L descriptions are received. Some of the received K-sets of descriptions correspond to uniform sampling while others to non-uniform sampling. This scheme achieves the optimum rate-distortion performance for uniform-sampling K-sets, but suffers noise amplification for nonuniform-sampling K-sets. We then show that by increasing the sampling rate and adding a random-binning stage, the optimal operation point is achieved for any K-set. △ Less

Submitted 19 November, 2013; originally announced November 2013.

Comments: Presented at the ITW'13. 5 pages, two-column mode, 3 figures

Sparse Command Generator for Remote Control

Authors: Masaaki Nagahara, Daniel E. Quevedo, Jan Ostergaard, Takahiro Matsuda, Kazunori Hayashi

Abstract: In this article, we consider remote-controlled systems, where the command generator and the controlled object are connected with a bandwidth-limited communication link. In the remote-controlled systems, efficient representation of control commands is one of the crucial issues because of the bandwidth limitations of the link. We propose a new representation method for control commands based on comp… ▽ More In this article, we consider remote-controlled systems, where the command generator and the controlled object are connected with a bandwidth-limited communication link. In the remote-controlled systems, efficient representation of control commands is one of the crucial issues because of the bandwidth limitations of the link. We propose a new representation method for control commands based on compressed sensing. In the proposed method, compressed sensing reduces the number of bits in each control signal by representing it as a sparse vector. The compressed sensing problem is solved by an L1-L2 optimization, which can be effectively implemented with an iterative shrinkage algorithm. A design example also shows the effectiveness of the proposed method. △ Less

Submitted 12 August, 2013; originally announced August 2013.

Journal ref: The 9th IEEE International Conference on Control & Automation (ICCA), pp. 1055-1059, Dec. 2011

Power Control and Coding Formulation for State Estimation with Wireless Sensors

Authors: Daniel E. Quevedo, Jan Ostergaard, Anders Ahlen

Abstract: Technological advances have made wireless sensors cheap and reliable enough to be brought into industrial use. A major challenge arises from the fact that wireless channels introduce random packet dropouts. Power control and coding are key enabling technologies in wireless communications to ensure efficient communications. In the present work, we examine the role of power control and coding for Ka… ▽ More Technological advances have made wireless sensors cheap and reliable enough to be brought into industrial use. A major challenge arises from the fact that wireless channels introduce random packet dropouts. Power control and coding are key enabling technologies in wireless communications to ensure efficient communications. In the present work, we examine the role of power control and coding for Kalman filtering over wireless correlated channels. Two estimation architectures are considered: In the first, the sensors send their measurements directly to a single gateway. In the second scheme, wireless relay nodes provide additional links. The gateway decides on the coding scheme and the transmitter power levels of the wireless nodes. The decision process is carried out on-line and adapts to varying channel conditions in order to improve the trade-off between state estimation accuracy and energy expenditure. In combination with predictive power control, we investigate the use of multiple-description coding, zero-error coding and network coding and provide sufficient conditions for the expectation of the estimation error covariance matrix to be bounded. Numerical results suggest that the proposed method may lead to energy savings of around 50 %, when compared to an alternative scheme, wherein transmission power levels and bit-rates are governed by simple logic. In particular, zero-error coding is preferable at time instances with high channel gains, whereas multiple-description coding is superior for time instances with low gains. When channels between the sensors and the gateway are in deep fades, network coding improves estimation accuracy significantly without sacrificing energy efficiency. △ Less

Submitted 7 August, 2013; originally announced August 2013.

Comments: 15 pages, 12 figures

MSC Class: 93E11; 94A29

Sparsely-Packetized Predictive Control by Orthogonal Matching Pursuit

Authors: Masaaki Nagahara, Daniel E. Quevedo, Jan Ostergaard

Abstract: We study packetized predictive control, known to be robust against packet dropouts in networked systems. To obtain sparse packets for rate-limited networks, we design control packets via an L0 optimization, which can be effectively solved by orthogonal matching pursuit. Our formulation ensures asymptotic stability of the control loop in the presence of bounded packet dropouts. We study packetized predictive control, known to be robust against packet dropouts in networked systems. To obtain sparse packets for rate-limited networks, we design control packets via an L0 optimization, which can be effectively solved by orthogonal matching pursuit. Our formulation ensures asymptotic stability of the control loop in the presence of bounded packet dropouts. △ Less

Submitted 2 August, 2013; originally announced August 2013.

Comments: 3-page extended abstract for MTNS 2012 with 3 figures

MSC Class: 37N35; 47N70; 49J15; 49M20; 93C41; 93D20

Planar Hypohamiltonian Graphs on 40 Vertices

Authors: Mohammadreza Jooyandeh, Brendan D. McKay, Patric R. J. Östergård, Ville H. Pettersson, Carol T. Zamfirescu

Abstract: A graph is hypohamiltonian if it is not Hamiltonian, but the deletion of any single vertex gives a Hamiltonian graph. Until now, the smallest known planar hypohamiltonian graph had 42 vertices, a result due to Araya and Wiener. That result is here improved upon by 25 planar hypohamiltonian graphs of order 40, which are found through computer-aided generation of certain families of planar graphs wi… ▽ More A graph is hypohamiltonian if it is not Hamiltonian, but the deletion of any single vertex gives a Hamiltonian graph. Until now, the smallest known planar hypohamiltonian graph had 42 vertices, a result due to Araya and Wiener. That result is here improved upon by 25 planar hypohamiltonian graphs of order 40, which are found through computer-aided generation of certain families of planar graphs with girth 4 and a fixed number of 4-faces. It is further shown that planar hypohamiltonian graphs exist for all orders greater than or equal to 42. If Hamiltonian cycles are replaced by Hamiltonian paths throughout the definition of hypohamiltonian graphs, we get the definition of hypotraceable graphs. It is shown that there is a planar hypotraceable graph of order 154 and of all orders greater than or equal to 156. We also show that the smallest hypohamiltonian planar graph of girth 5 has 45 vertices. △ Less

Submitted 6 December, 2015; v1 submitted 11 February, 2013; originally announced February 2013.

MSC Class: 05C10; 05C30; 05C38; 05C45; 05C85

Fundamental Inequalities and Identities Involving Mutual and Directed Informations in Closed-Loop Systems

Authors: Milan S. Derpich, Eduardo I. Silva, Jan Østergaard

Abstract: We present several novel identities and inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal map**s, but are allowed to be non-linear, stochastic and time varying. Moreover, the involved signals can be arbitrarily distributed. We bound the directed information between signals i… ▽ More We present several novel identities and inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal map**s, but are allowed to be non-linear, stochastic and time varying. Moreover, the involved signals can be arbitrarily distributed. We bound the directed information between signals inside the feedback loop by the mutual information between signals inside and outside the feedback loop. This fundamental result has an interesting interpretation as a law of conservation of information flow. Building upon it, we derive several novel identities and inequalities, which allow us to prove some existing information inequalities under less restrictive assumptions. Finally, we establish new relationships between nested directed informations inside a feedback loop. This yields a new and general data-processing inequality for systems with feedback. △ Less

Submitted 27 January, 2013; originally announced January 2013.

Comments: Submitted to IEEE Trans. on Information Theory

Analysis of LT Codes with Unequal Recovery Time

Authors: Jesper H. Sørensen, Petar Popovski, Jan Østergaard

Abstract: In this paper we analyze a specific class of rateless codes, called LT codes with unequal recovery time. These codes provide the option of prioritizing different segments of the transmitted data over other. The result is that segments are decoded in stages during the rateless transmission, where higher prioritized segments are decoded at lower overhead. Our analysis focuses on quantifying the expe… ▽ More In this paper we analyze a specific class of rateless codes, called LT codes with unequal recovery time. These codes provide the option of prioritizing different segments of the transmitted data over other. The result is that segments are decoded in stages during the rateless transmission, where higher prioritized segments are decoded at lower overhead. Our analysis focuses on quantifying the expected amount of received symbols, which are redundant already upon arrival, i.e. all input symbols contained in the received symbols have already been decoded. This analysis gives novel insights into the probabilistic mechanisms of LT codes with unequal recovery time, which has not yet been available in the literature. We show that while these rateless codes successfully provide the unequal recovery time, they do so at a significant price in terms of redundancy in the lower prioritized segments. We propose and analyze a modification where a single intermediate feedback is transmitted, when the first segment is decoded in a code with two segments. Our analysis shows that this modification provides a dramatic improvement on the decoding performance of the lower prioritized segment. △ Less

Submitted 20 April, 2012; originally announced April 2012.

Comments: 33 pages, 10 figures

Incremental Refinement using a Gaussian Test Channel

Authors: Jan Ostergaard, Ram Zamir

Abstract: The additive rate-distortion function (ARDF) was developed in order to universally bound the rate loss in the Wyner-Ziv problem, and has since then been instrumental in e.g., bounding the rate loss in successive refinements, universal quantization, and other multi-terminal source coding settings. The ARDF is defined as the minimum mutual information over an additive test channel followed by estima… ▽ More The additive rate-distortion function (ARDF) was developed in order to universally bound the rate loss in the Wyner-Ziv problem, and has since then been instrumental in e.g., bounding the rate loss in successive refinements, universal quantization, and other multi-terminal source coding settings. The ARDF is defined as the minimum mutual information over an additive test channel followed by estimation. In the limit of high resolution, the ADRF coincides with the true RDF for many sources and fidelity criterions. In the other extreme, i.e., the limit of low resolutions, the behavior of the ARDF has not previously been rigorously addressed. In this work, we consider the special case of quadratic distortion and where the noise in the test channel is Gaussian distributed. We first establish a link to the I-MMSE relation of Guo et al. and use this to show that for any source the slope of the ARDF near zero rate, converges to the slope of the Gaussian RDF near zero rate. We then consider the multiplicative rate loss of the ARDF, and show that for bursty sources it may be unbounded, contrary to the additive rate loss, which is upper bounded by 1/2 bit for all sources. We finally show that unconditional incremental refinement, i.e., where each refinement is encoded independently of the other refinements, is ARDF optimal in the limit of low resolution, independently of the source distribution. Our results also reveal under which conditions linear estimation is ARDF optimal in the low rate regime. △ Less

Submitted 25 May, 2011; originally announced May 2011.

Comments: 5 pages, to appear at IEEE International Symposium on Information Theory (ISIT), August 2011

On Optimal Binary One-Error-Correcting Codes of Lengths $2^m-4$ and $2^m-3$

Authors: Denis S. Krotov, Patric R. J. Östergård, Olli Pottonen

Abstract: Best and Brouwer [Discrete Math. 17 (1977), 235-245] proved that triply-shortened and doubly-shortened binary Hamming codes (which have length $2^m-4$ and $2^m-3$, respectively) are optimal. Properties of such codes are here studied, determining among other things parameters of certain subcodes. A utilization of these properties makes a computer-aided classification of the optimal binary one-error… ▽ More Best and Brouwer [Discrete Math. 17 (1977), 235-245] proved that triply-shortened and doubly-shortened binary Hamming codes (which have length $2^m-4$ and $2^m-3$, respectively) are optimal. Properties of such codes are here studied, determining among other things parameters of certain subcodes. A utilization of these properties makes a computer-aided classification of the optimal binary one-error-correcting codes of lengths 12 and 13 possible; there are 237610 and 117823 such codes, respectively (with 27375 and 17513 inequivalent extensions). This completes the classification of optimal binary one-error-correcting codes for all lengths up to 15. Some properties of the classified codes are further investigated. Finally, it is proved that for any $m \geq 4$, there are optimal binary one-error-correcting codes of length $2^m-4$ and $2^m-3$ that cannot be lengthened to perfect codes of length $2^m-1$. △ Less

Submitted 20 April, 2011; originally announced April 2011.

Comments: Accepted for publication in IEEE Transactions on Information Theory. Data available at http://www.iki.fi/opottone/codes

Journal ref: IEEE Trans. Inf. Theory 57(10) 2011, 6771-6779