Showing 1–47 of 47 results for author: Kostina, V

Coded Kalman Filtering over MIMO Gaussian Channels with Feedback

Authors: Barron Han, Oron Sabag, Victoria Kostina, Babak Hassibi

Abstract: We consider the problem of remotely stabilizing a linear dynamical system. In this setting, a sensor co-located with the system communicates the system's state to a controller over a noisy communication channel with feedback. The objective of the controller (decoder) is to use the channel outputs to estimate the vector state with finite zero-delay mean squared error (MSE) at the infinite horizon.… ▽ More We consider the problem of remotely stabilizing a linear dynamical system. In this setting, a sensor co-located with the system communicates the system's state to a controller over a noisy communication channel with feedback. The objective of the controller (decoder) is to use the channel outputs to estimate the vector state with finite zero-delay mean squared error (MSE) at the infinite horizon. It has been shown in [1] that for a vector Gauss-Markov source and either a single-input multiple-output (SIMO) or a multiple-input single-output (MISO) channel, linear codes require the minimum capacity to achieve finite MSE. This paper considers the more general problem of linear zero-delay joint-source channel coding (JSCC) of a vector-valued source over a multiple-input multiple-output (MIMO) Gaussian channel with feedback. We study sufficient and necessary conditions for linear codes to achieve finite MSE. For sufficiency, we introduce a coding scheme where each unstable source mode is allocated to a single channel for estimation. Our proof for the necessity of this scheme relies on a matrix-algebraic conjecture that we prove to be true if either the source or channel is scalar. We show that linear codes achieve finite MSE for a scalar source over a MIMO channel if and only if the best scalar sub-channel can achieve finite MSE. Finally, we provide a new counter-example demonstrating that linear codes are generally sub-optimal for coding over MIMO channels. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: Accepted for presentation at the 2024 IEEE International Symposium on Information Theory

A Distributionally Robust Approach to Shannon Limits using the Wasserstein Distance

Authors: Vikrant Malik, Taylan Kargin, Victoria Kostina, Babak Hassibi

Abstract: We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive channel noise is unknown and lies within a Wasserstein-2 ambiguity set of a given radius centered around a specified nominal distribution, and we look for the wors… ▽ More We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive channel noise is unknown and lies within a Wasserstein-2 ambiguity set of a given radius centered around a specified nominal distribution, and we look for the worst-case asymptotically optimal coding rate over such an ambiguity set. Varying the radius of the ambiguity set allows us to interpolate between the worst-case and stochastic scenarios using probabilistic tools. Our problem setting fits into the paradigm of compound source / channel models introduced by Sakrison and Blackwell, respectively. This paper shows that if the nominal distribution is Gaussian, then so is the worst-case source / noise distribution, and the compound rate-distortion / channel capacity functions admit convex formulations with Linear Matrix Inequality (LMI) constraints. These formulations yield simple closed-form expressions in the scalar case, offering insights into the behavior of Shannon limits with the changing radius of the Wasserstein-2 ambiguity set. △ Less

Submitted 13 May, 2024; v1 submitted 10 May, 2024; originally announced May 2024.

Coded Kalman Filtering Over Gaussian Channels with Feedback

Authors: Barron Han, Oron Sabag, Victoria Kostina, Babak Hassibi

Abstract: This paper investigates the problem of zero-delay joint source-channel coding of a vector Gauss-Markov source over a multiple-input multiple-output (MIMO) additive white Gaussian noise (AWGN) channel with feedback. In contrast to the classical problem of causal estimation using noisy observations, we examine a system where the source can be encoded before transmission. An encoder, equipped with fe… ▽ More This paper investigates the problem of zero-delay joint source-channel coding of a vector Gauss-Markov source over a multiple-input multiple-output (MIMO) additive white Gaussian noise (AWGN) channel with feedback. In contrast to the classical problem of causal estimation using noisy observations, we examine a system where the source can be encoded before transmission. An encoder, equipped with feedback of past channel outputs, observes the source state and encodes the information in a causal manner as inputs to the channel while adhering to a power constraint. The objective of the code is to estimate the source state with minimum mean square error at the infinite horizon. This work shows a fundamental theorem for two scenarios: for the transmission of an unstable vector Gauss-Markov source over either a multiple-input single-output (MISO) or a single-input multiple-output (SIMO) AWGN channel, finite estimation error is achievable if and only if the sum of logs of the unstable eigenvalues of the state gain matrix is less than the Shannon channel capacity. We prove these results by showing an optimal linear innovations encoder that can be applied to sources and channels of any dimension and analyzing it together with the corresponding Kalman filter decoder. △ Less

Submitted 18 October, 2023; originally announced October 2023.

Comments: Presented at 59th Allerton Conference on Communication, Control, and Computing

Variable-Length Codes with Bursty Feedback

Authors: James Y. Chen, Recep Can Yavas, Victoria Kostina

Abstract: We study variable-length codes for point-to-point discrete memoryless channels with noiseless unlimited-rate feedback that occurs in $L$ bursts. We term such codes variable-length bursty-feedback (VLBF) codes. Unlike classical codes with feedback after each transmitted code symbol, bursty feedback fits better with protocols that employ sparse feedback after a packet is sent and also with half-dupl… ▽ More We study variable-length codes for point-to-point discrete memoryless channels with noiseless unlimited-rate feedback that occurs in $L$ bursts. We term such codes variable-length bursty-feedback (VLBF) codes. Unlike classical codes with feedback after each transmitted code symbol, bursty feedback fits better with protocols that employ sparse feedback after a packet is sent and also with half-duplex end devices that cannot transmit and listen to the channel at the same time. We present a novel non-asymptotic achievability bound for VLBF codes with $L$ bursts of feedback over any discrete memoryless channel. We numerically evaluate the bound over the binary symmetric channel (BSC). We perform optimization over the time instances at which feedback occurs for both our own bound and Yavas et al.'s non-asymptotic achievability bound for variable-length stop-feedback (VLSF) codes, where only a single bit is sent at each feedback instance. Our results demonstrate the advantages of richer feedback: VLBF codes significantly outperform VLSF codes at short blocklengths, especially as the error probability $ε$ decreases. Remarkably, for BSC(0.11) and error probability $10^{-10}$, our VLBF code with $L=5$ and expected decoding time $N\leq 400$ outperforms the achievability bound given by Polyanskiy et al. for VLSF codes with $L=\infty$, and our VLBF code with $L=3$. △ Less

Submitted 23 June, 2023; originally announced June 2023.

Comments: Presented at ISIT 2023

Capacity of Finite-State Channels with Delayed Feedback

Authors: Bashar Huleihel, Oron Sabag, Haim H. Permuter, Victoria Kostina

Abstract: In this paper, we investigate the capacity of finite-state channels (FSCs) in presence of delayed feedback. We show that the capacity of a FSC with delayed feedback can be computed as that of a new FSC with instantaneous feedback and an extended state. Consequently, graph-based methods to obtain computable upper and lower bounds on the delayed feedback capacity of unifilar FSCs are proposed. Based… ▽ More In this paper, we investigate the capacity of finite-state channels (FSCs) in presence of delayed feedback. We show that the capacity of a FSC with delayed feedback can be computed as that of a new FSC with instantaneous feedback and an extended state. Consequently, graph-based methods to obtain computable upper and lower bounds on the delayed feedback capacity of unifilar FSCs are proposed. Based on these methods, we establish that the capacity of the trapdoor channel with delayed feedback of two time instances is given by $\log_2(3/2)$. In addition, we derive an analytical upper bound on the delayed feedback capacity of the binary symmetric channel with a no consecutive ones input constraint. This bound also serves as a novel upper bound on its non-feedback capacity, which outperforms all previously known bounds. Lastly, we demonstrate that feedback does improve the capacity of the dicode erasure channel. △ Less

Submitted 22 June, 2024; v1 submitted 31 March, 2023; originally announced March 2023.

Feedback capacity of Gaussian channels with memory

Authors: Oron Sabag, Victoria Kostina, Babak Hassibi

Abstract: We consider the feedback capacity of a MIMO channel whose channel output is given by a linear state-space model driven by the channel inputs and a Gaussian process. The generality of our state-space model subsumes all previous studied models such as additive channels with colored Gaussian noise, and channels with an arbitrary dependence on previous channel inputs or outputs. The main result is a c… ▽ More We consider the feedback capacity of a MIMO channel whose channel output is given by a linear state-space model driven by the channel inputs and a Gaussian process. The generality of our state-space model subsumes all previous studied models such as additive channels with colored Gaussian noise, and channels with an arbitrary dependence on previous channel inputs or outputs. The main result is a computable feedback capacity expression that is given as a convex optimization problem subject to a detectability condition. We demonstrate the capacity result on the auto-regressive Gaussian noise channel, where we show that even a single time-instance delay in the feedback reduces the feedback capacity significantly in the stationary regime. On the other hand, for large regression parameters (in the non-stationary regime), the feedback capacity can be approached with delayed feedback. Finally, we show that the detectability condition is satisfied for scalar models and conjecture that it is true for MIMO models. △ Less

Submitted 21 July, 2022; originally announced July 2022.

Comments: This paper was presented at the ISIT 2022

Incremental Redundancy With ACK/NACK Feedback at a Few Optimal Decoding Times

Authors: Hengjie Yang, Recep Can Yavas, Victoria Kostina, Richard D. Wesel

Abstract: Incremental redundancy with ACK/NACK feedback produces a variable-length stop-feedback (VLSF) code constrained to have $m$ decoding times, with an ACK/NACK feedback to the transmitter at each decoding time. This paper focuses on the numerical evaluation of the maximal achievable rate of random VLSF codes as a function of $m$ for the binary-input additive white Gaussian noise channel, binary symmet… ▽ More Incremental redundancy with ACK/NACK feedback produces a variable-length stop-feedback (VLSF) code constrained to have $m$ decoding times, with an ACK/NACK feedback to the transmitter at each decoding time. This paper focuses on the numerical evaluation of the maximal achievable rate of random VLSF codes as a function of $m$ for the binary-input additive white Gaussian noise channel, binary symmetric channel, and binary erasure channel (BEC). Leveraging Edgeworth and Petrov expansions, we develop tight approximations to the tail probability of length-$n$ cumulative information density that are accurate for any blocklength $n$. We reduce Yavas et al.'s non-asymptotic achievability bound on VLSF codes with $m$ decoding times to an integer program of minimizing the upper bound on the average blocklength subject to the average error probability, minimum gap, and integer constraints. We develop two distinct methods to solve this program. Numerical evaluations show that Polyanskiy's achievability bound for VLSF codes, which assumes $m = \infty$, can be approached with a small $m$ for all three channels. For BEC, we consider systematic transmission followed by random linear fountain coding. This allows us to obtain a new achievability bound stronger than a previous bound and new VLSF codes whose rate further outperforms Polyanskiy's bound. △ Less

Submitted 4 February, 2023; v1 submitted 30 May, 2022; originally announced May 2022.

Comments: 19 pages, 12 figures; submitted to IEEE Transactions on Communications. An earlier version of this work was published at ISIT 2022. Comments are welcome!

Third-order Analysis of Channel Coding in the Small-to-Moderate Deviations Regime

Authors: Recep Can Yavas, Victoria Kostina, Michelle Effros

Abstract: This paper studies the third-order characteristic of nonsingular discrete memoryless channels and the Gaussian channel with a maximal power constraint. The third-order term in our expansions employs a new quantity here called the \emph{channel skewness}, which affects the approximation accuracy more significantly as the error probability decreases. For the Gaussian channel, evaluating Shannon's (1… ▽ More This paper studies the third-order characteristic of nonsingular discrete memoryless channels and the Gaussian channel with a maximal power constraint. The third-order term in our expansions employs a new quantity here called the \emph{channel skewness}, which affects the approximation accuracy more significantly as the error probability decreases. For the Gaussian channel, evaluating Shannon's (1959) random coding and sphere-packing bounds in the central limit theorem (CLT) regime enables exact computation of the channel skewness. For discrete memoryless channels, this work generalizes Moulin's (2017) bounds on the asymptotic expansion of the maximum achievable message set size for nonsingular channels from the CLT regime to include the moderate deviations (MD) regime, thereby refining Altuğ and Wagner's (2014) MD result. For an example binary symmetric channel and most practically important $(n, ε)$ pairs, including $n \in [100, 500]$ and $ε\in [10^{-10}, 10^{-1}]$, an approximation up to the channel skewness is the most accurate among several expansions in the literature. A derivation of the third-order term in the type-II error exponent of binary hypothesis testing in the MD regime is also included; the resulting third-order term is similar to the channel skewness. △ Less

Submitted 21 March, 2023; v1 submitted 2 March, 2022; originally announced March 2022.

Comments: Presented at ISIT 2022. Submitted to IT Transactions in January, 2023

MSC Class: 94A24 ACM Class: E.4

Reliability function for streaming over a DMC with feedback

Authors: Nian Guo, Victoria Kostina

Abstract: Conventionally, posterior matching is investigated in channel coding and block encoding contexts -- the source symbols are equiprobably distributed and are entirely known by the encoder before the transmission. In this paper, we consider a streaming source, whose symbols arrive at the encoder at a sequence of deterministic times. We derive the joint source-channel coding (JSCC) reliability functio… ▽ More Conventionally, posterior matching is investigated in channel coding and block encoding contexts -- the source symbols are equiprobably distributed and are entirely known by the encoder before the transmission. In this paper, we consider a streaming source, whose symbols arrive at the encoder at a sequence of deterministic times. We derive the joint source-channel coding (JSCC) reliability function for streaming over a discrete memoryless channel (DMC) with feedback. We propose a novel instantaneous encoding phase that operates during the symbol arriving period and achieves the JSCC reliability function for streaming when followed by a block encoding scheme that achieves the JSCC reliability function for a classical source whose symbols are fully accessible before the transmission. During the instantaneous encoding phase, the evolving message alphabet is partitioned into groups, and the encoder determines the index of the group that contains the symbols arrived so far and applies randomization to match the distribution of the transmitted index to the capacity-achieving one. Surprisingly, the JSCC reliability function for streaming is equal to that for a fully accessible source, implying that the knowledge of the entire symbol sequence before the transmission offers no advantage regarding the reliability function. For streaming over a symmetric 2-input DMC, we propose an instantaneous small-enough difference (SED) code that not only achieves the JSCC reliability function but also can be used to stabilize an unstable linear system over a noisy channel. We design low complexity algorithms to implement both the instantaneous encoding phase and the instantaneous SED code. While the reliability function is derived for non-degenerate DMCs, for degenerate DMCs we design a code with instantaneous encoding that achieves zero error for all rates below Shannon's JSCC limit. △ Less

Submitted 30 November, 2022; v1 submitted 11 February, 2022; originally announced February 2022.

Variable-Length Stop-Feedback Codes With Finite Optimal Decoding Times for BI-AWGN Channels

Authors: Hengjie Yang, Recep Can Yavas, Victoria Kostina, Richard D. Wesel

Abstract: In this paper, we are interested in the performance of a variable-length stop-feedback (VLSF) code with $m$ optimal decoding times for the binary-input additive white Gaussian noise channel. We first develop tight approximations on the tail probability of length-$n$ cumulative information density. Building on the work of Yavas \emph{et al.}, for a given information density threshold, we formulate… ▽ More In this paper, we are interested in the performance of a variable-length stop-feedback (VLSF) code with $m$ optimal decoding times for the binary-input additive white Gaussian noise channel. We first develop tight approximations on the tail probability of length-$n$ cumulative information density. Building on the work of Yavas \emph{et al.}, for a given information density threshold, we formulate the integer program of minimizing the upper bound on average blocklength over all decoding times subject to the average error probability, minimum gap and integer constraints. Eventually, minimization of locally minimum upper bounds over all thresholds will yield the globally minimum upper bound and this is called the two-step minimization. For the integer program, we present a greedy algorithm that yields possibly suboptimal integer decoding times. By allowing a positive real-valued decoding time, we develop the gap-constrained sequential differential optimization (SDO) procedure that sequentially produces the optimal, real-valued decoding times. We identify the error regime in which Polyanskiy's scheme of stop** at zero does not improve the achievability bound. In this error regime, the two-step minimization with the gap-constrained SDO shows that a finite $m$ suffices to attain Polyanskiy's bound for VLSF codes with $m = \infty$. △ Less

Submitted 5 February, 2022; v1 submitted 27 January, 2022; originally announced January 2022.

Comments: 8 pages, 4 figures; fixed some errors in v1 and added some new results; a short version of this preprint was submitted to ISIT 2022

How to Query An Oracle? Efficient Strategies to Label Data

Authors: Farshad Lahouti, Victoria Kostina, Babak Hassibi

Abstract: We consider the basic problem of querying an expert oracle for labeling a dataset in machine learning. This is typically an expensive and time consuming process and therefore, we seek ways to do so efficiently. The conventional approach involves comparing each sample with (the representative of) each class to find a match. In a setting with $N$ equally likely classes, this involves $N/2$ pairwise… ▽ More We consider the basic problem of querying an expert oracle for labeling a dataset in machine learning. This is typically an expensive and time consuming process and therefore, we seek ways to do so efficiently. The conventional approach involves comparing each sample with (the representative of) each class to find a match. In a setting with $N$ equally likely classes, this involves $N/2$ pairwise comparisons (queries per sample) on average. We consider a $k$-ary query scheme with $k\ge 2$ samples in a query that identifies (dis)similar items in the set while effectively exploiting the associated transitive relations. We present a randomized batch algorithm that operates on a round-by-round basis to label the samples and achieves a query rate of $O(\frac{N}{k^2})$. In addition, we present an adaptive greedy query scheme, which achieves an average rate of $\approx 0.2N$ queries per sample with triplet queries. For the proposed algorithms, we investigate the query rate performance analytically and with simulations. Empirical studies suggest that each triplet query takes an expert at most 50\% more time compared with a pairwise query, indicating the effectiveness of the proposed $k$-ary query schemes. We generalize the analyses to nonuniform class distributions when possible. △ Less

Submitted 5 October, 2021; originally announced October 2021.

Comments: To Appear in IEEE Transactions on Pattern Analysis and Machine Intelligence

Reducing the LQG Cost with Minimal Communication

Authors: Oron Sabag, Peida Tian, Victoria Kostina, Babak Hassibi

Abstract: We study the linear quadratic Gaussian (LQG) control problem, in which the controller's observation of the system state is such that a desired cost is unattainable. To achieve the desired LQG cost, we introduce a communication link from the observer (encoder) to the controller. We investigate the optimal trade-off between the improved LQG cost and the consumed communication (information) resources… ▽ More We study the linear quadratic Gaussian (LQG) control problem, in which the controller's observation of the system state is such that a desired cost is unattainable. To achieve the desired LQG cost, we introduce a communication link from the observer (encoder) to the controller. We investigate the optimal trade-off between the improved LQG cost and the consumed communication (information) resources, measured with the conditional directed information, across all encoding-decoding policies. The main result is a semidefinite programming formulation for that optimization problem in the finite-horizon scenario, which applies to time-varying linear dynamical systems. This result extends a seminal work by Tanaka et al., where the only information the controller knows about the system state arrives via a communication channel, to the scenario where the controller has also access to a noisy observation of the system state. As part of our derivation to show the optimiality of an encoder that transmits a memoryless Gaussian measurement of the state, we show that the presence of the controller's observations at the encoder can not reduce the minimal directed information. For time-invariant systems, where the optimal policy may be time-varying, we show in the infinite-horizon scenario that the optimal policy is time-invariant and can be computed explicitly from a solution of a finite-dimensional semidefinite programming. The results are demonstrated via examples that show that even low-quality measurements can have a significant impact on the required communication resources. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Comments: A short version published in https://ieeexplore.ieee.org/document/9304490

Feedback Capacity of MIMO Gaussian Channels

Authors: Oron Sabag, Victoria Kostina, Babak Hassibi

Abstract: Finding a computable expression for the feedback capacity of channels with colored Gaussian, additive noise is a long standing open problem. In this paper, we solve this problem in the scenario where the channel has multiple inputs and multiple outputs (MIMO) and the noise process is generated as the output of a time-invariant state-space model. Our main result is a computable expression for the f… ▽ More Finding a computable expression for the feedback capacity of channels with colored Gaussian, additive noise is a long standing open problem. In this paper, we solve this problem in the scenario where the channel has multiple inputs and multiple outputs (MIMO) and the noise process is generated as the output of a time-invariant state-space model. Our main result is a computable expression for the feedback capacity in terms of a finite-dimensional convex optimization. The solution to the feedback capacity problem is obtained by formulating the finite-block counterpart of the capacity problem as a \emph{sequential convex optimization problem} which leads in turn to a single-letter upper bound. This converse derivation integrates tools and ideas from information theory, control, filtering and convex optimization. A tight lower bound is realized by optimizing over a family of time-invariant policies thus showing that time-invariant inputs are optimal even when the noise process may not be stationary. The optimal time-invariant policy is used to construct a capacity-achieving and simple coding scheme for scalar channels, and its analysis reveals an interesting relation between a smoothing problem and the feedback capacity expression. △ Less

Submitted 19 January, 2023; v1 submitted 3 June, 2021; originally announced June 2021.

Variable-Length Sparse Feedback Codes for Point-to-Point, Multiple Access, and Random Access Channels

Authors: Recep Can Yavas, Victoria Kostina, Michelle Effros

Abstract: This paper investigates variable-length stop-feedback codes for memoryless channels in point-to-point, multiple access, and random access communication scenarios. The proposed codes employ $L$ decoding times $n_1, n_2, \dots, n_L$ for the point-to-point and multiple access channels and $KL + 1$ decoding times for the random access channel with at most $K$ active transmitters. In the point-to-point… ▽ More This paper investigates variable-length stop-feedback codes for memoryless channels in point-to-point, multiple access, and random access communication scenarios. The proposed codes employ $L$ decoding times $n_1, n_2, \dots, n_L$ for the point-to-point and multiple access channels and $KL + 1$ decoding times for the random access channel with at most $K$ active transmitters. In the point-to-point and multiple access channels, the decoder uses the observed channel outputs to decide whether to decode at each of the allowed decoding times $n_1, \dots, n_L$, at each time telling the encoder whether or not to stop transmitting using a single bit of feedback. In the random access scenario, the decoder estimates the number of active transmitters at time $n_0$ and then chooses among decoding times $n_{k, 1}, \dots, n_{k, L}$ if it believes that there are $k$ active transmitters. In all cases, the choice of allowed decoding times is part of the code design; given fixed value $L$, allowed decoding times are chosen to minimize the expected decoding time for a given codebook size and target average error probability. The number $L$ in each scenario is assumed to be constant even when the blocklength is allowed to grow; the resulting code therefore requires only sparse feedback. The central results are asymptotic approximations of achievable rates as a function of the error probability, the expected decoding time, and the number of decoding times. A converse for variable-length stop-feedback codes with uniformly-spaced decoding times is included for the point-to-point channel. △ Less

Submitted 8 December, 2023; v1 submitted 16 March, 2021; originally announced March 2021.

Comments: 29 Pages. Presented at ISIT 2021. Accepted for publication at IEEE Transactions on Information Theory Dec. 2023

MSC Class: 94A15 ACM Class: E.4

Instantaneous SED coding over a DMC

Authors: Nian Guo, Victoria Kostina

Abstract: In this paper, we propose a novel code for transmitting a sequence of $n$ message bits in real time over a discrete-memoryless channel (DMC) with noiseless feedback, where the message bits stream into the encoder one by one at random time instants. Similar to existing posterior matching schemes with block encoding, the encoder in our work takes advantage of the channel feedback to form channel inp… ▽ More In this paper, we propose a novel code for transmitting a sequence of $n$ message bits in real time over a discrete-memoryless channel (DMC) with noiseless feedback, where the message bits stream into the encoder one by one at random time instants. Similar to existing posterior matching schemes with block encoding, the encoder in our work takes advantage of the channel feedback to form channel inputs that contain the information the decoder does not yet have, and that are distributed close to the capacity-achieving input distribution, but dissimilar to the existing posterior matching schemes, the encoder performs instantaneous encoding--it immediately weaves the new message bits into a continuing transmission. A posterior matching scheme by Naghshvar et al. partitions the source messages into groups so that the group posteriors have a small-enough difference (SED) to the capacity-achieving distribution, and transmits the group index that contains the actual message. Our code adopts the SED rule to apply to the evolving message alphabet that contains all the possible variable-length strings that the source could have emitted up to that time. Our instantaneous SED code achieves better delay-reliability tradeoffs than existing feedback codes over $2$-input DMCs: we establish this dominance both by simulations and via an analysis comparing the performance of the instantaneous SED code to Burnashev's reliability function. We also design a low-complexity code for binary symmetric channels that we name the instantaneous type set SED code with complexity $O(t^4)$. Simulation results show that the gap in performance between the instantaneous SED code and the instantaneous type-set SED code is negligible. △ Less

Submitted 6 May, 2021; v1 submitted 14 March, 2021; originally announced March 2021.

Differentially Quantized Gradient Methods

Authors: Chung-Yi Lin, Victoria Kostina, Babak Hassibi

Abstract: Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay constraints. The server receives all its information about the problem instance from the worker via a rate-limited noiseless communication channel. We introduce the princi… ▽ More Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay constraints. The server receives all its information about the problem instance from the worker via a rate-limited noiseless communication channel. We introduce the principle we call Differential Quantization (DQ) that prescribes compensating the past quantization errors to direct the descent trajectory of a quantized algorithm towards that of its unquantized counterpart. Assuming that the objective function is smooth and strongly convex, we prove that Differentially Quantized Gradient Descent (DQ-GD) attains a linear contraction factor of $\max\{σ_{\mathrm{GD}}, ρ_n 2^{-R}\}$, where $σ_{\mathrm{GD}}$ is the contraction factor of unquantized gradient descent (GD), $ρ_n \geq 1$ is the covering efficiency of the quantizer, and $R$ is the bitrate per problem dimension $n$. Thus at any $R\geq\log_2 ρ_n /σ_{\mathrm{GD}}$ bits, the contraction factor of DQ-GD is the same as that of unquantized GD, i.e., there is no loss due to quantization. We show that no algorithm within a certain class can converge faster than $\max\{σ_{\mathrm{GD}}, 2^{-R}\}$. Since quantizers exist with $ρ_n \to 1$ as $n \to \infty$ (Rogers, 1963), this means that DQ-GD is asymptotically optimal. The principle of differential quantization continues to apply to gradient methods with momentum such as Nesterov's accelerated gradient descent, and Polyak's heavy ball method. For these algorithms as well, if the rate is above a certain threshold, there is no loss in contraction factor obtained by the differentially quantized algorithm compared to its unquantized counterpart. Experimental results on least-squares problems validate our theoretical analysis. △ Less

Submitted 26 April, 2022; v1 submitted 6 February, 2020; originally announced February 2020.

Journal ref: IEEE Transactions of Information Theory, Apr. 2022

Optimal Causal Rate-Constrained Sampling for a Class of Continuous Markov Processes

Authors: Nian Guo, Victoria Kostina

Abstract: Consider the following communication scenario. An encoder observes a stochastic process and causally decides when and what to transmit about it, under a constraint on the expected number of bits transmitted per second. A decoder uses the received codewords to causally estimate the process in real time. The encoder and the decoder are synchronized in time. For a class of continuous Markov processes… ▽ More Consider the following communication scenario. An encoder observes a stochastic process and causally decides when and what to transmit about it, under a constraint on the expected number of bits transmitted per second. A decoder uses the received codewords to causally estimate the process in real time. The encoder and the decoder are synchronized in time. For a class of continuous Markov processes satisfying regularity conditions, we find the optimal encoding and decoding policies that minimize the end-to-end estimation mean-square error under the rate constraint. We show that the optimal encoding policy transmits a $1$-bit codeword once the process innovation passes one of two thresholds. The optimal decoder noiselessly recovers the last sample from the 1-bit codewords and codeword-generating time stamps, and uses it to decide the running estimate of the current process, until the next codeword arrives. In particular, we show the optimal causal code for the Ornstein-Uhlenbeck process and calculate its distortion-rate function. Furthermore, we show that the optimal causal code also minimizes the mean-square cost of a continuous-time control system driven by a continuous Markov process and controlled by an additive control signal. △ Less

Submitted 20 September, 2021; v1 submitted 4 February, 2020; originally announced February 2020.

Gaussian Multiple and Random Access in the Finite Blocklength Regime

Authors: Recep Can Yavas, Victoria Kostina, Michelle Effros

Abstract: This paper presents finite-blocklength achievability bounds for the Gaussian multiple access channel (MAC) and random access channel (RAC) under average-error and maximal-power constraints. Using random codewords uniformly distributed on a sphere and a maximum likelihood decoder, the derived MAC bound on each transmitter's rate matches the MolavianJazi-Laneman bound (2015) in its first- and second… ▽ More This paper presents finite-blocklength achievability bounds for the Gaussian multiple access channel (MAC) and random access channel (RAC) under average-error and maximal-power constraints. Using random codewords uniformly distributed on a sphere and a maximum likelihood decoder, the derived MAC bound on each transmitter's rate matches the MolavianJazi-Laneman bound (2015) in its first- and second-order terms, improving the remaining terms to $\frac12\frac{\log n}{n}+O \left(\frac 1 n \right)$ bits per channel use. The result then extends to a RAC model in which neither the encoders nor the decoder knows which of $K$ possible transmitters are active. In the proposed rateless coding strategy, decoding occurs at a time $n_t$ that depends on the decoder's estimate $t$ of the number of active transmitters $k$. Single-bit feedback from the decoder to all encoders at each potential decoding time $n_i$, $i \leq t$, informs the encoders when to stop transmitting. For this RAC model, the proposed code achieves the same first-, second-, and third-order performance as the best known result for the Gaussian MAC in operation. △ Less

Submitted 3 May, 2022; v1 submitted 12 January, 2020; originally announced January 2020.

Comments: 27 pages, IEEE Transactions on Information Theory, ISIT 2020

ACM Class: E.4

Journal ref: IEEE Transactions on Information Theory, vol. 67, no. 11, pp. 6983-7009, Nov. 2021

The CEO problem with inter-block memory

Authors: Victoria Kostina, Babak Hassibi

Abstract: An $n$-dimensional source with memory is observed by $K$ isolated encoders via parallel channels, who compress their observations to transmit to the decoder via noiseless rate-constrained links while leveraging their memory of the past. At each time instant, the decoder receives $K$ new codewords from the observers, combines them with the past received codewords, and produces a minimum-distortion… ▽ More An $n$-dimensional source with memory is observed by $K$ isolated encoders via parallel channels, who compress their observations to transmit to the decoder via noiseless rate-constrained links while leveraging their memory of the past. At each time instant, the decoder receives $K$ new codewords from the observers, combines them with the past received codewords, and produces a minimum-distortion estimate of the latest block of $n$ source symbols. This scenario extends the classical one-shot CEO problem to multiple rounds of communication with communicators maintaining the memory of the past. We extend the Berger-Tung inner and outer bounds to the scenario with inter-block memory, showing that the minimum asymptotically (as $n \to \infty$) achievable sum rate required to achieve a target distortion is bounded by minimal directed mutual information problems. For the Gauss-Markov source observed via $K$ parallel AWGN channels, we show that the inner bound is tight and solve the corresponding minimal directed mutual information problem, thereby establishing the minimum asymptotically achievable sum rate. Finally, we explicitly bound the rate loss due to a lack of communication among the observers; that bound is attained with equality in the case of identical observation channels. The general coding theorem is proved via a new nonasymptotic bound that uses stochastic likelihood coders and whose asymptotic analysis yields an extension of the Berger-Tung inner bound to the causal setting. The analysis of the Gaussian case is facilitated by reversing the channels of the observers. △ Less

Submitted 23 November, 2021; v1 submitted 6 October, 2019; originally announced October 2019.

Journal ref: IEEE Transactions on Information Theory, v. 67, No. 12, pp. 7752--7768, Dec. 2021

Optimal Causal Rate-Constrained Sampling of the Wiener Process

Authors: Nian Guo, Victoria Kostina

Abstract: We consider the following communication scenario. An encoder causally observes the Wiener process and decides when and what to transmit about it. A decoder makes real-time estimation of the process using causally received codewords. We determine the causal encoding and decoding policies that jointly minimize the mean-square estimation error, under the long-term communication rate constraint of… ▽ More We consider the following communication scenario. An encoder causally observes the Wiener process and decides when and what to transmit about it. A decoder makes real-time estimation of the process using causally received codewords. We determine the causal encoding and decoding policies that jointly minimize the mean-square estimation error, under the long-term communication rate constraint of $R$ bits per second. We show that an optimal encoding policy can be implemented as a causal sampling policy followed by a causal compressing policy. We prove that the optimal encoding policy samples the Wiener process once the innovation passes either $\sqrt{\frac{1}{R}}$ or $-\sqrt{\frac{1}{R}}$, and compresses the sign of the innovation (SOI) using a 1-bit codeword. The SOI coding scheme achieves the operational distortion-rate function, which is equal to $D^{\mathrm{op}}(R)=\frac{1}{6R}$. Surprisingly, this is significantly better than the distortion-rate tradeoff achieved in the limit of infinite delay by the best non-causal code. This is because the SOI coding scheme leverages the free timing information supplied by the zero-delay channel between the encoder and the decoder. The key to unlock that gain is the event-triggered nature of the SOI sampling policy. In contrast, the distortion-rate tradeoffs achieved with deterministic sampling policies are much worse: we prove that the causal informational distortion-rate function in that scenario is as high as $D_{\mathrm{DET}}(R) = \frac{5}{6R}$. It is achieved by the uniform sampling policy with the sampling interval $\frac{1}{R}$. In either case, the optimal strategy is to sample the process as fast as possible and to transmit 1-bit codewords to the decoder without delay. △ Less

Submitted 13 May, 2020; v1 submitted 3 September, 2019; originally announced September 2019.

Two-layer Coded Channel Access with Collision Resolution: Design and Analysis

Authors: MohammadReza Ebrahimi, Farshad Lahouti, Victoria Kostina

Abstract: We propose a two-layer coding architecture for communication of multiple users over a shared slotted medium enabling joint collision resolution and decoding. Each user first encodes its information bits with an outer code for reliability, and then transmits these coded bits with possible repetitions over transmission time slots of the access channel. The transmission patterns are dictated by the i… ▽ More We propose a two-layer coding architecture for communication of multiple users over a shared slotted medium enabling joint collision resolution and decoding. Each user first encodes its information bits with an outer code for reliability, and then transmits these coded bits with possible repetitions over transmission time slots of the access channel. The transmission patterns are dictated by the inner collision-resolution code and collisions with other users' transmissions may occur. We analyze two types of codes for the outer layer: long-blocklength LDPC codes, and short-blocklength algebraic codes. With LDPC codes, a density evolution analysis enables joint optimization of both outer and inner code parameters for maximum throughput. With algebraic codes, we invoke a similar analysis by approximating their average erasure correcting capability while assuming a large number of active transmitters. The proposed low-complexity schemes operate at a significantly smaller gap to capacity than the state of the art. Our schemes apply both to a multiple access scenario where number of users within a frame is known a priori, and to a random access scenario where that number is known only to the decoder. In the latter case, we optimize an outage probability due to the variability in user activity. △ Less

Submitted 17 August, 2020; v1 submitted 30 August, 2019; originally announced September 2019.

Nonstationary Gauss-Markov Processes: Parameter Estimation and Dispersion

Authors: Peida Tian, Victoria Kostina

Abstract: This paper provides a precise error analysis for the maximum likelihood estimate $\hat{a}_{\text{ML}}(u_1^n)$ of the parameter $a$ given samples $u_1^n = (u_1, \ldots, u_n)'$ drawn from a nonstationary Gauss-Markov process $U_i = a U_{i-1} + Z_i,~i\geq 1$, where $U_0 = 0$, $a> 1$, and $Z_i$'s are independent Gaussian random variables with zero mean and variance $σ^2$. We show a tight nonasymptotic… ▽ More This paper provides a precise error analysis for the maximum likelihood estimate $\hat{a}_{\text{ML}}(u_1^n)$ of the parameter $a$ given samples $u_1^n = (u_1, \ldots, u_n)'$ drawn from a nonstationary Gauss-Markov process $U_i = a U_{i-1} + Z_i,~i\geq 1$, where $U_0 = 0$, $a> 1$, and $Z_i$'s are independent Gaussian random variables with zero mean and variance $σ^2$. We show a tight nonasymptotic exponentially decaying bound on the tail probability of the estimation error. Unlike previous works, our bound is tight already for a sample size of the order of hundreds. We apply the new estimation bound to find the dispersion for lossy compression of nonstationary Gauss-Markov sources. We show that the dispersion is given by the same integral formula that we derived previously for the asymptotically stationary Gauss-Markov sources, i.e., $|a| < 1$. New ideas in the nonstationary case include separately bounding the maximum eigenvalue (which scales exponentially) and the other eigenvalues (which are bounded by constants that depend only on $a$) of the covariance matrix of the source sequence, and new techniques in the derivation of our estimation error bound. △ Less

Submitted 23 March, 2021; v1 submitted 29 June, 2019; originally announced July 2019.

Comments: 25 pages, 5 figures

Journal ref: in IEEE Transactions on Information Theory, vol. 67, no. 4, pp. 2426-2449, April 2021

Lossless Source Coding in the Point-to-Point, Multiple Access, and Random Access Scenarios

Authors: Shuqing Chen, Michelle Effros, Victoria Kostina

Abstract: This work studies point-to-point, multiple access, and random access lossless source coding in the finite-blocklength regime. In each scenario, a random coding technique is developed and used to analyze third-order coding performance. Asymptotic results include a third-order characterization of the Slepian-Wolf rate region with an improved converse that relies on a connection to composite hypothes… ▽ More This work studies point-to-point, multiple access, and random access lossless source coding in the finite-blocklength regime. In each scenario, a random coding technique is developed and used to analyze third-order coding performance. Asymptotic results include a third-order characterization of the Slepian-Wolf rate region with an improved converse that relies on a connection to composite hypothesis testing. For dependent sources, the result implies that the independent encoders used by Slepian-Wolf codes can achieve the same third-order-optimal performance as a single joint encoder. The concept of random access source coding is introduced to generalize multiple access (Slepian-Wolf) source coding to the case where encoders decide independently whether or not to participate and the set of participating encoders is unknown {\em a priori} to both the encoders and the decoder. The proposed random access source coding strategy employs rateless coding with scheduled feedback. A random coding argument proves the existence of a single deterministic code of this structure that simultaneously achieves the third-order-optimal Slepian-Wolf performance for each possible active encoder set. △ Less

Submitted 9 October, 2020; v1 submitted 8 February, 2019; originally announced February 2019.

Comments: 35 pages, 10 figures

Journal ref: IEEE Transactions on Information Theory, 2020

The Dispersion of the Gauss-Markov Source

Authors: Peida Tian, Victoria Kostina

Abstract: The Gauss-Markov source produces $U_i = aU_{i-1} + Z_i$ for $i\geq 1$, where $U_0 = 0$, $|a|<1$ and $Z_i\sim\mathcal{N}(0, σ^2)$ are i.i.d. Gaussian random variables. We consider lossy compression of a block of $n$ samples of the Gauss-Markov source under squared error distortion. We obtain the Gaussian approximation for the Gauss-Markov source with excess-distortion criterion for any distortion… ▽ More The Gauss-Markov source produces $U_i = aU_{i-1} + Z_i$ for $i\geq 1$, where $U_0 = 0$, $|a|<1$ and $Z_i\sim\mathcal{N}(0, σ^2)$ are i.i.d. Gaussian random variables. We consider lossy compression of a block of $n$ samples of the Gauss-Markov source under squared error distortion. We obtain the Gaussian approximation for the Gauss-Markov source with excess-distortion criterion for any distortion $d>0$, and we show that the dispersion has a reverse waterfilling representation. This is the \emph{first} finite blocklength result for lossy compression of \emph{sources with memory}. We prove that the finite blocklength rate-distortion function $R(n,d,ε)$ approaches the rate-distortion function $\mathbb{R}(d)$ as $R(n,d,ε) = \mathbb{R}(d) + \sqrt{\frac{V(d)}{n}}Q^{-1}(ε) + o\left(\frac{1}{\sqrt{n}}\right)$, where $V(d)$ is the dispersion, $ε\in (0,1)$ is the excess-distortion probability, and $Q^{-1}$ is the inverse of the $Q$-function. We give a reverse waterfilling integral representation for the dispersion $V(d)$, which parallels that of the rate-distortion functions for Gaussian processes. Remarkably, for all $0 < d\leq \frac{σ^2}{(1+|a|)^2}$, $R(n,d,ε)$ of the Gauss-Markov source coincides with that of $Z_k$, the i.i.d. Gaussian noise driving the process, up to the second-order term. Among novel technical tools developed in this paper is a sharp approximation of the eigenvalues of the covariance matrix of $n$ samples of the Gauss-Markov source, and a construction of a typical set using the maximum likelihood estimate of the parameter $a$ based on $n$ observations. △ Less

Submitted 24 September, 2019; v1 submitted 25 April, 2018; originally announced April 2018.

Comments: 30 pages, 8 figures, shorter version published in the proceedings of IEEE ISIT 2018

Journal ref: in IEEE Transactions on Information Theory, vol. 65, no. 10, pp. 6355-6384, Oct. 2019

The Birthday Problem and Zero-Error List Codes

Authors: Parham Noorzad, Michelle Effros, Michael Langberg, Victoria Kostina

Abstract: As an attempt to bridge the gap between the probabilistic world of classical information theory and the combinatorial world of zero-error information theory, this paper studies the performance of randomly generated codebooks over discrete memoryless channels under a zero-error list-decoding constraint. This study allows the application of tools from one area to the other. Furthermore, it leads to… ▽ More As an attempt to bridge the gap between the probabilistic world of classical information theory and the combinatorial world of zero-error information theory, this paper studies the performance of randomly generated codebooks over discrete memoryless channels under a zero-error list-decoding constraint. This study allows the application of tools from one area to the other. Furthermore, it leads to an information-theoretic formulation of the birthday problem, which is concerned with the probability that in a given population, a fixed number of people have the same birthday. Due to the lack of a closed-form expression for this probability when the distribution of birthdays is not uniform, the resulting expression is not simple to analyze; in the information-theoretic formulation, however, the asymptotic behavior of this probability can be characterized exactly for all distributions. △ Less

Submitted 8 December, 2018; v1 submitted 13 February, 2018; originally announced February 2018.

Comments: Extended version of paper presented at ISIT 2017 in Aachen

Random Access Channel Coding in the Finite Blocklength Regime

Authors: Recep Can Yavas, Victoria Kostina, Michelle Effros

Abstract: Consider a random access communication scenario over a channel whose operation is defined for any number of possible transmitters. As in the model recently introduced by Polyanskiy for the Multiple Access Channel (MAC) with a fixed, known number of transmitters, the channel is assumed to be invariant to permutations on its inputs, and all active transmitters employ identical encoders. Unlike the P… ▽ More Consider a random access communication scenario over a channel whose operation is defined for any number of possible transmitters. As in the model recently introduced by Polyanskiy for the Multiple Access Channel (MAC) with a fixed, known number of transmitters, the channel is assumed to be invariant to permutations on its inputs, and all active transmitters employ identical encoders. Unlike the Polyanskiy model, in the proposed scenario, neither the transmitters nor the receiver knows which transmitters are active. We refer to this agnostic communication setup as the Random Access Channel (RAC). Scheduled feedback of a finite number of bits is used to synchronize the transmitters. The decoder is tasked with determining from the channel output the number of active transmitters, $k$, and their messages but not which transmitter sent which message. The decoding procedure occurs at a time $n_t$ depending on the decoder's estimate, $t$, of the number of active transmitters, $k$, thereby achieving a rate that varies with the number of active transmitters. Single-bit feedback at each time $n_i, i \leq t$, enables all transmitters to determine the end of one coding epoch and the start of the next. The central result of this work demonstrates the achievability on a RAC of performance that is first-order optimal for the MAC in operation during each coding epoch. While prior multiple access schemes for a fixed number of transmitters require $2^k - 1$ simultaneous threshold rules, the proposed scheme uses a single threshold rule and achieves the same dispersion. △ Less

Submitted 16 March, 2021; v1 submitted 26 January, 2018; originally announced January 2018.

Comments: Presented at ISIT 2018. IEEE Transactions on Information Theory

MSC Class: 94A24 ACM Class: E.4

Journal ref: IEEE Transactions on Information Theory, vol. 67, no. 4, pp. 2115-2140, April 2021

Successive Refinement of Abstract Sources

Authors: Victoria Kostina, Ertem Tuncel

Abstract: In successive refinement of information, the decoder refines its representation of the source progressively as it receives more encoded bits. The rate-distortion region of successive refinement describes the minimum rates required to attain the target distortions at each decoding stage. In this paper, we derive a parametric characterization of the rate-distortion region for successive refinement o… ▽ More In successive refinement of information, the decoder refines its representation of the source progressively as it receives more encoded bits. The rate-distortion region of successive refinement describes the minimum rates required to attain the target distortions at each decoding stage. In this paper, we derive a parametric characterization of the rate-distortion region for successive refinement of abstract sources. Our characterization extends Csiszar's result to successive refinement, and generalizes a result by Tuncel and Rose, applicable for finite alphabet sources, to abstract sources. This characterization spawns a family of outer bounds to the rate-distortion region. It also enables an iterative algorithm for computing the rate-distortion region, which generalizes Blahut's algorithm to successive refinement. Finally, it leads a new nonasymptotic converse bound. In all the scenarios where the dispersion is known, this bound is second-order optimal. In our proof technique, we avoid Karush-Kuhn-Tucker conditions of optimality, and we use basic tools of probability theory. We leverage the Donsker-Varadhan lemma for the minimization of relative entropy on abstract probability spaces. △ Less

Submitted 20 November, 2018; v1 submitted 29 July, 2017; originally announced July 2017.

Comments: Extended version of a paper presented at ISIT 2017

A lower bound on the differential entropy of log-concave random vectors with applications

Authors: Arnaud Marsiglietti, Victoria Kostina

Abstract: We derive a lower bound on the differential entropy of a log-concave random variable $X$ in terms of the $p$-th absolute moment of $X$. The new bound leads to a reverse entropy power inequality with an explicit constant, and to new bounds on the rate-distortion function and the channel capacity. Specifically, we study the rate-distortion function for log-concave sources and distortion measure… ▽ More We derive a lower bound on the differential entropy of a log-concave random variable $X$ in terms of the $p$-th absolute moment of $X$. The new bound leads to a reverse entropy power inequality with an explicit constant, and to new bounds on the rate-distortion function and the channel capacity. Specifically, we study the rate-distortion function for log-concave sources and distortion measure $| x - \hat x|^r$, and we establish that the difference between the rate distortion function and the Shannon lower bound is at most $\log(\sqrt{πe}) \approx 1.5$ bits, independently of $r$ and the target distortion $d$. For mean-square error distortion, the difference is at most $\log (\sqrt{\frac{πe}{2}}) \approx 1$ bits, regardless of $d$. We also provide bounds on the capacity of memoryless additive noise channels when the noise is log-concave. We show that the difference between the capacity of such channels and the capacity of the Gaussian channel with the same noise power is at most $\log (\sqrt{\frac{πe}{2}}) \approx 1$ bits. Our results generalize to the case of vector $X$ with possibly dependent coordinates, and to $γ$-concave random variables. Our proof technique leverages tools from convex geometry. △ Less

Submitted 31 August, 2017; v1 submitted 25 April, 2017; originally announced April 2017.

Comments: 31 pages, 2 figures

MSC Class: 94A17

Tracking and Control of Gauss-Markov Processes over Packet-Drop Channels with Acknowledgments

Authors: Anatoly Khina, Victoria Kostina, Ashish Khisti, Babak Hassibi

Abstract: We consider the problem of tracking the state of Gauss-Markov processes over rate-limited erasure-prone links. We concentrate first on the scenario in which several independent processes are seen by a single observer. The observer maps the processes into finite-rate packets that are sent over the erasure-prone links to a state estimator, and are acknowledged upon packet arrivals. The aim of the st… ▽ More We consider the problem of tracking the state of Gauss-Markov processes over rate-limited erasure-prone links. We concentrate first on the scenario in which several independent processes are seen by a single observer. The observer maps the processes into finite-rate packets that are sent over the erasure-prone links to a state estimator, and are acknowledged upon packet arrivals. The aim of the state estimator is to track the processes with zero delay and with minimum mean square error (MMSE). We show that, in the limit of many processes, greedy quantization with respect to the squared error distortion is optimal. That is, there is no tension between optimizing the MMSE of the process in the current time instant and that of future times. For the case of packet erasures with delayed acknowledgments, we connect the problem to that of compression with side information that is known at the observer and may be known at the state estimator - where the most recent packets serve as side information that may have been erased, and demonstrate that the loss due to a delay by one time unit is rather small. For the scenario where only one process is tracked by the observer-state estimator system, we further show that variable-length coding techniques are within a small gap of the many-process outer bound. We demonstrate the usefulness of the proposed approach for the simple setting of discrete-time scalar linear quadratic Gaussian control with a limited data-rate feedback that is susceptible to packet erasures. △ Less

Submitted 23 May, 2018; v1 submitted 6 February, 2017; originally announced February 2017.

Rate-cost tradeoffs in control. Part II: achievable scheme

Authors: Victoria Kostina, Babak Hassibi

Abstract: Consider a distributed control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate r bits/sec and the limsup of the exp… ▽ More Consider a distributed control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate r bits/sec and the limsup of the expected cost b. In the companion paper, which can be read independently of the current one, we show a lower bound on a certain cost function, which quantifies the minimum mutual information between the channel input and output, given the past, that is compatible with a target LQR cost. The bound applies as long as the system noise has a probability density function, and it holds for a general class of codes that can take full advantage of the memory of the data observed so far and that are not constrained to have any particular structure. In this paper, we prove that the bound can be approached by a simple variable-length lattice quantization scheme, as long as the system noise satisfies a smoothness condition. The quantization scheme only quantizes the innovation, that is, the difference between the controller's belief about the current state and the encoder's state estimate. Our proof technique leverages some recent results on nonasymptotic high resolution vector quantization. △ Less

Submitted 18 October, 2017; v1 submitted 7 December, 2016; originally announced December 2016.

Comments: merged with Part I (1612.02126)

Rate-cost tradeoffs in control

Authors: Victoria Kostina, Babak Hassibi

Abstract: Consider a control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate $r$ bits/sec and the expected cost $b$. We obtai… ▽ More Consider a control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate $r$ bits/sec and the expected cost $b$. We obtain a lower bound on a certain rate-cost function, which quantifies the minimum directed mutual information between the channel input and output that is compatible with a target LQR cost. The rate-cost function has operational significance in multiple scenarios of interest: among others, it allows us to lower-bound the minimum communication rate for fixed and variable length quantization, and for control over noisy channels. We derive an explicit lower bound to the rate-cost function, which applies to the vector, non-Gaussian, and partially observed systems, thereby extending and generalizing an earlier explicit expression for the scalar Gaussian system, due to Tatikonda el al. The bound applies as long as the differential entropy of the system noise is not $-\infty$. It can be closely approached by a simple lattice quantization scheme that only quantizes the innovation, that is, the difference between the controller's belief about the current state and the true state. Via a separation principle between control and communication, similar results hold for causal lossy compression of additive noise Markov sources. Apart from standard dynamic programming arguments, our technical approach leverages the Shannon lower bound, develops new estimates for data compression with coding memory, and uses some recent results on high resolution variable-length vector quantization to prove that the new converse bounds are tight. △ Less

Submitted 20 November, 2018; v1 submitted 7 December, 2016; originally announced December 2016.

Comments: arXiv admin note: text overlap with arXiv:1612.02128

Multi-Rate Control over AWGN Channels via Analog Joint Source-Channel Coding

Authors: Anatoly Khina, Gustav M. Pettersson, Victoria Kostina, Babak Hassibi

Abstract: We consider the problem of controlling an unstable plant over an additive white Gaussian noise (AWGN) channel with a transmit power constraint, where the signaling rate of communication is larger than the sampling rate (for generating observations and applying control inputs) of the underlying plant. Such a situation is quite common since sampling is done at a rate that captures the dynamics of th… ▽ More We consider the problem of controlling an unstable plant over an additive white Gaussian noise (AWGN) channel with a transmit power constraint, where the signaling rate of communication is larger than the sampling rate (for generating observations and applying control inputs) of the underlying plant. Such a situation is quite common since sampling is done at a rate that captures the dynamics of the plant and which is often much lower than the rate that can be communicated. This setting offers the opportunity of improving the system performance by employing multiple channel uses to convey a single message (output plant observation or control input). Common ways of doing so are through either repeating the message, or by quantizing it to a number of bits and then transmitting a channel coded version of the bits whose length is commensurate with the number of channel uses per sampled message. We argue that such "separated source and channel coding" can be suboptimal and propose to perform joint source-channel coding. Since the block length is short we obviate the need to go to the digital domain altogether and instead consider analog joint source-channel coding. For the case where the communication signaling rate is twice the sampling rate, we employ the Archimedean bi-spiral-based Shannon-Kotel'nikov analog maps to show significant improvement in stability margins and linear-quadratic Gaussian (LQG) costs over simple schemes that employ repetition. △ Less

Submitted 27 October, 2016; v1 submitted 25 September, 2016; originally announced September 2016.

Comments: An extended version of a paper to be presented in CDC2016

Data compression with low distortion and finite blocklength

Authors: Victoria Kostina

Abstract: This paper considers lossy source coding of $n$-dimensional memoryless sources and shows an explicit approximation to the minimum source coding rate required to sustain the probability of exceeding distortion $d$ no greater than $ε$, which is simpler than known dispersion-based approximations. Our approach takes inspiration in the celebrated classical result stating that the Shannon lower bound… ▽ More This paper considers lossy source coding of $n$-dimensional memoryless sources and shows an explicit approximation to the minimum source coding rate required to sustain the probability of exceeding distortion $d$ no greater than $ε$, which is simpler than known dispersion-based approximations. Our approach takes inspiration in the celebrated classical result stating that the Shannon lower bound to rate-distortion function becomes tight in the limit $d \to 0$. We formulate an abstract version of the Shannon lower bound that recovers both the classical Shannon lower bound and the rate-distortion function itself as special cases. Likewise, we show that a nonasymptotic version of the abstract Shannon lower bound recovers all previously known nonasymptotic converses. A necessary and sufficient condition for the Shannon lower bound to be attained exactly is presented. It is demonstrated that whenever that condition is met, the rate-dispersion function is given simply by the varentropy of the source. Remarkably, all finite alphabet sources with balanced distortion measures satisfy that condition in the range of low distortions. Most continuous sources violate that condition. Still, we show that lattice quantizers closely approach the nonasymptotic Shannon lower bound, provided that the source density is smooth enough and the distortion is low. This implies that fine multidimensional lattice coverings are nearly optimal in the rate-distortion sense even at finite $n$. The achievability proof technique is based on a new bound on the output entropy of lattice quantizers in terms of the differential entropy of the source, the lattice cell size and a smoothness parameter of the source density. The technique avoids both the usual random coding argument and the simplifying assumption of the presence of a dither signal. △ Less

Submitted 25 February, 2017; v1 submitted 7 October, 2015; originally announced October 2015.

Comments: To appear in IEEE Transactions on Information Theory

Joint source-channel coding with feedback

Authors: Victoria Kostina, Yury Polyanskiy, Sergio Verdú

Abstract: This paper quantifies the fundamental limits of variable-length transmission of a general (possibly analog) source over a memoryless channel with noiseless feedback, under a distortion constraint. We consider excess distortion, average distortion and guaranteed distortion ($d$-semifaithful codes). In contrast to the asymptotic fundamental limit, a general conclusion is that allowing variable-lengt… ▽ More This paper quantifies the fundamental limits of variable-length transmission of a general (possibly analog) source over a memoryless channel with noiseless feedback, under a distortion constraint. We consider excess distortion, average distortion and guaranteed distortion ($d$-semifaithful codes). In contrast to the asymptotic fundamental limit, a general conclusion is that allowing variable-length codes and feedback leads to a sizable improvement in the fundamental delay-distortion tradeoff. In addition, we investigate the minimum energy required to reproduce $k$ source samples with a given fidelity after transmission over a memoryless Gaussian channel, and we show that the required minimum energy is reduced with feedback and an average (rather than maximal) power constraint. △ Less

Submitted 25 February, 2017; v1 submitted 29 January, 2015; originally announced January 2015.

Comments: To appear in IEEE Transactions on Information Theory

Variable-length compression allowing errors

Authors: Victoria Kostina, Yury Polyanskiy, Sergio Verdú

Abstract: This paper studies the fundamental limits of the minimum average length of lossless and lossy variable-length compression, allowing a nonzero error probability $ε$, for lossless compression. We give non-asymptotic bounds on the minimum average length in terms of Erokhin's rate-distortion function and we use those bounds to obtain a Gaussian approximation on the speed of approach to the limit which… ▽ More This paper studies the fundamental limits of the minimum average length of lossless and lossy variable-length compression, allowing a nonzero error probability $ε$, for lossless compression. We give non-asymptotic bounds on the minimum average length in terms of Erokhin's rate-distortion function and we use those bounds to obtain a Gaussian approximation on the speed of approach to the limit which is quite accurate for all but small blocklengths: $$(1 - ε) k H(\mathsf S) - \sqrt{\frac{k V(\mathsf S)}{2 π} } e^{- \frac {(Q^{-1}(ε))^2} 2 }$$ where $Q^{-1}(\cdot)$ is the functional inverse of the standard Gaussian complementary cdf, and $V(\mathsf S)$ is the source dispersion. A nonzero error probability thus not only reduces the asymptotically achievable rate by a factor of $1 - ε$, but this asymptotic limit is approached from below, i.e. larger source dispersions and shorter blocklengths are beneficial. Variable-length lossy compression under an excess distortion constraint is shown to exhibit similar properties. △ Less

Submitted 7 October, 2015; v1 submitted 3 February, 2014; originally announced February 2014.

Journal ref: IEEE Transactions on Information Theory, vol. 61, no. 9, pp. 4316-4330, Aug. 2015

Nonasymptotic noisy lossy source coding

Authors: Victoria Kostina, Sergio Verdú

Abstract: This paper shows new general nonasymptotic achievability and converse bounds and performs their dispersion analysis for the lossy compression problem in which the compressor observes the source through a noisy channel. While this problem is asymptotically equivalent to a noiseless lossy source coding problem with a modified distortion function, nonasymptotically there is a noticeable gap in how fa… ▽ More This paper shows new general nonasymptotic achievability and converse bounds and performs their dispersion analysis for the lossy compression problem in which the compressor observes the source through a noisy channel. While this problem is asymptotically equivalent to a noiseless lossy source coding problem with a modified distortion function, nonasymptotically there is a noticeable gap in how fast their minimum achievable coding rates approach the common rate-distortion function, as evidenced both by the refined asymptotic analysis (dispersion) and the numerical results. The size of the gap between the dispersions of the noisy problem and the asymptotically equivalent noiseless problem depends on the stochastic variability of the channel through which the compressor observes the source. △ Less

Submitted 15 September, 2016; v1 submitted 20 January, 2014; originally announced January 2014.

Comments: IEEE Transactions on Information Theory, 2016

Channels with cost constraints: strong converse and dispersion

Authors: Victoria Kostina, Sergio Verdú

Abstract: This paper shows the strong converse and the dispersion of memoryless channels with cost constraints and performs refined analysis of the third order term in the asymptotic expansion of the maximum achievable channel coding rate, showing that it is equal to $\frac 1 2 \frac {\log n}{n}$ in most cases of interest. The analysis is based on a non-asymptotic converse bound expressed in terms of the di… ▽ More This paper shows the strong converse and the dispersion of memoryless channels with cost constraints and performs refined analysis of the third order term in the asymptotic expansion of the maximum achievable channel coding rate, showing that it is equal to $\frac 1 2 \frac {\log n}{n}$ in most cases of interest. The analysis is based on a non-asymptotic converse bound expressed in terms of the distribution of a random variable termed the $\mathsf b$-tilted information density, which plays a role similar to that of the $\mathsf d$-tilted information in lossy source coding. We also analyze the fundamental limits of lossy joint-source-channel coding over channels with cost constraints. △ Less

Submitted 7 October, 2015; v1 submitted 20 January, 2014; originally announced January 2014.

Journal ref: IEEE Transactions on Information Theory, vol. 61, no. 5, pp. 2415-2429, May 2015

On Convexity of Error Rates in Digital Communications

Authors: Sergey Loyka, Victoria Kostina, Francois Gagnon

Abstract: Convexity properties of error rates of a class of decoders, including the ML/min-distance one as a special case, are studied for arbitrary constellations, bit map** and coding. Earlier results obtained for the AWGN channel are extended to a wide class of noise densities, including unimodal and spherically-invariant noise. Under these broad conditions, symbol and bit error rates are shown to be c… ▽ More Convexity properties of error rates of a class of decoders, including the ML/min-distance one as a special case, are studied for arbitrary constellations, bit map** and coding. Earlier results obtained for the AWGN channel are extended to a wide class of noise densities, including unimodal and spherically-invariant noise. Under these broad conditions, symbol and bit error rates are shown to be convex functions of the SNR in the high-SNR regime with an explicitly-determined threshold, which depends only on the constellation dimensionality and minimum distance, thus enabling an application of the powerful tools of convex optimization to such digital communication systems in a rigorous way. It is the decreasing nature of the noise power density around the decision region boundaries that insures the convexity of symbol error rates in the general case. The known high/low SNR bounds of the convexity/concavity regions are tightened and no further improvement is shown to be possible in general. The high SNR bound fits closely into the channel coding theorem: all codes, including capacity-achieving ones, whose decision regions include the hardened noise spheres (from the noise sphere hardening argument in the channel coding theorem) satisfies this high SNR requirement and thus has convex error rates in both SNR and noise power. We conjecture that all capacity-achieving codes have convex error rates. Convexity properties in signal amplitude and noise power are also investigated. Some applications of the results are discussed. In particular, it is shown that fading is convexity-preserving and is never good in low dimensions under spherically-invariant noise, which may also include any linear diversity combining. △ Less

Submitted 30 April, 2013; originally announced April 2013.

Comments: Accepted by IEEE Information Theory Transactions, 2013

Lossy joint source-channel coding in the finite blocklength regime

Authors: Victoria Kostina, Sergio Verdú

Abstract: This paper finds new tight finite-blocklength bounds for the best achievable lossy joint source-channel code rate, and demonstrates that joint source-channel code design brings considerable performance advantage over a separate one in the non-asymptotic regime. A joint source-channel code maps a block of $k$ source symbols onto a length$-n$ channel codeword, and the fidelity of reproduction at the… ▽ More This paper finds new tight finite-blocklength bounds for the best achievable lossy joint source-channel code rate, and demonstrates that joint source-channel code design brings considerable performance advantage over a separate one in the non-asymptotic regime. A joint source-channel code maps a block of $k$ source symbols onto a length$-n$ channel codeword, and the fidelity of reproduction at the receiver end is measured by the probability $ε$ that the distortion exceeds a given threshold $d$. For memoryless sources and channels, it is demonstrated that the parameters of the best joint source-channel code must satisfy $nC - kR(d) \approx \sqrt{nV + k \mathcal V(d)} Q(ε)$, where $C$ and $V$ are the channel capacity and channel dispersion, respectively; $R(d)$ and $\mathcal V(d)$ are the source rate-distortion and rate-dispersion functions; and $Q$ is the standard Gaussian complementary cdf. Symbol-by-symbol (uncoded) transmission is known to achieve the Shannon limit when the source and channel satisfy a certain probabilistic matching condition. In this paper we show that even when this condition is not satisfied, symbol-by-symbol transmission is, in some cases, the best known strategy in the non-asymptotic regime. △ Less

Submitted 3 February, 2014; v1 submitted 6 September, 2012; originally announced September 2012.

Journal ref: IEEE Transactions on Information Theory, vol. 59, no. 5, pp. 2545-2575, May 2013

Fixed-length lossy compression in the finite blocklength regime

Authors: Victoria Kostina, Sergio Verdú

Abstract: This paper studies the minimum achievable source coding rate as a function of blocklength $n$ and probability $ε$ that the distortion exceeds a given level $d$. Tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength. For stationary memoryless sources with separable distortion, the minimum rate achievable is shown to be closely approximated by… ▽ More This paper studies the minimum achievable source coding rate as a function of blocklength $n$ and probability $ε$ that the distortion exceeds a given level $d$. Tight general achievability and converse bounds are derived that hold at arbitrary fixed blocklength. For stationary memoryless sources with separable distortion, the minimum rate achievable is shown to be closely approximated by $R(d) + \sqrt{\frac{V(d)}{n}} Q^{-1}(ε)$, where $R(d)$ is the rate-distortion function, $V(d)$ is the rate dispersion, a characteristic of the source which measures its stochastic variability, and $Q^{-1}(ε)$ is the inverse of the standard Gaussian complementary cdf. △ Less

Submitted 3 February, 2014; v1 submitted 18 February, 2011; originally announced February 2011.

Journal ref: IEEE Transactions on Information Theory, vol. 58, no. 6, pp. 3309-3338, June 2012

Optimum Power and Rate Allocation for Coded V-BLAST: Average Optimization

Authors: Victoria Kostina, Sergey Loyka

Abstract: An analytical framework for performance analysis and optimization of coded V-BLAST is developed. Average power and/or rate allocations to minimize the outage probability as well as their robustness and dual problems are investigated. Compact, closed-form expressions for the optimum allocations and corresponding system performance are given. The uniform power allocation is shown to be near optimum… ▽ More An analytical framework for performance analysis and optimization of coded V-BLAST is developed. Average power and/or rate allocations to minimize the outage probability as well as their robustness and dual problems are investigated. Compact, closed-form expressions for the optimum allocations and corresponding system performance are given. The uniform power allocation is shown to be near optimum in the low outage regime in combination with the optimum rate allocation. The average rate allocation provides the largest performance improvement (extra diversity gain), and the average power allocation offers a modest SNR gain limited by the number of transmit antennas but does not increase the diversity gain. The dual problems are shown to have the same solutions as the primal ones. All these allocation strategies are shown to be robust. The reported results also apply to coded multiuser detection and channel equalization systems relying on successive interference cancelation. △ Less

Submitted 13 October, 2010; originally announced October 2010.

Comments: accepted by IEEE Transactions on Communications

Error Rates of Capacity-Achieving Codes Are Convex

Authors: Sergey Loyka, Francois Gagnon, Victoria Kostina

Abstract: Motivated by a wide-spread use of convex optimization techniques, convexity properties of bit error rate of the maximum likelihood detector operating in the AWGN channel are studied for arbitrary constellations and bit map**s, which also includes coding under maximum-likelihood decoding. Under this generic setting, the pairwise probability of error and bit error rate are shown to be convex funct… ▽ More Motivated by a wide-spread use of convex optimization techniques, convexity properties of bit error rate of the maximum likelihood detector operating in the AWGN channel are studied for arbitrary constellations and bit map**s, which also includes coding under maximum-likelihood decoding. Under this generic setting, the pairwise probability of error and bit error rate are shown to be convex functions of the SNR and noise power in the high SNR/low noise regime with explicitly-determined boundary. Any code, including capacity-achieving ones, whose decision regions include the hardened noise spheres (from the noise sphere hardening argument in the channel coding theorem) satisfies this high SNR requirement and thus has convex error rates in both SNR and noise power. We conjecture that all capacity-achieving codes have convex error rates. △ Less

Submitted 15 April, 2010; originally announced April 2010.

Comments: accepted by ISIT-10

Bit Error Rate is Convex at High SNR

Authors: Sergey Loyka, Victoria Kostina, Francois Gagnon

Abstract: Motivated by a wide-spread use of convex optimization techniques, convexity properties of bit error rate of the maximum likelihood detector operating in the AWGN channel are studied for arbitrary constellations and bit map**s, which may also include coding under maximum-likelihood decoding. Under this generic setting, the pairwise probability of error and bit error rate are shown to be convex… ▽ More Motivated by a wide-spread use of convex optimization techniques, convexity properties of bit error rate of the maximum likelihood detector operating in the AWGN channel are studied for arbitrary constellations and bit map**s, which may also include coding under maximum-likelihood decoding. Under this generic setting, the pairwise probability of error and bit error rate are shown to be convex functions of the SNR in the high SNR regime with explicitly-determined boundary. The bit error rate is also shown to be a convex function of the noise power in the low noise/high SNR regime. △ Less

Submitted 26 December, 2009; originally announced December 2009.

Comments: to be presented at IZS-10, March 2010

Error Rates of the Maximum-Likelihood Detector for Arbitrary Constellations: Convex/Concave Behavior and Applications

Authors: Sergey Loyka, Victoria Kostina, Francois Gagnon

Abstract: Motivated by a recent surge of interest in convex optimization techniques, convexity/concavity properties of error rates of the maximum likelihood detector operating in the AWGN channel are studied and extended to frequency-flat slow-fading channels. Generic conditions are identified under which the symbol error rate (SER) is convex/concave for arbitrary multi-dimensional constellations. In part… ▽ More Motivated by a recent surge of interest in convex optimization techniques, convexity/concavity properties of error rates of the maximum likelihood detector operating in the AWGN channel are studied and extended to frequency-flat slow-fading channels. Generic conditions are identified under which the symbol error rate (SER) is convex/concave for arbitrary multi-dimensional constellations. In particular, the SER is convex in SNR for any one- and two-dimensional constellation, and also in higher dimensions at high SNR. Pairwise error probability and bit error rate are shown to be convex at high SNR, for arbitrary constellations and bit map**. Universal bounds for the SER 1st and 2nd derivatives are obtained, which hold for arbitrary constellations and are tight for some of them. Applications of the results are discussed, which include optimum power allocation in spatial multiplexing systems, optimum power/time sharing to decrease or increase (jamming problem) error rate, an implication for fading channels ("fading is never good in low dimensions") and optimization of a unitary-precoded OFDM system. For example, the error rate bounds of a unitary-precoded OFDM system with QPSK modulation, which reveal the best and worst precoding, are extended to arbitrary constellations, which may also include coding. The reported results also apply to the interference channel under Gaussian approximation, to the bit error rate when it can be expressed or approximated as a non-negative linear combination of individual symbol error rates, and to coded systems. △ Less

Submitted 19 November, 2009; originally announced November 2009.

Comments: accepted by IEEE IT Transactions

Journal ref: IEEE Transactions on Information Theory, v. 56, N. 4, pp. 1948-1960, Apr. 2010

Optimum Power and Rate Allocation for Coded V-BLAST

Authors: Victoria Kostina, Sergey Loyka

Abstract: An analytical framework for minimizing the outage probability of a coded spatial multiplexing system while kee** the rate close to the capacity is developed. Based on this framework, specific strategies of optimum power and rate allocation for the coded V-BLAST architecture are obtained and its performance is analyzed. A fractional waterfilling algorithm, which is shown to optimize both the ca… ▽ More An analytical framework for minimizing the outage probability of a coded spatial multiplexing system while kee** the rate close to the capacity is developed. Based on this framework, specific strategies of optimum power and rate allocation for the coded V-BLAST architecture are obtained and its performance is analyzed. A fractional waterfilling algorithm, which is shown to optimize both the capacity and the outage probability of the coded V-BLAST, is proposed. Compact, closed-form expressions for the optimum allocation of the average power are given. The uniform allocation of average power is shown to be near optimum at moderate to high SNR for the coded V-BLAST with the average rate allocation (when per-stream rates are set to match the per-stream capacity). The results reported also apply to multiuser detection and channel equalization relying on successive interference cancelation. △ Less

Submitted 2 February, 2009; originally announced February 2009.

Comments: accepted by ICC-09

On Optimum Power Allocation for the V-BLAST

Authors: Victoria Kostina, Sergey Loyka

Abstract: A unified analytical framework for optimum power allocation in the unordered V-BLAST algorithm and its comparative performance analysis are presented. Compact closed-form approximations for the optimum power allocation are derived, based on average total and block error rates. The choice of the criterion has little impact on the power allocation and, overall, the optimum strategy is to allocate… ▽ More A unified analytical framework for optimum power allocation in the unordered V-BLAST algorithm and its comparative performance analysis are presented. Compact closed-form approximations for the optimum power allocation are derived, based on average total and block error rates. The choice of the criterion has little impact on the power allocation and, overall, the optimum strategy is to allocate more power to lower step transmitters and less to higher ones. High-SNR approximations for optimized average block and total error rates are given. The SNR gain of optimization is rigorously defined and studied using analytical tools, including lower and upper bounds, high and low SNR approximations. The gain is upper bounded by the number of transmitters, for any modulation format and type of fading channel. While the average optimization is less complex than the instantaneous one, its performance is almost as good at high SNR. A measure of robustness of the optimized algorithm is introduced and evaluated. The optimized algorithm is shown to be robust to perturbations in individual and total transmit powers. Based on the algorithm robustness, a pre-set power allocation is suggested as a low-complexity alternative to the other optimization strategies, which exhibits only a minor loss in performance over the practical SNR range. △ Less

Submitted 9 May, 2007; originally announced May 2007.

Comments: Accepted by IEEE Transactions on Communications, Apr. 2007

Symbol Error Rates of Maximum-Likelihood Detector: Convex/Concave Behavior and Applications

Authors: Sergey Loyka, Victoria Kostina, Francois Gagnon

Abstract: Convexity/concavity properties of symbol error rates (SER) of the maximum likelihood detector operating in the AWGN channel (non-fading and fading) are studied. Generic conditions are identified under which the SER is a convex/concave function of the SNR. Universal bounds for the SER 1st and 2nd derivatives are obtained, which hold for arbitrary constellations and are tight for some of them. App… ▽ More Convexity/concavity properties of symbol error rates (SER) of the maximum likelihood detector operating in the AWGN channel (non-fading and fading) are studied. Generic conditions are identified under which the SER is a convex/concave function of the SNR. Universal bounds for the SER 1st and 2nd derivatives are obtained, which hold for arbitrary constellations and are tight for some of them. Applications of the results are discussed, which include optimum power allocation in spatial multiplexing systems, optimum power/time sharing to decrease or increase (jamming problem) error rate, and implication for fading channels. △ Less

Submitted 8 May, 2007; originally announced May 2007.

Comments: To appear in 2007 IEEE International Symposium on Information Theory (ISIT 2007), Nice, June 2007