Showing 1–6 of 6 results for author: Haikin, M

Asymptotic Frame Theory for Analog Coding

Authors: Marina Haikin, Matan Gavish, Dustin G. Mixon, Ram Zamir

Abstract: Over-complete systems of vectors, or in short, frames, play the role of analog codes in many areas of communication and signal processing. To name a few, spreading sequences for code-division multiple access (CDMA), over-complete representations for multiple-description (MD) source coding, space-time codes, sensing matrices for compressed sensing (CS), and more recently, codes for unreliable distr… ▽ More Over-complete systems of vectors, or in short, frames, play the role of analog codes in many areas of communication and signal processing. To name a few, spreading sequences for code-division multiple access (CDMA), over-complete representations for multiple-description (MD) source coding, space-time codes, sensing matrices for compressed sensing (CS), and more recently, codes for unreliable distributed computation. In this survey paper we observe an information-theoretic random-like behavior of frame subsets. Such sub-frames arise in setups involving erasures (communication), random user activity (multiple access), or sparsity (signal processing), in addition to channel or quantization noise. The goodness of a frame as an analog code is a function of the eigenvalues of a sub-frame, averaged over all sub-frames. Within the highly symmetric class of Equiangular Tight Frames (ETF), as well as other "near ETF" families, we show a universal behavior of the empirical eigenvalue distribution (ESD) of a randomly-selected sub-frame: (i) the ESD is asymptotically indistinguishable from Wachter's MANOVA distribution; and (ii) it exhibits a convergence rate to this limit that is indistinguishable from that of a matrix sequence drawn from MANOVA (Jacobi) ensembles of corresponding dimensions. Some of these results follow from careful statistical analysis of empirical evidence, and some are proved analytically using random matrix theory arguments of independent interest. The goodness measures of the MANOVA limit distribution are better, in a concrete formal sense, than those of the Marchenko-Pastur distribution at the same aspect ratio, implying that deterministic analog codes are better than random (i.i.d.) analog codes. We further give evidence that the ETF (and near ETF) family is in fact superior to any other frame family in terms of its typical sub-frame goodness. △ Less

Submitted 14 November, 2021; v1 submitted 28 August, 2021; originally announced August 2021.

Moments of Subsets of General Equiangular Tight Frames

Authors: Marina Haikin, Ram Zamir

Abstract: This note outlines the steps for proving that the moments of a randomly-selected subset of a general ETF (complex, with aspect ratio $0<γ<1$) converge to the corresponding MANOVA moments. We bring here an extension for the proof of the 'Kesten-Mckay' moments (real ETF, $γ=1/2$) \cite{magsino2020kesten}. In particular, we establish a recursive computation of the $r$th moment, for $r = 1,2,\ldots$,… ▽ More This note outlines the steps for proving that the moments of a randomly-selected subset of a general ETF (complex, with aspect ratio $0<γ<1$) converge to the corresponding MANOVA moments. We bring here an extension for the proof of the 'Kesten-Mckay' moments (real ETF, $γ=1/2$) \cite{magsino2020kesten}. In particular, we establish a recursive computation of the $r$th moment, for $r = 1,2,\ldots$, and verify, using a symbolic program, that the recursion output coincides with the MANOVA moments. △ Less

Submitted 2 July, 2021; originally announced July 2021.

Analog Coding Frame-work

Authors: Marina Haikin

Abstract: Analog coding is a low-complexity method to combat erasures, based on linear redundancy in the signal space domain. Previous work examined "band-limited discrete Fourier transform (DFT)" codes for Gaussian channels with erasures or impulses. We extend this concept to source coding with "erasure side-information" at the encoder and show that the performance of band-limited DFT can be significantly… ▽ More Analog coding is a low-complexity method to combat erasures, based on linear redundancy in the signal space domain. Previous work examined "band-limited discrete Fourier transform (DFT)" codes for Gaussian channels with erasures or impulses. We extend this concept to source coding with "erasure side-information" at the encoder and show that the performance of band-limited DFT can be significantly improved using irregular spectrum, and more generally, using equiangular tight frames (ETF). Frames are overcomplete bases and are widely used in mathematics, computer science, engineering, and statistics since they provide a stable and robust decomposition. Design of frames with favorable properties of random subframes is motivated in variety of applications, including code-devision multiple access (CDMA), compressed sensing and analog coding. We present a novel relation between deterministic frames and random matrix theory. We show empirically that the MANOVA ensemble offers a universal description of the spectra of randomly selected subframes with constant aspect ratios, taken from deterministic near-ETFs. Moreover, we derive an analytic framework and bring a formal validation for some of the empirical results, specifically that the asymptotic form for the moments of high orders of subsets of ETF agree with that of MANOVA. Finally, when exploring over-complete bases, the Welch bound is a lower bound on the root mean square cross correlation between vectors. We extend the Welch bound to an erasure setting, in which a reduced frame, composed of a random subset of Bernoulli selected vectors, is of interest. The lower bound involves moment of the reduced frame, and it is tight for ETFs and asymptotically coincides with the MANOVA moments. This result offers a novel perspective on the superiority of ETFs over other frames. △ Less

Submitted 19 September, 2018; originally announced September 2018.

Frame Moments and Welch Bound with Erasures

Authors: Marina Haikin, Ram Zamir, Matan Gavish

Abstract: The Welch Bound is a lower bound on the root mean square cross correlation between $n$ unit-norm vectors $f_1,...,f_n$ in the $m$ dimensional space ($\mathbb{R} ^m$ or $\mathbb{C} ^m$), for $n\geq m$. Letting $F = [f_1|...|f_n]$ denote the $m$-by-$n$ frame matrix, the Welch bound can be viewed as a lower bound on the second moment of $F$, namely on the trace of the squared Gram matrix $(F'F)^2$. W… ▽ More The Welch Bound is a lower bound on the root mean square cross correlation between $n$ unit-norm vectors $f_1,...,f_n$ in the $m$ dimensional space ($\mathbb{R} ^m$ or $\mathbb{C} ^m$), for $n\geq m$. Letting $F = [f_1|...|f_n]$ denote the $m$-by-$n$ frame matrix, the Welch bound can be viewed as a lower bound on the second moment of $F$, namely on the trace of the squared Gram matrix $(F'F)^2$. We consider an erasure setting, in which a reduced frame, composed of a random subset of Bernoulli selected vectors, is of interest. We extend the Welch bound to this setting and present the {\em erasure Welch bound} on the expected value of the Gram matrix of the reduced frame. Interestingly, this bound generalizes to the $d$-th order moment of $F$. We provide simple, explicit formulae for the generalized bound for $d=2,3,4$, which is the sum of the $d$-th moment of Wachter's classical MANOVA distribution and a vanishing term (as $n$ goes to infinity with $\frac{m}{n}$ held constant). The bound holds with equality if (and for $d = 4$ only if) $F$ is an Equiangular Tight Frame (ETF). Our results offer a novel perspective on the superiority of ETFs over other frames in a variety of applications, including spread spectrum communications, compressed sensing and analog coding. △ Less

Submitted 14 January, 2018; originally announced January 2018.

Random Subsets of Structured Deterministic Frames have MANOVA Spectra

Authors: Marina Haikin, Ram Zamir, Matan Gavish

Abstract: We draw a random subset of $k$ rows from a frame with $n$ rows (vectors) and $m$ columns (dimensions), where $k$ and $m$ are proportional to $n$. For a variety of important deterministic equiangular tight frames (ETFs) and tight non-ETF frames, we consider the distribution of singular values of the $k$-subset matrix. We observe that for large $n$ they can be precisely described by a known probabil… ▽ More We draw a random subset of $k$ rows from a frame with $n$ rows (vectors) and $m$ columns (dimensions), where $k$ and $m$ are proportional to $n$. For a variety of important deterministic equiangular tight frames (ETFs) and tight non-ETF frames, we consider the distribution of singular values of the $k$-subset matrix. We observe that for large $n$ they can be precisely described by a known probability distribution -- Wachter's MANOVA spectral distribution, a phenomenon that was previously known only for two types of random frames. In terms of convergence to this limit, the $k$-subset matrix from all these frames is shown to be empirically indistinguishable from the classical MANOVA (Jacobi) random matrix ensemble. Thus empirically the MANOVA ensemble offers a universal description of the spectra of randomly selected $k$-subframes, even those taken from deterministic frames. The same universality phenomena is shown to hold for notable random frames as well. This description enables exact calculations of properties of solutions for systems of linear equations based on a random choice of $k$ frame vectors out of $n$ possible vectors, and has a variety of implications for erasure coding, compressed sensing, and sparse recovery. When the aspect ratio $m/n$ is small, the MANOVA spectrum tends to the well known Marcenko-Pastur distribution of the singular values of a Gaussian matrix, in agreement with previous work on highly redundant frames. Our results are empirical, but they are exhaustive, precise and fully reproducible. △ Less

Submitted 12 April, 2017; v1 submitted 5 January, 2017; originally announced January 2017.

Analog Coding of a Source with Erasures

Authors: Marina Haikin, Ram Zamir

Abstract: Analog coding decouples the tasks of protecting against erasures and noise. For erasure correction, it creates an "analog redundancy" by means of band-limited discrete Fourier transform (DFT) interpolation, or more generally, by an over-complete expansion based on a frame. We examine the analog coding paradigm for the dual setup of a source with "erasure" side-information (SI) at the encoder. The… ▽ More Analog coding decouples the tasks of protecting against erasures and noise. For erasure correction, it creates an "analog redundancy" by means of band-limited discrete Fourier transform (DFT) interpolation, or more generally, by an over-complete expansion based on a frame. We examine the analog coding paradigm for the dual setup of a source with "erasure" side-information (SI) at the encoder. The excess rate of analog coding above the rate-distortion function (RDF) is associated with the energy of the inverse of submatrices of the frame, where each submatrix corresponds to a possible erasure pattern. We give a partial theoretical as well as numerical evidence that a variety of structured frames, in particular DFT frames with difference-set spectrum and more general equiangular tight frames (ETFs), with a common MANOVA limiting spectrum, minimize the excess rate over all possible frames. However, they do not achieve the RDF even in the limit as the dimension goes to infinity. △ Less

Submitted 24 May, 2016; v1 submitted 10 February, 2016; originally announced February 2016.