Showing 1–50 of 85 results for author: Goldsmith, J

Privacy Preserving Semi-Decentralized Mean Estimation over Intermittently-Connected Networks

Authors: Rajarshi Saha, Mohamed Seif, Michal Yemini, Andrea J. Goldsmith, H. Vincent Poor

Abstract: We consider the problem of privately estimating the mean of vectors distributed across different nodes of an unreliable wireless network, where communications between nodes can fail intermittently. We adopt a semi-decentralized setup, wherein to mitigate the impact of intermittently connected links, nodes can collaborate with their neighbors to compute a local consensus, which they relay to a cent… ▽ More We consider the problem of privately estimating the mean of vectors distributed across different nodes of an unreliable wireless network, where communications between nodes can fail intermittently. We adopt a semi-decentralized setup, wherein to mitigate the impact of intermittently connected links, nodes can collaborate with their neighbors to compute a local consensus, which they relay to a central server. In such a setting, the communications between any pair of nodes must ensure that the privacy of the nodes is rigorously maintained to prevent unauthorized information leakage. We study the tradeoff between collaborative relaying and privacy leakage due to the data sharing among nodes and, subsequently, propose PriCER: Private Collaborative Estimation via Relaying -- a differentially private collaborative algorithm for mean estimation to optimize this tradeoff. The privacy guarantees of PriCER arise (i) implicitly, by exploiting the inherent stochasticity of the flaky network connections, and (ii) explicitly, by adding Gaussian perturbations to the estimates exchanged by the nodes. Local and central privacy guarantees are provided against eavesdroppers who can observe different signals, such as the communications amongst nodes during local consensus and (possibly multiple) transmissions from the relays to the central server. We substantiate our theoretical findings with numerical simulations. Our implementation is available at https://github.com/rajarshisaha95/private-collaborative-relaying. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: 14 pages, 6 figures. arXiv admin note: text overlap with arXiv:2303.00035

Compressing Large Language Models using Low Rank and Low Precision Decomposition

Authors: Rajarshi Saha, Naomi Sagan, Varun Srivastava, Andrea J. Goldsmith, Mert Pilanci

Abstract: The prohibitive sizes of Large Language Models (LLMs) today make it difficult to deploy them on memory-constrained edge devices. This work introduces $\rm CALDERA$ -- a new post-training LLM compression algorithm that harnesses the inherent low-rank structure of a weight matrix $\mathbf{W}$ by approximating it via a low-rank, low-precision decomposition as… ▽ More The prohibitive sizes of Large Language Models (LLMs) today make it difficult to deploy them on memory-constrained edge devices. This work introduces $\rm CALDERA$ -- a new post-training LLM compression algorithm that harnesses the inherent low-rank structure of a weight matrix $\mathbf{W}$ by approximating it via a low-rank, low-precision decomposition as $\mathbf{W} \approx \mathbf{Q} + \mathbf{L}\mathbf{R}$. Here, $\mathbf{L}$ and $\mathbf{R}$ are low rank factors, and the entries of $\mathbf{Q}$, $\mathbf{L}$ and $\mathbf{R}$ are quantized. The model is compressed by substituting each layer with its $\mathbf{Q} + \mathbf{L}\mathbf{R}$ decomposition, and the zero-shot performance of the compressed model is evaluated. Additionally, $\mathbf{L}$ and $\mathbf{R}$ are readily amenable to low-rank adaptation, consequently enhancing the zero-shot performance. $\rm CALDERA$ obtains this decomposition by formulating it as an optimization problem $\min_{\mathbf{Q},\mathbf{L},\mathbf{R}}\lVert(\mathbf{Q} + \mathbf{L}\mathbf{R} - \mathbf{W})\mathbf{X}^\top\rVert_{\rm F}^2$, where $\mathbf{X}$ is the calibration data, and $\mathbf{Q}, \mathbf{L}, \mathbf{R}$ are constrained to be representable using low-precision formats. Theoretical upper bounds on the approximation error of $\rm CALDERA$ are established using a rank-constrained regression framework, and the tradeoff between compression ratio and model performance is studied by analyzing the impact of target rank and quantization bit budget. Results illustrate that compressing LlaMa-$2$ $7$B/$70$B and LlaMa-$3$ $8$B models obtained using $\rm CALDERA$ outperforms existing post-training LLM compression techniques in the regime of less than $2.5$ bits per parameter. The implementation is available at: \href{https://github.com/pilancilab/caldera}{https://github.com/pilancilab/caldera}. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: 30 pages, 9 figures, 7 tables

Private Online Community Detection for Censored Block Models

Authors: Mohamed Seif, Liyan Xie, Andrea J. Goldsmith, H. Vincent Poor

Abstract: We study the private online change detection problem for dynamic communities, using a censored block model (CBM). Focusing on the notion of edge differential privacy (DP), we seek to understand the fundamental tradeoffs between the privacy budget, detection delay, and exact community recovery of community labels. We establish the theoretical lower bound on the delay in detecting changes privately… ▽ More We study the private online change detection problem for dynamic communities, using a censored block model (CBM). Focusing on the notion of edge differential privacy (DP), we seek to understand the fundamental tradeoffs between the privacy budget, detection delay, and exact community recovery of community labels. We establish the theoretical lower bound on the delay in detecting changes privately and propose an algorithm capable of identifying changes in the community structure, while maintaining user privacy. Further, we provide theoretical guarantees for the effectiveness of our proposed method by showing necessary and sufficient conditions on change detection and exact recovery under edge DP. Simulation and real data examples are provided to validate the proposed method. △ Less

Submitted 9 May, 2024; originally announced May 2024.

Expert-Level Annotation Quality Achieved by Gamified Crowdsourcing for B-line Segmentation in Lung Ultrasound

Authors: Mike **, Nicole M Duggan, Varoon Bashyakarla, Maria Alejandra Duran Mendicuti, Stephen Hallisey, Denie Bernier, Joseph Stegeman, Erik Duhaime, Tina Kapur, Andrew J Goldsmith

Abstract: Accurate and scalable annotation of medical data is critical for the development of medical AI, but obtaining time for annotation from medical experts is challenging. Gamified crowdsourcing has demonstrated potential for obtaining highly accurate annotations for medical data at scale, and we demonstrate the same in this study for the segmentation of B-lines, an indicator of pulmonary congestion, o… ▽ More Accurate and scalable annotation of medical data is critical for the development of medical AI, but obtaining time for annotation from medical experts is challenging. Gamified crowdsourcing has demonstrated potential for obtaining highly accurate annotations for medical data at scale, and we demonstrate the same in this study for the segmentation of B-lines, an indicator of pulmonary congestion, on still frames within point-of-care lung ultrasound clips. We collected 21,154 annotations from 214 annotators over 2.5 days, and we demonstrated that the concordance of crowd consensus segmentations with reference standards exceeds that of individual experts with the same reference standards, both in terms of B-line count (mean squared error 0.239 vs. 0.308, p<0.05) as well as the spatial precision of B-line annotations (mean Dice-H score 0.755 vs. 0.643, p<0.05). These results suggest that expert-quality segmentations can be achieved using gamified crowdsourcing. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Syn3DWound: A Synthetic Dataset for 3D Wound Bed Analysis

Authors: Léo Lebrat, Rodrigo Santa Cruz, Remi Chierchia, Yulia Arzhaeva, Mohammad Ali Armin, Joshua Goldsmith, Jeremy Oorloff, Prithvi Reddy, Chuong Nguyen, Lars Petersson, Michelle Barakat-Johnson, Georgina Luscombe, Clinton Fookes, Olivier Salvado, David Ahmedt-Aristizabal

Abstract: Wound management poses a significant challenge, particularly for bedridden patients and the elderly. Accurate diagnostic and healing monitoring can significantly benefit from modern image analysis, providing accurate and precise measurements of wounds. Despite several existing techniques, the shortage of expansive and diverse training datasets remains a significant obstacle to constructing machine… ▽ More Wound management poses a significant challenge, particularly for bedridden patients and the elderly. Accurate diagnostic and healing monitoring can significantly benefit from modern image analysis, providing accurate and precise measurements of wounds. Despite several existing techniques, the shortage of expansive and diverse training datasets remains a significant obstacle to constructing machine learning-based frameworks. This paper introduces Syn3DWound, an open-source dataset of high-fidelity simulated wounds with 2D and 3D annotations. We propose baseline methods and a benchmarking framework for automated 3D morphometry analysis and 2D/3D wound segmentation. △ Less

Submitted 3 March, 2024; v1 submitted 27 November, 2023; originally announced November 2023.

Comments: In the IEEE International Symposium on Biomedical Imaging (ISBI) 2024

How Physicality Enables Trust: A New Era of Trust-Centered Cyberphysical Systems

Authors: Stephanie Gil, Michal Yemini, Arsenia Chorti, Angelia Nedić, H. Vincent Poor, Andrea J. Goldsmith

Abstract: Multi-agent cyberphysical systems enable new capabilities in efficiency, resilience, and security. The unique characteristics of these systems prompt a reevaluation of their security concepts, including their vulnerabilities, and mechanisms to mitigate these vulnerabilities. This survey paper examines how advancement in wireless networking, coupled with the sensing and computing in cyberphysical s… ▽ More Multi-agent cyberphysical systems enable new capabilities in efficiency, resilience, and security. The unique characteristics of these systems prompt a reevaluation of their security concepts, including their vulnerabilities, and mechanisms to mitigate these vulnerabilities. This survey paper examines how advancement in wireless networking, coupled with the sensing and computing in cyberphysical systems, can foster novel security capabilities. This study delves into three main themes related to securing multi-agent cyberphysical systems. First, we discuss the threats that are particularly relevant to multi-agent cyberphysical systems given the potential lack of trust between agents. Second, we present prospects for sensing, contextual awareness, and authentication, enabling the inference and measurement of ``inter-agent trust" for these systems. Third, we elaborate on the application of quantifiable trust notions to enable ``resilient coordination," where ``resilient" signifies sustained functionality amid attacks on multiagent cyberphysical systems. We refer to the capability of cyberphysical systems to self-organize, and coordinate to achieve a task as autonomy. This survey unveils the cyberphysical character of future interconnected systems as a pivotal catalyst for realizing robust, trust-centered autonomy in tomorrow's world. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Gamified Crowdsourcing as a Novel Approach to Lung Ultrasound Dataset Labeling

Authors: Nicole M Duggan, Mike **, Maria Alejandra Duran Mendicuti, Stephen Hallisey, Denie Bernier, Lauren A Selame, Ameneh Asgari-Targhi, Chanel E Fischetti, Ruben Lucassen, Anthony E Samir, Erik Duhaime+, Tina Kapur, Andrew J Goldsmith

Abstract: Study Objective: Machine learning models have advanced medical image processing and can yield faster, more accurate diagnoses. Despite a wealth of available medical imaging data, high-quality labeled data for model training is lacking. We investigated whether a gamified crowdsourcing platform enhanced with inbuilt quality control metrics can produce lung ultrasound clip labels comparable to those… ▽ More Study Objective: Machine learning models have advanced medical image processing and can yield faster, more accurate diagnoses. Despite a wealth of available medical imaging data, high-quality labeled data for model training is lacking. We investigated whether a gamified crowdsourcing platform enhanced with inbuilt quality control metrics can produce lung ultrasound clip labels comparable to those from clinical experts. Methods: 2,384 lung ultrasound clips were retrospectively collected from 203 patients. Six lung ultrasound experts classified 393 of these clips as having no B-lines, one or more discrete B-lines, or confluent B-lines to create two sets of reference standard labels (195 training set clips and 198 test set clips). Sets were respectively used to A) train users on a gamified crowdsourcing platform, and B) compare concordance of the resulting crowd labels to the concordance of individual experts to reference standards. Results: 99,238 crowdsourced opinions on 2,384 lung ultrasound clips were collected from 426 unique users over 8 days. On the 198 test set clips, mean labeling concordance of individual experts relative to the reference standard was 85.0% +/- 2.0 (SEM), compared to 87.9% crowdsourced label concordance (p=0.15). When individual experts' opinions were compared to reference standard labels created by majority vote excluding their own opinion, crowd concordance was higher than the mean concordance of individual experts to reference standards (87.4% vs. 80.8% +/- 1.6; p<0.001). Conclusion: Crowdsourced labels for B-line classification via a gamified approach achieved expert-level quality. Scalable, high-quality labeling approaches may facilitate training dataset creation for machine learning model development. △ Less

Submitted 11 June, 2023; originally announced June 2023.

Comments: 27 pages total

Collaborative Mean Estimation over Intermittently Connected Networks with Peer-To-Peer Privacy

Authors: Rajarshi Saha, Mohamed Seif, Michal Yemini, Andrea J. Goldsmith, H. Vincent Poor

Abstract: This work considers the problem of Distributed Mean Estimation (DME) over networks with intermittent connectivity, where the goal is to learn a global statistic over the data samples localized across distributed nodes with the help of a central server. To mitigate the impact of intermittent links, nodes can collaborate with their neighbors to compute local consensus which they forward to the centr… ▽ More This work considers the problem of Distributed Mean Estimation (DME) over networks with intermittent connectivity, where the goal is to learn a global statistic over the data samples localized across distributed nodes with the help of a central server. To mitigate the impact of intermittent links, nodes can collaborate with their neighbors to compute local consensus which they forward to the central server. In such a setup, the communications between any pair of nodes must satisfy local differential privacy constraints. We study the tradeoff between collaborative relaying and privacy leakage due to the additional data sharing among nodes and, subsequently, propose a novel differentially private collaborative algorithm for DME to achieve the optimal tradeoff. Finally, we present numerical simulations to substantiate our theoretical findings. △ Less

Submitted 28 February, 2023; originally announced March 2023.

Comments: 10 pages, 4 figures

Resilient Distributed Optimization for Multi-Agent Cyberphysical Systems

Authors: Michal Yemini, Angelia Nedić, Andrea J. Goldsmith, Stephanie Gil

Abstract: This work focuses on the problem of distributed optimization in multi-agent cyberphysical systems, where a legitimate agents' iterates are influenced both by the values it receives from potentially malicious neighboring agents, and by its own self-serving target function. We develop a new algorithmic and analytical framework to achieve resilience for the class of problems where stochastic values o… ▽ More This work focuses on the problem of distributed optimization in multi-agent cyberphysical systems, where a legitimate agents' iterates are influenced both by the values it receives from potentially malicious neighboring agents, and by its own self-serving target function. We develop a new algorithmic and analytical framework to achieve resilience for the class of problems where stochastic values of trust between agents exist and can be exploited. In this case we show that convergence to the true global optimal point can be recovered, both in mean and almost surely, even in the presence of malicious agents. Furthermore, we provide expected convergence rate guarantees in the form of upper bounds on the expected squared distance to the optimal value. Finally, numerical results are presented that validate our analytical convergence guarantees even when the malicious agents compose the majority of agents in the network and where existing methods fail to converge to the optimal nominal points. △ Less

Submitted 6 June, 2024; v1 submitted 5 December, 2022; originally announced December 2022.

Visual Perception of Building and Household Vulnerability from Streets

Authors: Chaofeng Wang, Sarah Elizabeth Antos, Jessica Grayson Gosling Goldsmith, Luis Miguel Triveno

Abstract: In develo** countries, building codes often are outdated or not enforced. As a result, a large portion of the housing stock is substandard and vulnerable to natural hazards and climate related events. Assessing housing quality is key to inform public policies and private investments. Standard assessment methods are typically carried out only on a sample / pilot basis due to its high costs or, wh… ▽ More In develo** countries, building codes often are outdated or not enforced. As a result, a large portion of the housing stock is substandard and vulnerable to natural hazards and climate related events. Assessing housing quality is key to inform public policies and private investments. Standard assessment methods are typically carried out only on a sample / pilot basis due to its high costs or, when complete, tend to be obsolete due to the lack of compliance with recommended updating standards or not accessible to most users with the level of detail needed to take key policy or business decisions. Thus, we propose an evaluation framework that is cost-efficient for first capture and future updates, and is reliable at the block level. The framework complements existing work of using street view imagery combined with deep learning to automatically extract building information to assist the identification of housing characteristics. We then check its potential for scalability and higher level reliability. For that purpose, we create an index, which synthesises the highest possible level of granularity of data at the housing unit and at the household level at the block level, and assess whether the predictions made by our model could be used to approximate vulnerability conditions with a lower budget and in selected areas. Our results indicated that the predictions from the images are clearly correlated with the index. △ Less

Submitted 28 May, 2022; originally announced May 2022.

Semi-Decentralized Federated Learning with Collaborative Relaying

Authors: Michal Yemini, Rajarshi Saha, Emre Ozfatura, Deniz Gündüz, Andrea J. Goldsmith

Abstract: We present a semi-decentralized federated learning algorithm wherein clients collaborate by relaying their neighbors' local updates to a central parameter server (PS). At every communication round to the PS, each client computes a local consensus of the updates from its neighboring clients and eventually transmits a weighted average of its own update and those of its neighbors to the PS. We approp… ▽ More We present a semi-decentralized federated learning algorithm wherein clients collaborate by relaying their neighbors' local updates to a central parameter server (PS). At every communication round to the PS, each client computes a local consensus of the updates from its neighboring clients and eventually transmits a weighted average of its own update and those of its neighbors to the PS. We appropriately optimize these averaging weights to ensure that the global update at the PS is unbiased and to reduce the variance of the global update at the PS, consequently improving the rate of convergence. Numerical simulations substantiate our theoretical claims and demonstrate settings with intermittent connectivity between the clients and the PS, where our proposed algorithm shows an improved convergence rate and accuracy in comparison with the federated averaging algorithm. △ Less

Submitted 22 May, 2022; originally announced May 2022.

Comments: Accepted for presentation at the IEEE ISIT 2022. This is a conference version of arXiv:2202.11850

Composite IG/FTR Channel Performance in Wireless Communication Systems

Authors: Maryam Olyaee, Juan M. Romero-Jerez, F. Javier López-Martínez, Andrea J. Goldsmith

Abstract: We present a composite wireless fading model encompassing multipath fading and shadowing based on fluctuating two-ray (FTR) fading and inverse gamma (IG) shadowing. We first determine an alternative framework for the statistical characterization and performance evaluation of the FTR fading model, which is based on the fact that the FTR fading distribution can be described as an underlying Rician S… ▽ More We present a composite wireless fading model encompassing multipath fading and shadowing based on fluctuating two-ray (FTR) fading and inverse gamma (IG) shadowing. We first determine an alternative framework for the statistical characterization and performance evaluation of the FTR fading model, which is based on the fact that the FTR fading distribution can be described as an underlying Rician Shadowed (RS) distribution with continuously varying parameter Kr (ratio of specular to diffuse components). We demonstrate that this new formulation permits to obtain a closed-form expression of the generalized moment generating function (GMGF) of the FTR model, from which the PDF and CDF of the composite IG/FTR model can be obtained in closed-form. The exact and asymptotic outage probability of the IG/FTR model are analyzed and verified by Monte Carlo simulations. △ Less

Submitted 9 May, 2022; originally announced May 2022.

Comments: This work has been submitted to the IEEE for publication. Copyright may be transferred without notice, after which this version may no longer be accessible. arXiv admin note: substantial text overlap with arXiv:2108.02990

Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying

Authors: Michal Yemini, Rajarshi Saha, Emre Ozfatura, Deniz Gündüz, Andrea J. Goldsmith

Abstract: Intermittent connectivity of clients to the parameter server (PS) is a major bottleneck in federated edge learning frameworks. The lack of constant connectivity induces a large generalization gap, especially when the local data distribution amongst clients exhibits heterogeneity. To overcome intermittent communication outages between clients and the central PS, we introduce the concept of collabor… ▽ More Intermittent connectivity of clients to the parameter server (PS) is a major bottleneck in federated edge learning frameworks. The lack of constant connectivity induces a large generalization gap, especially when the local data distribution amongst clients exhibits heterogeneity. To overcome intermittent communication outages between clients and the central PS, we introduce the concept of collaborative relaying wherein the participating clients relay their neighbors' local updates to the PS in order to boost the participation of clients with poor connectivity to the PS. We propose a semi-decentralized federated learning framework in which at every communication round, each client initially computes a local consensus of a subset of its neighboring clients' updates, and eventually transmits to the PS a weighted average of its own update and those of its neighbors'. We appropriately optimize these local consensus weights to ensure that the global update at the PS is unbiased with minimal variance - consequently improving the convergence rate. Numerical evaluations on the CIFAR-10 dataset demonstrate that our collaborative relaying approach outperforms federated averaging-based benchmarks for learning over intermittently-connected networks such as when the clients communicate over millimeter wave channels with intermittent blockages. △ Less

Submitted 20 October, 2022; v1 submitted 23 February, 2022; originally announced February 2022.

Minimax Optimal Quantization of Linear Models: Information-Theoretic Limits and Efficient Algorithms

Authors: Rajarshi Saha, Mert Pilanci, Andrea J. Goldsmith

Abstract: High-dimensional models often have a large memory footprint and must be quantized after training before being deployed on resource-constrained edge devices for inference tasks. In this work, we develop an information-theoretic framework for the problem of quantizing a linear regressor learned from training data $(\mathbf{X}, \mathbf{y})$, for some underlying statistical relationship… ▽ More High-dimensional models often have a large memory footprint and must be quantized after training before being deployed on resource-constrained edge devices for inference tasks. In this work, we develop an information-theoretic framework for the problem of quantizing a linear regressor learned from training data $(\mathbf{X}, \mathbf{y})$, for some underlying statistical relationship $\mathbf{y} = \mathbf{X}\boldsymbolθ + \mathbf{v}$. The learned model, which is an estimate of the latent parameter $\boldsymbolθ \in \mathbb{R}^d$, is constrained to be representable using only $Bd$ bits, where $B \in (0, \infty)$ is a pre-specified budget and $d$ is the dimension. We derive an information-theoretic lower bound for the minimax risk under this setting and propose a matching upper bound using randomized embedding-based algorithms which is tight up to constant factors. The lower and upper bounds together characterize the minimum threshold bit-budget required to achieve a performance risk comparable to the unquantized setting. We also propose randomized Hadamard embeddings that are computationally efficient and are optimal up to a mild logarithmic factor of the lower bound. Our model quantization strategy can be generalized and we show its efficacy by extending the method and upper-bounds to two-layer ReLU neural networks for non-linear regression. Numerical simulations show the improved performance of our proposed scheme as well as its closeness to the lower bound. △ Less

Submitted 30 August, 2022; v1 submitted 22 February, 2022; originally announced February 2022.

Comments: 50 pages, 31 figures, 9 tables

Using Non-Stationary Bandits for Learning in Repeated Cournot Games with Non-Stationary Demand

Authors: Kshitija Taywade, Brent Harrison, Judy Goldsmith

Abstract: Many past attempts at modeling repeated Cournot games assume that demand is stationary. This does not align with real-world scenarios in which market demands can evolve over a product's lifetime for a myriad of reasons. In this paper, we model repeated Cournot games with non-stationary demand such that firms/agents face separate instances of non-stationary multi-armed bandit problem. The set of ar… ▽ More Many past attempts at modeling repeated Cournot games assume that demand is stationary. This does not align with real-world scenarios in which market demands can evolve over a product's lifetime for a myriad of reasons. In this paper, we model repeated Cournot games with non-stationary demand such that firms/agents face separate instances of non-stationary multi-armed bandit problem. The set of arms/actions that an agent can choose from represents discrete production quantities; here, the action space is ordered. Agents are independent and autonomous, and cannot observe anything from the environment; they can only see their own rewards after taking an action, and only work towards maximizing these rewards. We propose a novel algorithm 'Adaptive with Weighted Exploration (AWE) $ε$-greedy' which is remotely based on the well-known $ε$-greedy approach. This algorithm detects and quantifies changes in rewards due to varying market demand and varies learning rate and exploration rate in proportion to the degree of changes in demand, thus enabling agents to better identify new optimal actions. For efficient exploration, it also deploys a mechanism for weighing actions that takes advantage of the ordered action space. We use simulations to study the emergence of various equilibria in the market. In addition, we study the scalability of our approach in terms number of total agents in the system and the size of action space. We consider both symmetric and asymmetric firms in our models. We found that using our proposed method, agents are able to swiftly change their course of action according to the changes in demand, and they also engage in collusive behavior in many simulations. △ Less

Submitted 3 January, 2022; originally announced January 2022.

Comments: 13 pages

Cloud-Cluster Architecture for Detection in Intermittently Connected Sensor Networks

Authors: Michal Yemini, Stephanie Gil, Andrea J. Goldsmith

Abstract: We consider a centralized detection problem where sensors experience noisy measurements and intermittent connectivity to a centralized fusion center. The sensors collaborate locally within predefined sensor clusters and fuse their noisy sensor data to reach a common local estimate of the detected event in each cluster. The connectivity of each sensor cluster is intermittent and depends on the avai… ▽ More We consider a centralized detection problem where sensors experience noisy measurements and intermittent connectivity to a centralized fusion center. The sensors collaborate locally within predefined sensor clusters and fuse their noisy sensor data to reach a common local estimate of the detected event in each cluster. The connectivity of each sensor cluster is intermittent and depends on the available communication opportunities of the sensors to the fusion center. Upon receiving the estimates from all the connected sensor clusters the fusion center fuses the received estimates to make a final determination regarding the occurrence of the event across the deployment area. We refer to this hybrid communication scheme as a \emph{cloud-cluster} architecture. We propose a method for optimizing the decision rule for each cluster and analyzing the expected detection performance resulting from our hybrid scheme. Our method is tractable and addresses the high computational complexity caused by heterogeneous sensors' and clusters' detection quality, heterogeneity in their communication opportunities, and non-convexity of the loss function. Our analysis shows that clustering the sensors provides resilience to noise in the case of low sensor communication probability with the cloud. For larger clusters, a steep improvement in detection performance is possible even for a low communication probability by using our cloud-cluster architecture. △ Less

Submitted 21 August, 2022; v1 submitted 3 October, 2021; originally announced October 2021.

Comments: Accepted for publication in the IEEE Transactions on Wireless Communications. Preliminary results were presented in part at the IEEE Global Communications Conference 2020, arXiv:2005.12495

Alternative Formulations for the Fluctuating Two-Ray Fading Model

Authors: Maryam Olyaee, Juan M. Romero-Jerez, F. Javier Lopez-Martinez, Andrea J. Goldsmith

Abstract: We present two alternative formulations for the distribution of the fluctuating two-ray (FTR) fading model, which simplify its statistical characterization and subsequent use for performance evaluation. New expressions for the probability density function (PDF) and cumulative distribution function of the FTR model are obtained based on the observation that the FTR fading distribution is described,… ▽ More We present two alternative formulations for the distribution of the fluctuating two-ray (FTR) fading model, which simplify its statistical characterization and subsequent use for performance evaluation. New expressions for the probability density function (PDF) and cumulative distribution function of the FTR model are obtained based on the observation that the FTR fading distribution is described, for arbitrary $m$, as an underlying Rician Shadowed (RS) distribution with continuously varying parameter $K$, while for the special case of $m$ being an integer, the FTR fading model is described in terms of a finite number of underlying squared Nakagami-$m$ distributions. It is shown that the chief statistics and any performance metric that are computed by averaging over the PDF of the FTR fading model can be expressed in terms of a finite-range integral over the corresponding statistic or performance metric for the RS (for arbitrary $m$) or the Nakagami-$m$ (for integer $m$) fading models, which have a simpler analytical characterization than the FTR model and for which many results are available in closed-form. New expressions for some Laplace-domain statistics of interest are also obtained; these are used to exemplify the practical relevance of this new formulation for performance analysis. △ Less

Submitted 23 August, 2021; v1 submitted 6 August, 2021; originally announced August 2021.

Comments: This work has been submitted to the IEEE for publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Efficient Randomized Subspace Embeddings for Distributed Optimization under a Communication Budget

Authors: Rajarshi Saha, Mert Pilanci, Andrea J. Goldsmith

Abstract: We study first-order optimization algorithms under the constraint that the descent direction is quantized using a pre-specified budget of $R$-bits per dimension, where $R \in (0 ,\infty)$. We propose computationally efficient optimization algorithms with convergence rates matching the information-theoretic performance lower bounds for: (i) Smooth and Strongly-Convex objectives with access to an Ex… ▽ More We study first-order optimization algorithms under the constraint that the descent direction is quantized using a pre-specified budget of $R$-bits per dimension, where $R \in (0 ,\infty)$. We propose computationally efficient optimization algorithms with convergence rates matching the information-theoretic performance lower bounds for: (i) Smooth and Strongly-Convex objectives with access to an Exact Gradient oracle, as well as (ii) General Convex and Non-Smooth objectives with access to a Noisy Subgradient oracle. The crux of these algorithms is a polynomial complexity source coding scheme that embeds a vector into a random subspace before quantizing it. These embeddings are such that with high probability, their projection along any of the canonical directions of the transform space is small. As a consequence, quantizing these embeddings followed by an inverse transform to the original space yields a source coding method with optimal covering efficiency while utilizing just $R$-bits per dimension. Our algorithms guarantee optimality for arbitrary values of the bit-budget $R$, which includes both the sub-linear budget regime ($R < 1$), as well as the high-budget regime ($R \geq 1$), while requiring $O\left(n^2\right)$ multiplications, where $n$ is the dimension. We also propose an efficient relaxation of this coding scheme using Hadamard subspaces that requires a near-linear time, i.e., $O\left(n \log n\right)$ additions.Furthermore, we show that the utility of our proposed embeddings can be extended to significantly improve the performance of gradient sparsification schemes. Numerical simulations validate our theoretical claims. Our implementations are available at https://github.com/rajarshisaha95/DistOptConstrComm. △ Less

Submitted 15 August, 2022; v1 submitted 12 March, 2021; originally announced March 2021.

Comments: 41 pages, 26 figures, 1 table. This work has been accepted for publication in the IEEE Journal on Selected Areas in Information Theory (JSAIT), Spl. issue on Distributed Coding and Computation

Model-Based Machine Learning for Communications

Authors: Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: We present an introduction to model-based machine learning for communication systems. We begin by reviewing existing strategies for combining model-based algorithms and machine learning from a high level perspective, and compare them to the conventional deep learning approach which utilizes established deep neural network (DNN) architectures trained in an end-to-end manner. Then, we focus on symbo… ▽ More We present an introduction to model-based machine learning for communication systems. We begin by reviewing existing strategies for combining model-based algorithms and machine learning from a high level perspective, and compare them to the conventional deep learning approach which utilizes established deep neural network (DNN) architectures trained in an end-to-end manner. Then, we focus on symbol detection, which is one of the fundamental tasks of communication receivers. We show how the different strategies of conventional deep architectures, deep unfolding, and DNN-aided hybrid algorithms, can be applied to this problem. The last two approaches constitute a middle ground between purely model-based and solely DNN-based receivers. By focusing on this specific task, we highlight the advantages and drawbacks of each strategy, and present guidelines to facilitate the design of future model-based deep learning systems for communications. △ Less

Submitted 12 January, 2021; originally announced January 2021.

Comments: arXiv admin note: text overlap with arXiv:2002.07806

Interference Reduction in Virtual Cell Optimization

Authors: Michal Yemini, Elza Erkip, Andrea J. Goldsmith

Abstract: Virtual cell optimization clusters cells into neighborhoods and performs optimized resource allocation over each neighborhood. In prior works we proposed resource allocation schemes to mitigate the interference caused by transmissions in the same virtual cell. This work aims at mitigating both the interference caused by the transmissions of users in the same virtual cell and the interference betwe… ▽ More Virtual cell optimization clusters cells into neighborhoods and performs optimized resource allocation over each neighborhood. In prior works we proposed resource allocation schemes to mitigate the interference caused by transmissions in the same virtual cell. This work aims at mitigating both the interference caused by the transmissions of users in the same virtual cell and the interference between transmissions in different virtual cells. We propose a resource allocation technique that reduces the number of users that cannot achieve their constant guaranteed bit rate, i.e., the "unsatisfied users", in an uplink virtual cell system with cooperative decoding. The proposed scheme requires only the knowledge of the number of users each base station serves and relies on creating the interference graph between base stations at the edges of virtual cells. Allocation of frequency bands to users is based on the number of users each base station would serve in a non cooperative setup. We evaluate the performance of our scheme for a mmWave system. Our numerical results show that our scheme decreases the number of users in the system whose rate falls below the guaranteed rate, set to $128$kbps, $256$kbps or $512$kbps, when compared with our previously proposed optimization methods. △ Less

Submitted 13 November, 2021; v1 submitted 30 October, 2020; originally announced October 2020.

Opinion Diffusion Software with Strategic Opinion Revelation and Unfriending

Authors: Patrick Shepherd, Mia Weaver, Judy Goldsmith

Abstract: We present a novel software suite for social network modeling and opinion diffusion processes. Much research on social network science has assumed networks with static topologies. More recently, attention has been turned to networks that evolve. Although software for modeling both the topological evolution of networks and diffusion processes are constantly improving, very little attention has been… ▽ More We present a novel software suite for social network modeling and opinion diffusion processes. Much research on social network science has assumed networks with static topologies. More recently, attention has been turned to networks that evolve. Although software for modeling both the topological evolution of networks and diffusion processes are constantly improving, very little attention has been paid to agent modeling. Our software is designed to be robust, modular, and extensible, providing the ability to model dynamic social network topologies and multidimensional diffusion processes, different styles of agent including non-homophilic paradigms, as well as a testing environment for multi-agent reinforcement learning (MARL) experiments with diverse sets of agent types. We also illustrate the value of diverse agent modeling, and environments that allow for strategic unfriending. Our work shows that polarization and consensus dynamics, as well as topological clustering effects, may rely more than previously known on individuals' goals for the composition of their neighborhood's opinions. △ Less

Submitted 22 June, 2020; originally announced June 2020.

Comments: 8 pages, 4 figures

Learned Factor Graphs for Inference from Stationary Time Sequences

Authors: Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: The design of methods for inference from time sequences has traditionally relied on statistical models that describe the relation between a latent desired sequence and the observed one. A broad family of model-based algorithms have been derived to carry out inference at controllable complexity using recursive computations over the factor graph representing the underlying distribution. An alternati… ▽ More The design of methods for inference from time sequences has traditionally relied on statistical models that describe the relation between a latent desired sequence and the observed one. A broad family of model-based algorithms have been derived to carry out inference at controllable complexity using recursive computations over the factor graph representing the underlying distribution. An alternative model-agnostic approach utilizes machine learning (ML) methods. Here we propose a framework that combines model-based algorithms and data-driven ML tools for stationary time sequences. In the proposed approach, neural networks are developed to separately learn specific components of a factor graph describing the distribution of the time sequence, rather than the complete inference task. By exploiting stationary properties of this distribution, the resulting approach can be applied to sequences of varying temporal duration. Learned factor graph can be realized using compact neural networks that are trainable using small training sets, or alternatively, be used to improve upon existing deep inference systems. We present an inference algorithm based on learned stationary factor graphs, which learns to implement the sum-product scheme from labeled data, and can be applied to sequences of different lengths. Our experimental results demonstrate the ability of the proposed learned factor graphs to learn to carry out accurate inference from small training sets for sleep stage detection using the Sleep-EDF dataset, as well as for symbol detection in digital communications with unknown channels. △ Less

Submitted 24 December, 2021; v1 submitted 5 June, 2020; originally announced June 2020.

Regulating Artificial Intelligence: Proposal for a Global Solution

Authors: Olivia J. Erdélyi, Judy Goldsmith

Abstract: With increasing ubiquity of artificial intelligence (AI) in modern societies, individual countries and the international community are working hard to create an innovation-friendly, yet safe, regulatory environment. Adequate regulation is key to maximize the benefits and minimize the risks stemming from AI technologies. Develo** regulatory frameworks is, however, challenging due to AI's global r… ▽ More With increasing ubiquity of artificial intelligence (AI) in modern societies, individual countries and the international community are working hard to create an innovation-friendly, yet safe, regulatory environment. Adequate regulation is key to maximize the benefits and minimize the risks stemming from AI technologies. Develo** regulatory frameworks is, however, challenging due to AI's global reach and the existence of widespread misconceptions about the notion of regulation. We argue that AI-related challenges cannot be tackled effectively without sincere international coordination supported by robust, consistent domestic and international governance arrangements. Against this backdrop, we propose the establishment of an international AI governance framework organized around a new AI regulatory agency that -- drawing on interdisciplinary expertise -- could help creating uniform standards for the regulation of AI technologies and inform the development of AI policies around the world. We also believe that a fundamental change of mindset on what constitutes regulation is necessary to remove existing barriers that hamper contemporary efforts to develop AI regulatory regimes, and put forward some recommendations on how to achieve this, and what opportunities doing so would present. △ Less

Submitted 22 May, 2020; originally announced May 2020.

Comments: 25 pages. A preliminary version appeared in the Proceedings of the First AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, pages 95-101, 2018

Multi-agent Reinforcement Learning for Decentralized Stable Matching

Authors: Kshitija Taywade, Judy Goldsmith, Brent Harrison

Abstract: In the real world, people/entities usually find matches independently and autonomously, such as finding jobs, partners, roommates, etc. It is possible that this search for matches starts with no initial knowledge of the environment. We propose the use of a multi-agent reinforcement learning (MARL) paradigm for a spatially formulated decentralized two-sided matching market with independent and auto… ▽ More In the real world, people/entities usually find matches independently and autonomously, such as finding jobs, partners, roommates, etc. It is possible that this search for matches starts with no initial knowledge of the environment. We propose the use of a multi-agent reinforcement learning (MARL) paradigm for a spatially formulated decentralized two-sided matching market with independent and autonomous agents. Having autonomous agents acting independently makes our environment very dynamic and uncertain. Moreover, agents lack the knowledge of preferences of other agents and have to explore the environment and interact with other agents to discover their own preferences through noisy rewards. We think such a setting better approximates the real world and we study the usefulness of our MARL approach for it. Along with conventional stable matching case where agents have strictly ordered preferences, we check the applicability of our approach for stable matching with incomplete lists and ties. We investigate our results for stability, level of instability (for unstable results), and fairness. Our MARL approach mostly yields stable and fair outcomes. △ Less

Submitted 3 December, 2021; v1 submitted 3 May, 2020; originally announced May 2020.

Comments: 16 pages

Journal ref: 7th International Conference on Algorithmic Decision Theory, 2021

Data-Driven Symbol Detection via Model-Based Machine Learning

Authors: Nariman Farsad, Nir Shlezinger, Andrea J. Goldsmith, Yonina C. Eldar

Abstract: The design of symbol detectors in digital communication systems has traditionally relied on statistical channel models that describe the relation between the transmitted symbols and the observed signal at the receiver. Here we review a data-driven framework to symbol detection design which combines machine learning (ML) and model-based algorithms. In this hybrid approach, well-known channel-model-… ▽ More The design of symbol detectors in digital communication systems has traditionally relied on statistical channel models that describe the relation between the transmitted symbols and the observed signal at the receiver. Here we review a data-driven framework to symbol detection design which combines machine learning (ML) and model-based algorithms. In this hybrid approach, well-known channel-model-based algorithms such as the Viterbi method, BCJR detection, and multiple-input multiple-output (MIMO) soft interference cancellation (SIC) are augmented with ML-based algorithms to remove their channel-model-dependence, allowing the receiver to learn to implement these algorithms solely from data. The resulting data-driven receivers are most suitable for systems where the underlying channel models are poorly understood, highly complex, or do not well-capture the underlying physics. Our approach is unique in that it only replaces the channel-model-based computations with dedicated neural networks that can be trained from a small amount of data, while kee** the general algorithm intact. Our results demonstrate that these techniques can yield near-optimal performance of model-based algorithms without knowing the exact channel input-output statistical relationship and in the presence of channel state information uncertainty. △ Less

Submitted 14 February, 2020; originally announced February 2020.

Comments: arXiv admin note: text overlap with arXiv:1905.10750

Data-Driven Factor Graphs for Deep Symbol Detection

Authors: Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: Many important schemes in signal processing and communications, ranging from the BCJR algorithm to the Kalman filter, are instances of factor graph methods. This family of algorithms is based on recursive message passing-based computations carried out over graphical models, representing a factorization of the underlying statistics. Consequently, in order to implement these algorithms, one must hav… ▽ More Many important schemes in signal processing and communications, ranging from the BCJR algorithm to the Kalman filter, are instances of factor graph methods. This family of algorithms is based on recursive message passing-based computations carried out over graphical models, representing a factorization of the underlying statistics. Consequently, in order to implement these algorithms, one must have accurate knowledge of the statistical model of the considered signals. In this work we propose to implement factor graph methods in a data-driven manner. In particular, we propose to use machine learning (ML) tools to learn the factor graph, instead of the overall system task, which in turn is used for inference by message passing over the learned graph. We apply the proposed approach to learn the factor graph representing a finite-memory channel, demonstrating the resulting ability to implement BCJR detection in a data-driven fashion. We demonstrate that the proposed system, referred to as BCJRNet, learns to implement the BCJR algorithm from a small training set, and that the resulting receiver exhibits improved robustness to inaccurate training compared to the conventional channel-model-based receiver operating under the same level of uncertainty. Our results indicate that by utilizing ML tools to learn factor graphs from labeled data, one can implement a broad range of model-based algorithms, which traditionally require full knowledge of the underlying statistics, in a data-driven fashion. △ Less

Submitted 31 January, 2020; originally announced February 2020.

Compressed Sensing Channel Estimation for OFDM with non-Gaussian Multipath Gains

Authors: Felipe Gomez-Cuba, Andrea J. Goldsmith

Abstract: This paper analyzes the impact of non-Gaussian multipath component (MPC) amplitude distributions on the performance of Compressed Sensing (CS) channel estimators for OFDM systems. The number of dominant MPCs that any CS algorithm needs to estimate in order to accurately represent the channel is characterized. This number relates to a Compressibility Index (CI) of the channel that depends on the fo… ▽ More This paper analyzes the impact of non-Gaussian multipath component (MPC) amplitude distributions on the performance of Compressed Sensing (CS) channel estimators for OFDM systems. The number of dominant MPCs that any CS algorithm needs to estimate in order to accurately represent the channel is characterized. This number relates to a Compressibility Index (CI) of the channel that depends on the fourth moment of the MPC amplitude distribution. A connection between the Mean Squared Error (MSE) of any CS estimation algorithm and the MPC amplitude distribution fourth moment is revealed that shows a smaller number of MPCs is needed to well-estimate channels when these components have large fourth moment amplitude gains. The analytical results are validated via simulations for channels with lognormal MPCs such as the NYU mmWave channel model. These simulations show that when the MPC amplitude distribution has a high fourth moment, the well known CS algorithm of Orthogonal Matching Pursuit performs almost identically to the Basis Pursuit De-Noising algorithm with a much lower computational cost. △ Less

Submitted 20 February, 2020; v1 submitted 22 November, 2019; originally announced November 2019.

Comments: Published in IEEE Transactions on Wireless Communications. arXiv admin note: text overlap with arXiv:1812.07236

Capacity scaling in a Non-coherent Wideband Massive SIMO Block Fading Channel

Authors: Felipe Gomez-Cuba, Mainak Chowdhury, Alexandros Manolakos, Elza Erkip, Andrea J. Goldsmith

Abstract: The scaling of coherent and non-coherent channel capacity is studied in a single-input multiple-output (SIMO) block Rayleigh fading channel as both the bandwidth and the number of receiver antennas go to infinity jointly with the transmit power fixed. The transmitter has no channel state information (CSI), while the receiver may have genie-provided CSI (coherent receiver), or the channel statistic… ▽ More The scaling of coherent and non-coherent channel capacity is studied in a single-input multiple-output (SIMO) block Rayleigh fading channel as both the bandwidth and the number of receiver antennas go to infinity jointly with the transmit power fixed. The transmitter has no channel state information (CSI), while the receiver may have genie-provided CSI (coherent receiver), or the channel statistics only (non-coherent receiver). Our results show that if the available bandwidth is smaller than a threshold bandwidth which is proportional (up to leading order terms) to the square root of the number of antennas, there is no gap between the coherent capacity and the non-coherent capacity in terms of capacity scaling behavior. On the other hand, when the bandwidth is larger than this threshold, there is a capacity scaling gap. Since achievable rates using pilot symbols for channel estimation are subject to the non-coherent capacity bound, this work reveals that pilot-assisted coherent receivers in systems with a large number of receive antennas are unable to exploit excess spectrum above a given threshold for capacity gain. △ Less

Submitted 20 February, 2020; v1 submitted 22 November, 2019; originally announced November 2019.

Comments: Published in IEEE Transactions on Wireless Communications

Virtual Cell Clustering with Optimal Resource Allocation to Maximize Capacity

Authors: Michal Yemini, Andrea J. Goldsmith

Abstract: This work proposes a new resource allocation optimization and network management framework for wireless networks using neighborhood-based optimization rather than fully centralized or fully decentralized methods. We propose hierarchical clustering with a minimax linkage criterion for the formation of the virtual cells. Once the virtual cells are formed, we consider two cooperation models: the inte… ▽ More This work proposes a new resource allocation optimization and network management framework for wireless networks using neighborhood-based optimization rather than fully centralized or fully decentralized methods. We propose hierarchical clustering with a minimax linkage criterion for the formation of the virtual cells. Once the virtual cells are formed, we consider two cooperation models: the interference coordination model and the coordinated multi-point decoding model. In the first model base stations in a virtual cell decode their signals independently, but allocate the communication resources cooperatively. In the second model base stations in the same virtual cell allocate the communication resources and decode their signals cooperatively. We address the resource allocation problem for each of these cooperation models. For the interference coordination model this problem is an NP-hard mixed-integer optimization problem whereas for the coordinated multi-point decoding model it is convex. Our numerical results indicate that proper design of the neighborhood-based optimization leads to significant gains in sum rate over fully decentralized optimization, yet may also have a significant sum rate penalty compared to fully centralized optimization. In particular, neighborhood-based optimization has a significant sum rate penalty compared to fully centralized optimization in the coordinated multi-point model, but not the interference coordination model. △ Less

Submitted 6 November, 2019; originally announced November 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1905.02891

Deep Neural Network Symbol Detection for Millimeter Wave Communications

Authors: Yun Liao, Nariman Farsad, Nir Shlezinger, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: This paper proposes to use a deep neural network (DNN)-based symbol detector for mmWave systems such that CSI acquisition can be bypassed. In particular, we consider a sliding bidirectional recurrent neural network (BRNN) architecture that is suitable for the long memory length of typical mmWave channels. The performance of the DNN detector is evaluated in comparison to that of the Viterbi detecto… ▽ More This paper proposes to use a deep neural network (DNN)-based symbol detector for mmWave systems such that CSI acquisition can be bypassed. In particular, we consider a sliding bidirectional recurrent neural network (BRNN) architecture that is suitable for the long memory length of typical mmWave channels. The performance of the DNN detector is evaluated in comparison to that of the Viterbi detector. The results show that the performance of the DNN detector is close to that of the optimal Viterbi detector with perfect CSI, and that it outperforms the Viterbi algorithm with CSI estimation error. Further experiments show that the DNN detector is robust to a wide range of noise levels and varying channel conditions, and that a pretrained detector can be reliably applied to different mmWave channel realizations with minimal overhead. △ Less

Submitted 25 July, 2019; originally announced July 2019.

ViterbiNet: A Deep Learning Based Viterbi Algorithm for Symbol Detection

Authors: Nir Shlezinger, Nariman Farsad, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: Symbol detection plays an important role in the implementation of digital receivers. In this work, we propose ViterbiNet, which is a data-driven symbol detector that does not require channel state information (CSI). ViterbiNet is obtained by integrating deep neural networks (DNNs) into the Viterbi algorithm. We identify the specific parts of the Viterbi algorithm that are channel-model-based, and… ▽ More Symbol detection plays an important role in the implementation of digital receivers. In this work, we propose ViterbiNet, which is a data-driven symbol detector that does not require channel state information (CSI). ViterbiNet is obtained by integrating deep neural networks (DNNs) into the Viterbi algorithm. We identify the specific parts of the Viterbi algorithm that are channel-model-based, and design a DNN to implement only those computations, leaving the rest of the algorithm structure intact. We then propose a meta-learning based approach to train ViterbiNet online based on recent decisions, allowing the receiver to track dynamic channel conditions without requiring new training samples for every coherence block. Our numerical evaluations demonstrate that the performance of ViterbiNet, which is ignorant of the CSI, approaches that of the CSI-based Viterbi algorithm, and is capable of tracking time-varying channels without needing instantaneous CSI or additional training data. Moreover, unlike conventional Viterbi detection, ViterbiNet is robust to CSI uncertainty, and it can be reliably implemented in complex channel models with constrained computational burden. More broadly, our results demonstrate the conceptual benefit of designing communication systems to that integrate DNNs into established algorithms. △ Less

Submitted 29 September, 2020; v1 submitted 26 May, 2019; originally announced May 2019.

Comments: arXiv admin note: text overlap with arXiv:2002.07806

Virtual Cell Clustering with Optimal Resource Allocation to Maximize Cellular System Capacity

Authors: Michal Yemini, Andrea J. Goldsmith

Abstract: This work presents a new network optimization framework for cellular networks using neighborhood-based optimization. Under this optimization framework resources are allocated within virtual cells encompassing several base-stations and the users within their coverage areas. We form the virtual cells using hierarchical clustering with a minimax linkage criterion given a particular number of such cel… ▽ More This work presents a new network optimization framework for cellular networks using neighborhood-based optimization. Under this optimization framework resources are allocated within virtual cells encompassing several base-stations and the users within their coverage areas. We form the virtual cells using hierarchical clustering with a minimax linkage criterion given a particular number of such cells. Once the virtual cells are formed, we consider an interference coordination model in which base-stations in a virtual cell jointly allocate the channels and power to users within the virtual cell. We propose two new schemes for solving this mixed integer NP-hard resource allocation problem. The first scheme transforms the problem into a continuous variables problem; the second scheme proposes a new channel allocation method and then alternately solves the channel allocation problem using this new method, and the power allocation problem. We evaluate the average system sum rate of these schemes for a variable number of virtual cells. These results quantify the sum-rate along a continuum of fully-centralized versus fully-distributed optimization for different clustering and resource allocation strategies. These results indicate that the penalty of fully-distributed optimization versus fully-centralized (cloud RAN) can be as high as 50%. However, if designed properly, a few base stations within a virtual cell using neighborhood-based optimization have almost the same performance as fully-centralized optimization. △ Less

Submitted 6 May, 2019; originally announced May 2019.

Comments: arXiv admin note: text overlap with arXiv:1901.06669, arXiv:1905.02184

Optimal Resource Allocation for Cellular Networks with Virtual Cell Joint Decoding

Authors: Michal Yemini, Andrea J. Goldsmith

Abstract: This work presents a new resource allocation optimization framework for cellular networks using neighborhood-based optimization. Under this optimization framework resources are allocated within virtual cells encompassing several base-stations and the users within their coverage area. Incorporating the virtual cell concept enables the utilization of more sophisticated cooperative communication sche… ▽ More This work presents a new resource allocation optimization framework for cellular networks using neighborhood-based optimization. Under this optimization framework resources are allocated within virtual cells encompassing several base-stations and the users within their coverage area. Incorporating the virtual cell concept enables the utilization of more sophisticated cooperative communication schemes such as coordinated multi-point decoding. We form the virtual cells using hierarchical clustering given a particular number of such cells. Once the virtual cells are formed, we consider a cooperative decoding scheme in which the base-stations in each virtual cell jointly decode the signals that they receive. We propose an iterative solution for the resource allocation problem resulting from the cooperative decoding within each virtual cell. Numerical results for the average system sum rate of our network design under hierarchical clustering are presented. These results indicate that virtual cells with neighborhood-based optimization leads to significant gains in sum rate over optimization within each cell, yet may also have a significant sum-rate penalty compared to fully-centralized optimization. △ Less

Submitted 6 May, 2019; originally announced May 2019.

Comments: Accepted to the International Symposium on Information Theory (ISIT-2019)

"Fog" Optimization via Virtual Cells in Cellular Network Resource Allocation

Authors: Michal Yemini, Andrea J. Goldsmith

Abstract: This work proposes a new resource allocation optimization framework for cellular networks using "fog" or neighborhood-based optimization rather than fully centralized or fully decentralized methods. In neighborhood-based optimization resources are allocated within virtual cells encompassing several base-stations and the users within their coverage area. As the number of base-stations within a virt… ▽ More This work proposes a new resource allocation optimization framework for cellular networks using "fog" or neighborhood-based optimization rather than fully centralized or fully decentralized methods. In neighborhood-based optimization resources are allocated within virtual cells encompassing several base-stations and the users within their coverage area. As the number of base-stations within a virtual cell increases, the framework reverts to centralized optimization, and as this number decreases it reverts to decentralized optimization. We address two tasks that must be carried out in the fog optimization framework: forming the virtual cells and allocating the communication resources in each virtual cell effectively. We propose hierarchical clustering for the formation of the virtual cells given a particular number of such cells. Once the virtual cells are formed, we consider several optimization methods to solve the NP-hard joint channel access and power allocation problem within each virtual cell in order to maximize the sum rate of the entire system. We present numerical results for the system sum rate of each scheme under hierarchical clustering. Our results indicate that proper design of the fog optimization results in little degradation relative to centralized optimization even for a relatively large number of virtual cells. However, improper design leads to a significant decrease in sum rate relative to centralized optimization. △ Less

Submitted 20 January, 2019; originally announced January 2019.

Sparse mmWave OFDM Channel Estimation Using Compressed Sensing in OFDM Systems

Authors: Felipe Gomez-Cuba, Andrea J. Goldsmith

Abstract: This paper proposes and analyzes a mmWave sparse channel estimation technique for OFDM systems that uses the Orthogonal Matching Pursuit (OMP) algorithm. This greedy algorithm retrieves one additional multipath component (MPC) per iteration until a stop condition is met. We obtain an analytical approximation for the OMP estimation error variance that grows with the number of retrieved MPCs (iterat… ▽ More This paper proposes and analyzes a mmWave sparse channel estimation technique for OFDM systems that uses the Orthogonal Matching Pursuit (OMP) algorithm. This greedy algorithm retrieves one additional multipath component (MPC) per iteration until a stop condition is met. We obtain an analytical approximation for the OMP estimation error variance that grows with the number of retrieved MPCs (iterations). The OMP channel estimator error variance outperforms a classic maximum-likelihood (ML) non-sparse channel estimator by a factor of approximately $2\hat{L}/M$ where $\hat{L}$ is the number of retrieved MPCs (iterations) and $M$ the number of taps of the Discrete Equivalent Channel. When the MPC amplitude distribution is heavy-tailed, the channel power is concentrated in a subset of dominant MPCs. In this case OMP performs fewer iterations as it retrieves only these dominant large MPCs. Hence for this MPC amplitude distribution the estimation error advantage of OMP over ML is improved. In particular, for channels with MPCs that have lognormally-distributed amplitudes, the OMP estimator recovers approximately 5-15 dominant MPCs in typical mmWave channels, with 15-45 weak MPCs that remain undetected. △ Less

Submitted 18 December, 2018; originally announced December 2018.

Comments: Preprint submitted to IEEE ICC 2019

Orthogonal Time Frequency Space Modulation

Authors: Ronny Hadani, Shlomo Rakib, Shachar Kons, Michael Tsatsanis, Anton Monk, Christian Ibars, Jim Delfeld, Yoav Hebron, Andrea J. Goldsmith, Andreas F. Molisch, Robert Calderbank

Abstract: This paper introduces a new two-dimensional modulation technique called Orthogonal Time Frequency Space (OTFS) modulation. OTFS has the novel and important feature of being designed in the delay-Doppler domain. When coupled with a suitable equalizer, OTFS modulation is able to exploit the full channel diversity over both time and frequency. Moreover, it converts the fading, time-varying wireless c… ▽ More This paper introduces a new two-dimensional modulation technique called Orthogonal Time Frequency Space (OTFS) modulation. OTFS has the novel and important feature of being designed in the delay-Doppler domain. When coupled with a suitable equalizer, OTFS modulation is able to exploit the full channel diversity over both time and frequency. Moreover, it converts the fading, time-varying wireless channel experienced by modulated signals such as OFDM into a time-independent channel with a complex channel gain that is essentially constant for all symbols. This design obviates the need for transmitter adaptation, and greatly simplifies system operation. The paper describes the basic operating principles of OTFS as well as a possible implementation as an overlay to current or anticipated standardized systems. OTFS is shown to provide significant performance improvement in systems with high Doppler, short packets, and/or large antenna array. In particular, simulation results indicate at least several dB of block error rate performance improvement for OTFS over OFDM in all of these settings. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Comments: 13 pages, 9 figures

Blind Joint MIMO Channel Estimation and Decoding

Authors: Thomas R. Dean, Mary Wootters, Andrea J. Goldsmith

Abstract: We propose a method for MIMO decoding when channel state information (CSI) is unknown to both the transmitter and receiver. The proposed method requires some structure in the transmitted signal for the decoding to be effective, in particular that the underlying sources are drawn from a hypercubic space. Our proposed technique fits a minimum volume parallelepiped to the received samples. This probl… ▽ More We propose a method for MIMO decoding when channel state information (CSI) is unknown to both the transmitter and receiver. The proposed method requires some structure in the transmitted signal for the decoding to be effective, in particular that the underlying sources are drawn from a hypercubic space. Our proposed technique fits a minimum volume parallelepiped to the received samples. This problem can be expressed as a non-convex optimization problem that can be solved with high probability by gradient descent. Our blind decoding algorithm can be used when communicating over unknown MIMO wireless channels using either BPSK or MPAM modulation. We apply our technique to jointly estimate MIMO channel gain matrices and decode the underlying transmissions with only knowledge of the transmitted constellation and without the use of pilot symbols. Our results provide theoretical guarantees that the proposed algorithm is correct when applied to small MIMO systems. Empirical results show small sample size requirements, making this algorithm suitable for block-fading channels with coherence times typically seen in practice. Our approach has a loss of less than 3dB compared to zero-forcing with perfect CSI, imposing a similar performance penalty as space-time coding techniques without the loss of rate incurred by those techniques. △ Less

Submitted 3 February, 2018; originally announced February 2018.

Comments: 17 pages, 7 figures, submitted to IEEE Transactions on Information Theory, presented in part at IEEE Globecom 2017

Analog-to-Digital Compression: A New Paradigm for Converting Signals to Bits

Authors: Alon Kipnis, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: Processing, storing and communicating information that originates as an analog signal involves conversion of this information to bits. This conversion can be described by the combined effect of sampling and quantization, as illustrated in Fig. 1. The digital representation is achieved by first sampling the analog signal so as to represent it by a set of discrete-time samples and then quantizing th… ▽ More Processing, storing and communicating information that originates as an analog signal involves conversion of this information to bits. This conversion can be described by the combined effect of sampling and quantization, as illustrated in Fig. 1. The digital representation is achieved by first sampling the analog signal so as to represent it by a set of discrete-time samples and then quantizing these samples to a finite number of bits. Traditionally, these two operations are considered separately. The sampler is designed to minimize information loss due to sampling based on characteristics of the continuous-time input. The quantizer is designed to represent the samples as accurately as possible, subject to a constraint on the number of bits that can be used in the representation. The goal of this article is to revisit this paradigm by illuminating the dependency between these two operations. In particular, we explore the requirements on the sampling system subject to constraints on the available number of bits for storing, communicating or processing the analog information. △ Less

Submitted 20 January, 2018; originally announced January 2018.

Comments: to appear in "Signal Processing Magazine"

The Complexity of Campaigning

Authors: Cory Siler, Luke Harold Miles, Judy Goldsmith

Abstract: In "The Logic of Campaigning", Dean and Parikh consider a candidate making campaign statements to appeal to the voters. They model these statements as Boolean formulas over variables that represent stances on the issues, and study optimal candidate strategies under three proposed models of voter preferences based on the assignments that satisfy these formulas. We prove that voter utility evaluatio… ▽ More In "The Logic of Campaigning", Dean and Parikh consider a candidate making campaign statements to appeal to the voters. They model these statements as Boolean formulas over variables that represent stances on the issues, and study optimal candidate strategies under three proposed models of voter preferences based on the assignments that satisfy these formulas. We prove that voter utility evaluation is computationally hard under these preference models (in one case, #P-hard), along with certain problems related to candidate strategic reasoning. Our results raise questions about the desirable characteristics of a voter preference model and to what extent a polynomial-time-evaluable function can capture them. △ Less

Submitted 17 July, 2017; v1 submitted 19 June, 2017; originally announced June 2017.

Comments: Will be presented at the 2017 Algorithmic Decision Theory Conference

Geometrical morphology

Authors: John Goldsmith, Eric Rosen

Abstract: We explore inflectional morphology as an example of the relationship of the discrete and the continuous in linguistics. The grammar requests a form of a lexeme by specifying a set of feature values, which corresponds to a corner M of a hypercube in feature value space. The morphology responds to that request by providing a morpheme, or a set of morphemes, whose vector sum is geometrically closest… ▽ More We explore inflectional morphology as an example of the relationship of the discrete and the continuous in linguistics. The grammar requests a form of a lexeme by specifying a set of feature values, which corresponds to a corner M of a hypercube in feature value space. The morphology responds to that request by providing a morpheme, or a set of morphemes, whose vector sum is geometrically closest to the corner M. In short, the chosen morpheme $μ$ is the morpheme (or set of morphemes) that maximizes the inner product of $μ$ and M. △ Less

Submitted 13 March, 2017; originally announced March 2017.

Comments: 42 pages

Report number: TR-2017-2

Ethical Considerations in Artificial Intelligence Courses

Authors: Emanuelle Burton, Judy Goldsmith, Sven Koenig, Benjamin Kuipers, Nicholas Mattei, Toby Walsh

Abstract: The recent surge in interest in ethics in artificial intelligence may leave many educators wondering how to address moral, ethical, and philosophical issues in their AI courses. As instructors we want to develop curriculum that not only prepares students to be artificial intelligence practitioners, but also to understand the moral, ethical, and philosophical impacts that artificial intelligence wi… ▽ More The recent surge in interest in ethics in artificial intelligence may leave many educators wondering how to address moral, ethical, and philosophical issues in their AI courses. As instructors we want to develop curriculum that not only prepares students to be artificial intelligence practitioners, but also to understand the moral, ethical, and philosophical impacts that artificial intelligence will have on society. In this article we provide practical case studies and links to resources for use by AI educators. We also provide concrete suggestions on how to integrate AI ethics into a general artificial intelligence course and how to teach a stand-alone artificial intelligence ethics course. △ Less

Submitted 26 January, 2017; originally announced January 2017.

Comments: 29 pages including all case studies and links to video media on YouTube

ACM Class: K.3.2; K.4.1; K.7.m

The Fluctuating Two-Ray Fading Model: Statistical Characterization and Performance Analysis

Authors: Juan M. Romero-Jerez, F. Javier Lopez-Martinez, José F. Paris, Andrea J. Goldsmith

Abstract: We introduce the Fluctuating Two-Ray (FTR) fading model, a new statistical channel model that consists of two fluctuating specular components with random phases plus a diffuse component. The FTR model arises as the natural generalization of the two-wave with diffuse power (TWDP) fading model; this generalization allows its two specular components to exhibit a random amplitude fluctuation. Unlike t… ▽ More We introduce the Fluctuating Two-Ray (FTR) fading model, a new statistical channel model that consists of two fluctuating specular components with random phases plus a diffuse component. The FTR model arises as the natural generalization of the two-wave with diffuse power (TWDP) fading model; this generalization allows its two specular components to exhibit a random amplitude fluctuation. Unlike the TWDP model, all the chief probability functions of the FTR fading model (PDF, CDF and MGF) are expressed in closed-form, having a functional form similar to other state-of-the-art fading models. We also provide approximate closed-form expressions for the PDF and CDF in terms of a finite number of elementary functions, which allow for a simple evaluation of these statistics to an arbitrary level of precision. We show that the FTR fading model provides a much better fit than Rician fading for recent small-scale fading measurements in 28 GHz outdoor millimeter-wave channels. Finally, the performance of wireless communication systems over FTR fading is evaluated in terms of the bit error rate and the outage capacity, and the interplay between the FTR fading model parameters and the system performance is discussed. Monte Carlo simulations have been carried out in order to validate the obtained theoretical expressions. △ Less

Submitted 27 May, 2017; v1 submitted 15 November, 2016; originally announced November 2016.

Comments: This work has been submitted to the IEEE for publication. Copyright may be transferred without notice, after which this version may no longer be accesible

The Distortion-Rate Function of Sampled Wiener Processes

Authors: Alon Kipnis, Andrea J. Goldsmith, Yonina C. Eldar

Abstract: We consider the recovery of a continuous-time Wiener process from a quantized or lossy compressed version of its uniform samples under limited bitrate and sampling rate. We derive a closed form expression for the optimal tradeoff among sampling rate, bitrate, and quadratic distortion in this setting. This expression is given in terms of a reverse waterfilling formula over the asymptotic spectral d… ▽ More We consider the recovery of a continuous-time Wiener process from a quantized or lossy compressed version of its uniform samples under limited bitrate and sampling rate. We derive a closed form expression for the optimal tradeoff among sampling rate, bitrate, and quadratic distortion in this setting. This expression is given in terms of a reverse waterfilling formula over the asymptotic spectral distribution of a sequence of finite-rank operators associated with the optimal estimator of the Wiener process from its samples. We show that the ratio between this expression and the standard distortion rate function of the Wiener process, describing the optimal tradeoff between bitrate and distortion without a sampling constraint, is only a function of the number of bits per sample. For example using one bit per sample on average, the expected distortion is approximately 1.2 times the standard distortion rate function, indicating a performance loss of about 20% due to sampling. We next consider the distortion when the continuous-time process is estimated from the output of an encoder that is optimal with respect to the discrete-time samples. We show that while the latter is strictly greater than the distortion under optimal encoding, the ratio between the two does not exceed 1.027. We therefore conclude that nearly optimal performance is attained even if the encoder is unaware of the sampling rate and encodes the samples without taking into account the continuous-time underlying process. △ Less

Submitted 26 July, 2018; v1 submitted 16 August, 2016; originally announced August 2016.

Comments: Under review. An extended version of a work presented in ISIT 2016 under the title "Information rates of sampled Wiener processes"

Optimal Rate Allocation in Mismatched Multiterminal Source Coding

Authors: Ruiyang Song, Stefano Rini, Alon Kipnis, Andrea J. Goldsmith

Abstract: We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor which obtains a noisy version of the source and compresses this observation minimizing a local distortion measure which depends only on the marginal distribution of its observation. The… ▽ More We consider a multiterminal source coding problem in which a source is estimated at a central processing unit from lossy-compressed remote observations. Each lossy-encoded observation is produced by a remote sensor which obtains a noisy version of the source and compresses this observation minimizing a local distortion measure which depends only on the marginal distribution of its observation. The central node, on the other hand, has knowledge of the joint distribution of the source and all the observations and produces the source estimate which minimizes a different distortion measure between the source and its reconstruction. In this correspondence, we investigate the problem of optimally choosing the rate of each lossy-compressed remote estimate so as to minimize the distortion at the central processing unit subject to a bound on the overall communication rate between the remote sensors and the central unit. We focus, in particular, on two models of practical relevance: the case of a Gaussian source observed in additive Gaussian noise and reconstructed under quadratic distortion, and the case of a binary source observed in bit-flip** noise and reconstructed under Hamming distortion. In both scenarios we show that there exist regimes under which having more remote encoders does reduce the source distortion: in other words, having fewer, high-quality remote estimates provides a smaller distortion than having more, lower-quality estimates. △ Less

Submitted 12 May, 2016; originally announced May 2016.

The Rate-Distortion Risk in Estimation from Compressed Data

Authors: Alon Kipnis, Stefano Rini, Andrea J. Goldsmith

Abstract: Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compr… ▽ More Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the rate-distortion risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon's RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data. △ Less

Submitted 10 January, 2021; v1 submitted 5 February, 2016; originally announced February 2016.

Comments: Second revision. IEEE Transactions on Information Theory

Fundamental Distortion Limits of Analog-to-Digital Compression

Authors: Alon Kipnis, Yonina C. Eldar, Andrea J. Goldsmith

Abstract: Representing a continuous-time signal by a set of samples is a classical problem in signal processing. We study this problem under the additional constraint that the samples are quantized or compressed in a lossy manner under a limited bitrate budget. To this end, we consider a combined sampling and source coding problem in which an analog stationary Gaussian signal is reconstructed from its encod… ▽ More Representing a continuous-time signal by a set of samples is a classical problem in signal processing. We study this problem under the additional constraint that the samples are quantized or compressed in a lossy manner under a limited bitrate budget. To this end, we consider a combined sampling and source coding problem in which an analog stationary Gaussian signal is reconstructed from its encoded samples. These samples are obtained by a set of bounded linear functionals of the continuous-time path, with a limitation on the average number of samples obtained per unit time available in this setting. We provide a full characterization of the minimal distortion in terms of the sampling frequency, the bitrate, and the signal's spectrum. Assuming that the signal's energy is not uniformly distributed over its spectral support, we show that for each compression bitrate there exists a critical sampling frequency smaller than the Nyquist rate, such that the distortion in signal reconstruction when sampling at this frequency is minimal. Our results can be seen as an extension of the classical sampling theorem for bandlimited random processes in the sense that it describes the minimal amount of excess distortion in the reconstruction due to lossy compression of the samples, and provides the minimal sampling frequency required in order to achieve this distortion. Finally, we compare the fundamental limits in the combined source coding and sampling problem to the performance of pulse code modulation (PCM), where each sample is quantized by a scalar quantizer using a fixed number of bits. △ Less

Submitted 10 April, 2018; v1 submitted 24 January, 2016; originally announced January 2016.

Comments: 20 pages, 14 figures

Energy-based Modulation for Noncoherent Massive SIMO Systems

Authors: Alexandros Manolakos, Mainak Chowdhury, Andrea J. Goldsmith

Abstract: An uplink system with a single antenna transmitter and a single receiver with a large number of antennas is considered. We propose an energy-detection-based single-shot noncoherent communication scheme which does not use the instantaneous channel state information (CSI), but rather only the knowledge of the channel statistics. The suggested system uses a transmitter that modulates information on t… ▽ More An uplink system with a single antenna transmitter and a single receiver with a large number of antennas is considered. We propose an energy-detection-based single-shot noncoherent communication scheme which does not use the instantaneous channel state information (CSI), but rather only the knowledge of the channel statistics. The suggested system uses a transmitter that modulates information on the power of the symbols, and a receiver which measures only the average energy across the antennas. We propose constellation designs which are asymptotically optimal with respect to symbol error rate (SER) with an increasing number of antennas, for any finite signal to noise ratio (SNR) at the receiver, under different assumptions on the availability of CSI statistics (exact channel fading distribution or the first few moments of the channel fading distribution). We also consider the case of imperfect knowledge of the channel statistics and describe in detail the case when there is a bounded uncertainty on the moments of the fading distribution. We present numerical results on the SER performance achieved by these designs in typical scenarios and find that they may outperform existing noncoherent constellations, e.g., conventional Amplitude Shift Keying (ASK), and pilot-based schemes, e.g., Pulse Amplitude Modulation (PAM). We also observe that an optimized constellation for a specific channel distribution makes it very sensitive to uncertainties in the channel statistics. In particular, constellation designs based on optimistic channel conditions could lead to significant performance degradation in terms of the achieved symbol error rates. △ Less

Submitted 17 July, 2015; originally announced July 2015.

The Distortion Rate Function of Cyclostationary Gaussian Processes

Authors: Alon Kipnis, Andrea J. Goldsmith, Yonina C. Eldar

Abstract: A general expression for the distortion rate function (DRF) of cyclostationary Gaussian processes in terms of their spectral properties is derived. This expression can be seen as the result of orthogonalization over the different components in the polyphase decomposition of the process. We use this expression to derive, in a closed form, the DRF of several cyclostationary processes arising in prac… ▽ More A general expression for the distortion rate function (DRF) of cyclostationary Gaussian processes in terms of their spectral properties is derived. This expression can be seen as the result of orthogonalization over the different components in the polyphase decomposition of the process. We use this expression to derive, in a closed form, the DRF of several cyclostationary processes arising in practice. We first consider the DRF of a combined sampling and source coding problem. It is known that the optimal coding strategy for this problem involves source coding applied to a signal with the same structure as one resulting from pulse amplitude modulation (PAM). Since a PAM-modulated signal is cyclostationary, our DRF expression can be used to solve for the minimal distortion in the combined sampling and source coding problem. We also analyze in more detail the DRF of a source with the same structure as a PAM-modulated signal, and show that it is obtained by reverse waterfilling over an expression that depends on the energy of the pulse and the baseband process modulated to obtain the PAM signal. This result is then used to study the information content of a PAM-modulated signal as a function of its symbol time relative to the bandwidth of the underlying baseband process. In addition, we also study the DRF of sources with an amplitude-modulation structure, and show that the DRF of a narrow-band Gaussian stationary process modulated by either a deterministic or a random phase sine-wave equals the DRF of the baseband process. △ Less

Submitted 10 August, 2016; v1 submitted 20 May, 2015; originally announced May 2015.

Comments: First revision for the IEEE Transactions on Information Theory

Indirect Rate-Distortion Function of a Binary i.i.d Source

Authors: Alon Kipnis, Stefano Rini, Andrea J. Goldsmith

Abstract: The indirect source-coding problem in which a Bernoulli process is compressed in a lossy manner from its noisy observations is considered. These noisy observations are obtained by passing the source sequence through a The indirect source-coding problem in which a Bernoulli process is compressed in a lossy manner from its noisy observations is considered. These noisy observations are obtained by pa… ▽ More The indirect source-coding problem in which a Bernoulli process is compressed in a lossy manner from its noisy observations is considered. These noisy observations are obtained by passing the source sequence through a The indirect source-coding problem in which a Bernoulli process is compressed in a lossy manner from its noisy observations is considered. These noisy observations are obtained by passing the source sequence through a binary symmetric channel so that the channel crossover probability controls the amount of information available about the source realization at the encoder. We use classic results in rate-distortion theory to compute an expression of the rate-distortion function for this model, where the Bernoulli source is not necessarily symmetric. The indirect rate-distortion function is given in terms of a solution to a simple equation. In addition, we derive an upper bound on the indirect rate-distortion function which is given in a closed. These expressions capture precisely the expected behavior that the noisier the observations, the smaller the return from increasing bit-rate to reduce distortion. △ Less

Submitted 3 June, 2015; v1 submitted 19 May, 2015; originally announced May 2015.

Information Recovery from Pairwise Measurements

Authors: Yuxin Chen, Changho Suh, Andrea J. Goldsmith

Abstract: This paper is concerned with jointly recovering $n$ node-variables $\left\{ x_{i}\right\}_{1\leq i\leq n}$ from a collection of pairwise difference measurements. Imagine we acquire a few observations taking the form of $x_{i}-x_{j}$; the observation pattern is represented by a measurement graph $\mathcal{G}$ with an edge set $\mathcal{E}$ such that $x_{i}-x_{j}$ is observed if and only if… ▽ More This paper is concerned with jointly recovering $n$ node-variables $\left\{ x_{i}\right\}_{1\leq i\leq n}$ from a collection of pairwise difference measurements. Imagine we acquire a few observations taking the form of $x_{i}-x_{j}$; the observation pattern is represented by a measurement graph $\mathcal{G}$ with an edge set $\mathcal{E}$ such that $x_{i}-x_{j}$ is observed if and only if $(i,j)\in\mathcal{E}$. To account for noisy measurements in a general manner, we model the data acquisition process by a set of channels with given input/output transition measures. Employing information-theoretic tools applied to channel decoding problems, we develop a \emph{unified} framework to characterize the fundamental recovery criterion, which accommodates general graph structures, alphabet sizes, and channel transition measures. In particular, our results isolate a family of \emph{minimum} \emph{channel divergence measures} to characterize the degree of measurement corruption, which together with the size of the minimum cut of $\mathcal{G}$ dictates the feasibility of exact information recovery. For various homogeneous graphs, the recovery condition depends almost only on the edge sparsity of the measurement graph irrespective of other graphical metrics; alternatively, the minimum sample complexity required for these graphs scales like \[ \text{minimum sample complexity }\asymp\frac{n\log n}{\mathsf{Hel}_{1/2}^{\min}} \] for certain information metric $\mathsf{Hel}_{1/2}^{\min}$ defined in the main text, as long as the alphabet size is not super-polynomial in $n$. We apply our general theory to three concrete applications, including the stochastic block model, the outlier model, and the haplotype assembly problem. Our theory leads to order-wise tight recovery conditions for all these scenarios. △ Less

Submitted 5 May, 2016; v1 submitted 6 April, 2015; originally announced April 2015.

Comments: This work has been presented in part in ISIT 2014 (http://arxiv.longhoe.net/abs/1404.7105) and ISIT 2015