Information Theory
See recent articles
- [1] arXiv:2407.02816 [pdf, other]
-
Title: Large and Small Deviations for Statistical Sequence MatchingComments: Extended version of ISIT paperSubjects: Information Theory (cs.IT); Signal Processing (eess.SP); Statistics Theory (math.ST)
We revisit the problem of statistical sequence matching between two databases of sequences initiated by Unnikrishnan (TIT 2015) and derive theoretical performance guarantees for the generalized likelihood ratio test (GLRT). We first consider the case where the number of matched pairs of sequences between the databases is known. In this case, the task is to accurately find the matched pairs of sequences among all possible matches between the sequences in the two databases. We analyze the performance of the GLRT by Unnikrishnan and explicitly characterize the tradeoff between the mismatch and false reject probabilities under each hypothesis in both large and small deviations regimes. Furthermore, we demonstrate the optimality of Unnikrishnan's GLRT test under the generalized Neyman-Person criterion for both regimes and illustrate our theoretical results via numerical examples. Subsequently, we generalize our achievability analyses to the case where the number of matched pairs is unknown, and an additional error probability needs to be considered. When one of the two databases contains a single sequence, the problem of statistical sequence matching specializes to the problem of multiple classification introduced by Gutman (TIT 1989). For this special case, our result for the small deviations regime strengthens previous result of Zhou, Tan and Motani (Information and Inference 2020) by removing unnecessary conditions on the generating distributions.
- [2] arXiv:2407.02877 [pdf, html, other]
-
Title: Resource Allocation Design for Next-Generation Multiple Access: A Tutorial OverviewComments: 69 pages, 10 figures, 5 tablesSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
Multiple access is the cornerstone technology for each generation of wireless cellular networks and resource allocation design plays a crucial role in multiple access. In this paper, we present a comprehensive tutorial overview for junior researchers in this field, aiming to offer a foundational guide for resource allocation design in the context of next-generation multiple access (NGMA). Initially, we identify three types of channels in future wireless cellular networks over which NGMA will be implemented, namely: natural channels, reconfigurable channels, and functional channels. Natural channels are traditional uplink and downlink communication channels; reconfigurable channels are defined as channels that can be proactively reshaped via emerging platforms or techniques, such as intelligent reflecting surface (IRS), unmanned aerial vehicle (UAV), and movable/fluid antenna (M/FA); and functional channels support not only communication but also other functionalities simultaneously, with typical examples including integrated sensing and communication (ISAC) and joint computing and communication (JCAC) channels. Then, we introduce NGMA models applicable to these three types of channels that cover most of the practical communication scenarios of future wireless communications. Subsequently, we articulate the key optimization technical challenges inherent in the resource allocation design for NGMA, categorizing them into rate-oriented, power-oriented, and reliability-oriented resource allocation designs. The corresponding optimization approaches for solving the formulated resource allocation design problems are then presented. Finally, simulation results are presented and discussed to elucidate the practical implications and insights derived from resource allocation designs in NGMA.
- [3] arXiv:2407.02919 [pdf, html, other]
-
Title: Efficient IoT Devices Localization Through Wi-Fi CSI Feature Fusion and Anomaly DetectionComments: Accepted in IEEE Internet of Things Journal, Early Access, 2024Journal-ref: IEEE Internet of Things Journal, Early Access, 2024Subjects: Information Theory (cs.IT)
Internet of Things (IoT) device localization is fundamental to smart home functionalities, including indoor navigation and tracking of individuals. Traditional localization relies on relative methods utilizing the positions of anchors within a home environment, yet struggles with precision due to inherent inaccuracies in these anchor positions. In response, we introduce a cutting-edge smartphone-based localization system for IoT devices, leveraging the precise positioning capabilities of smartphones equipped with motion sensors. Our system employs artificial intelligence (AI) to merge channel state information from proximal trajectory points of a single smartphone, significantly enhancing line of sight (LoS) angle of arrival (AoA) estimation accuracy, particularly under severe multipath conditions. Additionally, we have developed an AI-based anomaly detection algorithm to further increase the reliability of LoSAoA estimation. This algorithm improves measurement reliability by analyzing the correlation between the accuracy of reversed feature reconstruction and the LoS-AoA estimation. Utilizing a straightforward least squares algorithm in conjunction with accurate LoS-AoA estimation and smartphone positional data, our system efficiently identifies IoT device locations. Validated through extensive simulations and experimental tests with a receiving antenna array comprising just two patch antenna elements in the horizontal direction, our methodology has been shown to attain decimeter-level localization accuracy in nearly 90% of cases, demonstrating robust performance even in challenging real-world scenarios. Additionally, our proposed anomaly detection algorithm trained on Wi-Fi data can be directly applied to ultra-wideband, also outperforming the most advanced techniques.
- [4] arXiv:2407.02922 [pdf, html, other]
-
Title: Fair Resource Allocation for Probabilistic Semantic Communication in IIoTSiyun Liang, Zhouxiang Zhao, Chen Zhu, Zhaohui Yang, Yinchao Yang, Mohammad Shikh-Bahaei, Zhaoyang ZhangSubjects: Information Theory (cs.IT)
In this paper, the problem of minimum rate maximization for probabilistic semantic communication (PSCom) in industrial Internet of Things (IIoT) is investigated. In the considered model, users employ semantic information extraction techniques to compress the original data before sending it to the base station (BS). During this semantic compression process, knowledge graphs are employed to represent the semantic information, and the probability graph sharing between users and the BS is utilized to further compress the knowledge graph. The semantic compression process can significantly reduce the transmitted data size, but it inevitably introduces additional computation overhead. Considering the limited power budget of the user, we formulate a joint communication and computation optimization problem is formulated aiming to maximize the minimum equivalent rate among all users while meeting total power and semantic compression ratio constraints. To address this problem, two algorithms with different computational complexities are proposed to obtain suboptimal solutions. One algorithm is based on a prorate distribution of transmission power, while the other traverses the combinations of semantic compression ratios among all users. In both algorithms, bisection is employed in order to achieve the greatest minimum equivalent rate. The simulation results validate the effectiveness of the proposed algorithms.
- [5] arXiv:2407.02949 [pdf, html, other]
-
Title: Competitive Analysis of Arbitrary Varying ChannelsSubjects: Information Theory (cs.IT)
Arbitrary varying channels (AVC) are used to model communication settings in which a channel state may vary arbitrarily over time. Their primary objective is to circumvent statistical assumptions on channel variation. Traditional studies on AVCs optimize rate subject to the worst-case state sequence. While this approach is resilient to channel variations, it may result in low rates for state sequences that are associated with relatively good channels. This paper addresses the analysis of AVCs through the lens of competitive analysis, where solution quality is measured with respect to the optimal solution had the state sequence been known in advance. Our main result demonstrates that codes constructed by a single input distribution do not achieve optimal competitive performance over AVCs. This stands in contrast to the single-letter capacity formulae for AVCs, and it indicates, in our setting, that even though the encoder cannot predict the subsequent channel states, it benefits from varying its input distribution as time proceeds.
- [6] arXiv:2407.02953 [pdf, html, other]
-
Title: Affine Frequency Division Multiplexing for Compressed Sensing of Time-Varying ChannelsComments: Accepted in SPAWC 2024Subjects: Information Theory (cs.IT)
This paper addresses compressed sensing of linear time-varying (LTV) wireless propagation links under the assumption of double sparsity i.e., sparsity in both the delay and Doppler domains, using Affine Frequency Division Multiplexing (AFDM) measurements. By rigorously linking the double sparsity model to the hierarchical sparsity paradigm, a compressed sensing algorithm with recovery guarantees is proposed for extracting delay-Doppler profiles of LTV channels using AFDM. Through mathematical analysis and numerical results, the superiority of AFDM over other waveforms in terms of channel estimation overhead and minimal sampling rate requirements in sub-Nyquist radar applications is demonstrated.
- [7] arXiv:2407.03054 [pdf, html, other]
-
Title: Closed-Loop Binary Media-Based ModulationComments: 5 pages, 4 figures, 1 tableJournal-ref: EuCNC and 6G Summit 2024Subjects: Information Theory (cs.IT)
Presenting analytical results for Binary Media-Based Modulation (B-MBM) over fading channels for single-antenna receivers. Illustrating that open-loop B-MBM, in the absence of feedback, only achieves a diversity order of one. However, with feedback and optimal weight selection in closed-loop configurations, a diversity order of two becomes achievable. Notably, the closed-loop B-MBM, with analytically computed optimal weights, performs equivalent to Alamouti-coded BPSK transmission, demonstrating feasibility even with just one radio frequency chain when feedback is available.
- [8] arXiv:2407.03175 [pdf, html, other]
-
Title: Low-Rank Toeplitz Matrix Restoration: Descent Cone Analysis and Structured Random MatrixComments: 14pagesSubjects: Information Theory (cs.IT)
This note demonstrates that we can stably recover rank $r$ Toeplitz matrix $\pmb{X}\in\mathbb{R}^{n\times n}$ from a number of rank one subgaussian measurements on the order of $r\log^{2} n$ with an exponentially decreasing failure probability by employing a nuclear norm minimization program. Our approach utilizes descent cone analysis through Mendelson's small ball method with the Toeplitz constraint. The key ingredient is to determine the spectral norm of the random matrix of the Topelitz structure, which may be of independent interest.This improves upon earlier analyses and resolves the conjecture in Chen et al. (IEEE Transactions on Information Theory, 2015).
- [9] arXiv:2407.03280 [pdf, html, other]
-
Title: Cooperative Multi-Agent Deep Reinforcement Learning Methods for UAV-aided Mobile Edge Computing NetworksComments: 13 pages, 6 figuresSubjects: Information Theory (cs.IT)
This paper presents a cooperative multi-agent deep reinforcement learning (MADRL) approach for unmmaned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks. An UAV with computing capability can provide task offlaoding services to ground internet-of-things devices (IDs). With partial observation of the entire network state, the UAV and the IDs individually determine their MEC strategies, i.e., UAV trajectory, resource allocation, and task offloading policy. This requires joint optimization of decision-making process and coordination strategies among the UAV and the IDs. To address this difficulty, the proposed cooperative MADRL approach computes two types of action variables, namely message action and solution action, each of which is generated by dedicated actor neural networks (NNs). As a result, each agent can automatically encapsulate its coordination messages to enhance the MEC performance in the decentralized manner. The proposed actor structure is designed based on graph attention networks such that operations are possible regardless of the number of IDs. A scalable training algorithm is also proposed to train a group of NNs for arbitrary network configurations. Numerical results demonstrate the superiority of the proposed cooperative MADRL approach over conventional methods.
- [10] arXiv:2407.03289 [pdf, html, other]
-
Title: Correlated Privacy Mechanisms for Differentially Private Distributed Mean EstimationSubjects: Information Theory (cs.IT); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Differentially private distributed mean estimation (DP-DME) is a fundamental building block in privacy-preserving federated learning, where a central server estimates the mean of $d$-dimensional vectors held by $n$ users while ensuring $(\epsilon,\delta)$-DP. Local differential privacy (LDP) and distributed DP with secure aggregation (SecAgg) are the most common notions of DP used in DP-DME settings with an untrusted server. LDP provides strong resilience to dropouts, colluding users, and malicious server attacks, but suffers from poor utility. In contrast, SecAgg-based DP-DME achieves an $O(n)$ utility gain over LDP in DME, but requires increased communication and computation overheads and complex multi-round protocols to handle dropouts and malicious attacks. In this work, we propose CorDP-DME, a novel DP-DME mechanism that spans the gap between DME with LDP and distributed DP, offering a favorable balance between utility and resilience to dropout and collusion. CorDP-DME is based on correlated Gaussian noise, ensuring DP without the perfect conditional privacy guarantees of SecAgg-based approaches. We provide an information-theoretic analysis of CorDP-DME, and derive theoretical guarantees for utility under any given privacy parameters and dropout/colluding user thresholds. Our results demonstrate that (anti) correlated Gaussian DP mechanisms can significantly improve utility in mean estimation tasks compared to LDP -- even in adversarial settings -- while maintaining better resilience to dropouts and attacks compared to distributed DP.
New submissions for Thursday, 4 July 2024 (showing 10 of 10 entries )
- [11] arXiv:2407.02493 (cross-list from physics.bio-ph) [pdf, other]
-
Title: The dimensionality of a coiled helical coilComments: 10 pages, 8 figuresSubjects: Biological Physics (physics.bio-ph); Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT)
The helical coil is ubiquitous in biological and natural systems and often it is the basic form that leads to complex structures. This paper considers the question of its dimensionality in biological information as the helical coil goes through recursive coiling as happens to DNA and RNA molecules in chromatin. It has been shown that the dimensionality of coiled coils is virtually equal to e. Of the three forms of DNA, the dimensionality of the B-form is nearest to the optimal value and this might be the reason why it is most common.
- [12] arXiv:2407.02811 (cross-list from cs.LG) [pdf, html, other]
-
Title: SPLITZ: Certifiable Robustness via Split Lipschitz Randomized SmoothingSubjects: Machine Learning (cs.LG); Information Theory (cs.IT)
Certifiable robustness gives the guarantee that small perturbations around an input to a classifier will not change the prediction. There are two approaches to provide certifiable robustness to adversarial examples: a) explicitly training classifiers with small Lipschitz constants, and b) Randomized smoothing, which adds random noise to the input to create a smooth classifier. We propose \textit{SPLITZ}, a practical and novel approach which leverages the synergistic benefits of both the above ideas into a single framework. Our main idea is to \textit{split} a classifier into two halves, constrain the Lipschitz constant of the first half, and smooth the second half via randomization. Motivation for \textit{SPLITZ} comes from the observation that many standard deep networks exhibit heterogeneity in Lipschitz constants across layers. \textit{SPLITZ} can exploit this heterogeneity while inheriting the scalability of randomized smoothing. We present a principled approach to train \textit{SPLITZ} and provide theoretical analysis to derive certified robustness guarantees during inference. We present a comprehensive comparison of robustness-accuracy tradeoffs and show that \textit{SPLITZ} consistently improves upon existing state-of-the-art approaches on MNIST and CIFAR-10 datasets. For instance, with $\ell_2$ norm perturbation budget of \textbf{$\epsilon=1$}, \textit{SPLITZ} achieves $\textbf{43.2\%}$ top-1 test accuracy on CIFAR-10 dataset compared to state-of-art top-1 test accuracy $\textbf{39.8\%}
- [13] arXiv:2407.02963 (cross-list from eess.SP) [pdf, other]
-
Title: Subspace Coding for Spatial SensingComments: ©2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksSubjects: Signal Processing (eess.SP); Information Theory (cs.IT)
A subspace code is defined as a collection of subspaces of an ambient vector space, where each information-encoding codeword is a subspace. This paper studies a class of spatial sensing problems, notably direction of arrival (DoA) estimation using multisensor arrays, from a novel subspace coding perspective. Specifically, we demonstrate how a canonical (passive) sensing model can be mapped into a subspace coding problem, with the sensing operation defining a unique structure for the subspace codewords. We introduce the concept of sensing subspace codes following this structure, and show how these codes can be controlled by judiciously designing the sensor array geometry. We further present a construction of sensing subspace codes leveraging a certain class of Golomb rulers that achieve near-optimal minimum codeword distance. These designs inspire novel noise-robust sparse array geometries achieving high angular resolution. We also prove that codes corresponding to conventional uniform linear arrays are suboptimal in this regard. This work is the first to establish connections between subspace coding and spatial sensing, with the aim of leveraging insights and methodologies in one field to tackle challenging problems in the other.
- [14] arXiv:2407.02977 (cross-list from cs.CL) [pdf, html, other]
-
Title: Large Language Models as Evaluators for Scientific SynthesisComments: 4 pages, forthcoming as part of the KONVENS 2024 proceedings this https URLSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Theory (cs.IT)
Our study explores how well the state-of-the-art Large Language Models (LLMs), like GPT-4 and Mistral, can assess the quality of scientific summaries or, more fittingly, scientific syntheses, comparing their evaluations to those of human annotators. We used a dataset of 100 research questions and their syntheses made by GPT-4 from abstracts of five related papers, checked against human quality ratings. The study evaluates both the closed-source GPT-4 and the open-source Mistral model's ability to rate these summaries and provide reasons for their judgments. Preliminary results show that LLMs can offer logical explanations that somewhat match the quality ratings, yet a deeper statistical analysis shows a weak correlation between LLM and human ratings, suggesting the potential and current limitations of LLMs in scientific synthesis evaluation.
Cross submissions for Thursday, 4 July 2024 (showing 4 of 4 entries )
- [15] arXiv:2401.11141 (replaced) [pdf, html, other]
-
Title: Wideband Beamforming for RIS Assisted Near-Field CommunicationsSubjects: Information Theory (cs.IT); Signal Processing (eess.SP)
A near-field wideband beamforming scheme is investigated for reconfigurable intelligent surface (RIS) assisted multiple-input multiple-output (MIMO) systems, in which a deep learning-based end-to-end (E2E) optimization framework is proposed to maximize the system spectral efficiency. To deal with the near-field double beam split effect, the base station is equipped with frequency-dependent hybrid precoding architecture by introducing sub-connected true time delay (TTD) units, while two specific RIS architectures, namely true time delay-based RIS (TTD-RIS) and virtual subarray-based RIS (SA-RIS), are exploited to realize the frequency-dependent passive beamforming at the RIS. Furthermore, the efficient E2E beamforming models without explicit channel state information are proposed, which jointly exploits the uplink channel training module and the downlink wideband beamforming module. In the proposed network architecture of the E2E models, the classical communication signal processing methods, i.e., polarized filtering and sparsity transform, are leveraged to develop a signal-guided beamforming network. Numerical results show that the proposed E2E models have superior beamforming performance and robustness to conventional beamforming benchmarks. Furthermore, the tradeoff between the beamforming gain and the hardware complexity is investigated for different frequency-dependent RIS architectures, in which the TTD-RIS can achieve better spectral efficiency than the SA-RIS while requiring additional energy consumption and hardware cost.
- [16] arXiv:2404.08638 (replaced) [pdf, html, other]
-
Title: Age of Information Optimization and State Error Analysis for Correlated Multi-Process Multi-Sensor SystemsComments: fix typosSubjects: Information Theory (cs.IT)
In this paper, we examine a multi-sensor system where each sensor may monitor more than one time-varying information process and send status updates to a remote monitor over a common channel. We consider that each sensor's status update may contain information about more than one information process in the system subject to the system's constraints. To investigate the impact of this correlation on the overall system's performance, we conduct an analysis of both the average Age of Information (AoI) and source state estimation error at the monitor. Building upon this analysis, we subsequently explore the impact of the packet arrivals, correlation probabilities, and rate of processes' state change on the system's performance. Next, we consider the case where sensors have limited sensing abilities and distribute a portion of their sensing abilities across the different processes. We optimize this distribution to minimize the total AoI of the system. Interestingly, we show that monitoring multiple processes from a single source may not always be beneficial. Our results also reveal that the optimal sensing distribution for diverse arrival rates may exhibit a rapid regime switch, rather than smooth transitions, after crossing critical system values. This highlights the importance of identifying these critical thresholds to ensure effective system performance.
- [17] arXiv:2405.09497 (replaced) [pdf, html, other]
-
Title: Towards the limits: Sensing Capability Measurement for ISAC Through Channel EncoderSubjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)
Integrated Sensing and Communication (ISAC) is gradually becoming a reality due to the significant increase in frequency and bandwidth of next-generation wireless communication technologies. Therefore it becomes crucial to evaluate the communication and sensing performance using appropriate channel models to address resource competition from each other. Existing work only models the sensing capability based on the mutual information between the channel response and the received signal, and its theoretical resolution is difficult to support the high-precision requirements of ISAC for sensing tasks, and may even affect its communication optimal.
In this paper, we propose a sensing channel encoder model to measure the sensing capacity with higher resolution by discrete task mutual information. For the first time, derive upper and lower bounds on the sensing accuracy for a given channel. This model not only provides the possibility of optimizing the ISAC systems at a finer granularity and balancing communication and sensing resources, but also provides theoretical explanations for classical intuitive feelings (like more modalities more accuracy) in wireless sensing. Furthermore, we validate the effectiveness of the proposed channel model through real-case studies, including person identification, displacement detection, direction estimation, and device recognition. The evaluation results indicate a Pearson correlation coefficient exceeding 0.9 between our task mutual information and conventional experimental metrics (e.g., accuracy). - [18] arXiv:2308.14507 (replaced) [pdf, other]
-
Title: Spectral Estimators for Structured Generalized Linear Models via Approximate Message PassingSubjects: Statistics Theory (math.ST); Information Theory (cs.IT); Machine Learning (cs.LG); Probability (math.PR); Machine Learning (stat.ML)
We consider the problem of parameter estimation in a high-dimensional generalized linear model. Spectral methods obtained via the principal eigenvector of a suitable data-dependent matrix provide a simple yet surprisingly effective solution. However, despite their wide use, a rigorous performance characterization, as well as a principled way to preprocess the data, are available only for unstructured (i.i.d.\ Gaussian and Haar orthogonal) designs. In contrast, real-world data matrices are highly structured and exhibit non-trivial correlations. To address the problem, we consider correlated Gaussian designs capturing the anisotropic nature of the features via a covariance matrix $\Sigma$. Our main result is a precise asymptotic characterization of the performance of spectral estimators. This allows us to identify the optimal preprocessing that minimizes the number of samples needed for parameter estimation. Surprisingly, such preprocessing is universal across a broad set of designs, which partly addresses a conjecture on optimal spectral estimators for rotationally invariant models. Our principled approach vastly improves upon previous heuristic methods, including for designs common in computational imaging and genetics. The proposed methodology, based on approximate message passing, is broadly applicable and opens the way to the precise characterization of spiked matrices and of the corresponding spectral methods in a variety of settings.
- [19] arXiv:2311.18003 (replaced) [pdf, html, other]
-
Title: Subsystem CSS codes, a tighter stabilizer-to-CSS map**, and Goursat's LemmaComments: 27 pages, 2 figuresSubjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
The CSS code construction is a powerful framework used to express features of a quantum code in terms of a pair of underlying classical codes. Its subsystem extension allows for similar expressions, but the general case has not been fully explored. Extending previous work of Aly, Klappenecker, and Sarvepalli [quantph/0610153], we determine subsystem CSS code parameters, express codewords, and develop a Steane-type decoder using only data from the two underlying classical codes. Generalizing a result of Kovalev and Pryadko [Phys. Rev. A 88 012311 (2013)], we show that any subsystem stabilizer code can be "doubled" to yield a subsystem CSS code with twice the number of physical, logical, and gauge qudits and up to twice the code distance. This map** preserves locality and is tighter than the Majorana-based map** of Bravyi, Terhal, and Leemhuis [New J. Phys. 12 083039 (2010)]. Using Goursat's Lemma, we show that every subsystem stabilizer code can be constructed from two nested subsystem CSS codes satisfying certain constraints, and we characterize subsystem stabilizer codes based on the nested codes' properties.