Showing 1–30 of 30 results for author: Blum, R

Hidden Variables unseen by Random Forests

Authors: Ricardo Blum, Munir Hiabu, Enno Mammen, Joseph Theo Meyer

Abstract: Random Forests are widely claimed to capture interactions well. However, some simple examples suggest that they perform poorly in the presence of certain pure interactions that the conventional CART criterion struggles to capture during tree construction. We argue that simple alternative partitioning schemes used in the tree growing procedure can enhance identification of these interactions. In a… ▽ More Random Forests are widely claimed to capture interactions well. However, some simple examples suggest that they perform poorly in the presence of certain pure interactions that the conventional CART criterion struggles to capture during tree construction. We argue that simple alternative partitioning schemes used in the tree growing procedure can enhance identification of these interactions. In a simulation study we compare these variants to conventional Random Forests and Extremely Randomized trees. Our results validate that the modifications considered enhance the model's fitting ability in scenarios where pure interactions play a crucial role. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2309.01460

Incorporating Domain Differential Equations into Graph Convolutional Networks to Lower Generalization Discrepancy

Authors: Yue Sun, Chao Chen, Yuesheng Xu, Sihong Xie, Rick S. Blum, Parv Venkitasubramaniam

Abstract: Ensuring both accuracy and robustness in time series prediction is critical to many applications, ranging from urban planning to pandemic management. With sufficient training data where all spatiotemporal patterns are well-represented, existing deep-learning models can make reasonably accurate predictions. However, existing methods fail when the training data are drawn from different circumstances… ▽ More Ensuring both accuracy and robustness in time series prediction is critical to many applications, ranging from urban planning to pandemic management. With sufficient training data where all spatiotemporal patterns are well-represented, existing deep-learning models can make reasonably accurate predictions. However, existing methods fail when the training data are drawn from different circumstances (e.g., traffic patterns on regular days) compared to test data (e.g., traffic patterns after a natural disaster). Such challenges are usually classified under domain generalization. In this work, we show that one way to address this challenge in the context of spatiotemporal prediction is by incorporating domain differential equations into Graph Convolutional Networks (GCNs). We theoretically derive conditions where GCNs incorporating such domain differential equations are robust to mismatched training and testing data compared to baseline domain agnostic models. To support our theory, we propose two domain-differential-equation-informed networks called Reaction-Diffusion Graph Convolutional Network (RDGCN), which incorporates differential equations for traffic speed evolution, and Susceptible-Infectious-Recovered Graph Convolutional Network (SIRGCN), which incorporates a disease propagation model. Both RDGCN and SIRGCN are based on reliable and interpretable domain differential equations that allow the models to generalize to unseen patterns. We experimentally show that RDGCN and SIRGCN are more robust with mismatched testing data than the state-of-the-art deep learning methods. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descent

Authors: Yicheng Chen, Rick S. Blum, Brian M. Sadler

Abstract: Distributed machine learning enables scalability and computational offloading, but requires significant levels of communication. Consequently, communication efficiency in distributed learning settings is an important consideration, especially when the communications are wireless and battery-driven devices are employed. In this paper we develop a censoring-based heavy ball (CHB) method for distribu… ▽ More Distributed machine learning enables scalability and computational offloading, but requires significant levels of communication. Consequently, communication efficiency in distributed learning settings is an important consideration, especially when the communications are wireless and battery-driven devices are employed. In this paper we develop a censoring-based heavy ball (CHB) method for distributed learning in a server-worker architecture. Each worker self-censors unless its local gradient is sufficiently different from the previously transmitted one. The significant practical advantages of the HB method for learning problems are well known, but the question of reducing communications has not been addressed. CHB takes advantage of the HB smoothing to eliminate reporting small changes, and provably achieves a linear convergence rate equivalent to that of the classical HB method for smooth and strongly convex objective functions. The convergence guarantee of CHB is theoretically justified for both convex and nonconvex cases. In addition we prove that, under some conditions, at least half of all communications can be eliminated without any impact on convergence rate. Extensive numerical results validate the communication efficiency of CHB on both synthetic and real datasets, for convex, nonconvex, and nondifferentiable cases. Given a target accuracy, CHB can significantly reduce the number of communications compared to existing algorithms, achieving the same accuracy without slowing down the optimization process. △ Less

Submitted 24 September, 2022; originally announced September 2022.

Route Discovery and Capacity of Ad hoc Networks

Authors: Eugene Perevalov, Rick S. Blum, Xun Chen, Anthony Nigara

Abstract: Throughput capacity of large ad hoc networks has been shown to scale adversely with the size of network $n$. However the need for the nodes to find or repair routes has not been analyzed in this context. In this paper, we explicitly take route discovery into account and obtain the scaling law for the throughput capacity under general assumptions on the network environment, node behavior, and the q… ▽ More Throughput capacity of large ad hoc networks has been shown to scale adversely with the size of network $n$. However the need for the nodes to find or repair routes has not been analyzed in this context. In this paper, we explicitly take route discovery into account and obtain the scaling law for the throughput capacity under general assumptions on the network environment, node behavior, and the quality of route discovery algorithms. We also discuss a number of possible scenarios and show that the need for route discovery may change the scaling for the throughput capacity dramatically. △ Less

Submitted 18 February, 2022; originally announced February 2022.

Comments: extended version, originally published in conference (IEEE GLOBECOM '05)

Communication Efficient Federated Learning via Ordered ADMM in a Fully Decentralized Setting

Authors: Yicheng Chen, Rick S. Blum, Brian M. Sadler

Abstract: The challenge of communication-efficient distributed optimization has attracted attention in recent years. In this paper, a communication efficient algorithm, called ordering-based alternating direction method of multipliers (OADMM) is devised in a general fully decentralized network setting where a worker can only exchange messages with neighbors. Compared to the classical ADMM, a key feature of… ▽ More The challenge of communication-efficient distributed optimization has attracted attention in recent years. In this paper, a communication efficient algorithm, called ordering-based alternating direction method of multipliers (OADMM) is devised in a general fully decentralized network setting where a worker can only exchange messages with neighbors. Compared to the classical ADMM, a key feature of OADMM is that transmissions are ordered among workers at each iteration such that a worker with the most informative data broadcasts its local variable to neighbors first, and neighbors who have not transmitted yet can update their local variables based on that received transmission. In OADMM, we prohibit workers from transmitting if their current local variables are not sufficiently different from their previously transmitted value. A variant of OADMM, called SOADMM, is proposed where transmissions are ordered but transmissions are never stopped for each node at each iteration. Numerical results demonstrate that given a targeted accuracy, OADMM can significantly reduce the number of communications compared to existing algorithms including ADMM. We also show numerically that SOADMM can accelerate convergence, resulting in communication savings compared to the classical ADMM. △ Less

Submitted 5 February, 2022; originally announced February 2022.

Distributed Learning With Sparsified Gradient Differences

Authors: Yicheng Chen, Rick S. Blum, Martin Takac, Brian M. Sadler

Abstract: A very large number of communications are typically required to solve distributed learning tasks, and this critically limits scalability and convergence speed in wireless communications applications. In this paper, we devise a Gradient Descent method with Sparsification and Error Correction (GD-SEC) to improve the communications efficiency in a general worker-server architecture. Motivated by a va… ▽ More A very large number of communications are typically required to solve distributed learning tasks, and this critically limits scalability and convergence speed in wireless communications applications. In this paper, we devise a Gradient Descent method with Sparsification and Error Correction (GD-SEC) to improve the communications efficiency in a general worker-server architecture. Motivated by a variety of wireless communications learning scenarios, GD-SEC reduces the number of bits per communication from worker to server with no degradation in the order of the convergence rate. This enables larger-scale model learning without sacrificing convergence or accuracy. At each iteration of GD-SEC, instead of directly transmitting the entire gradient vector, each worker computes the difference between its current gradient and a linear combination of its previously transmitted gradients, and then transmits the sparsified gradient difference to the server. A key feature of GD-SEC is that any given component of the gradient difference vector will not be transmitted if its magnitude is not sufficiently large. An error correction technique is used at each worker to compensate for the error resulting from sparsification. We prove that GD-SEC is guaranteed to converge for strongly convex, convex, and nonconvex optimization problems with the same order of convergence rate as GD. Furthermore, if the objective function is strongly convex, GD-SEC has a fast linear convergence rate. Numerical results not only validate the convergence rate of GD-SEC but also explore the communication bit savings it provides. Given a target accuracy, GD-SEC can significantly reduce the communications load compared to the best existing algorithms without slowing down the optimization process. △ Less

Submitted 4 February, 2022; originally announced February 2022.

Deep-learning in the bioimaging wild: Handling ambiguous data with deepflash2

Authors: Matthias Griebel, Dennis Segebarth, Nikolai Stein, Nina Schukraft, Philip Tovote, Robert Blum, Christoph M. Flath

Abstract: We present deepflash2, a deep learning solution that facilitates the objective and reliable segmentation of ambiguous bioimages through multi-expert annotations and integrated quality assurance. Thereby, deepflash2 addresses typical challenges that arise during training, evaluation, and application of deep learning models in bioimaging. The tool is embedded in an easy-to-use graphical user interfa… ▽ More We present deepflash2, a deep learning solution that facilitates the objective and reliable segmentation of ambiguous bioimages through multi-expert annotations and integrated quality assurance. Thereby, deepflash2 addresses typical challenges that arise during training, evaluation, and application of deep learning models in bioimaging. The tool is embedded in an easy-to-use graphical user interface and offers best-in-class predictive performance for semantic and instance segmentation under economical usage of computational resources. △ Less

Submitted 12 November, 2021; originally announced November 2021.

Training Robust Graph Neural Networks with Topology Adaptive Edge Drop**

Authors: Zhan Gao, Subhrajit Bhattacharya, Leiming Zhang, Rick S. Blum, Alejandro Ribeiro, Brian M. Sadler

Abstract: Graph neural networks (GNNs) are processing architectures that exploit graph structural information to model representations from network data. Despite their success, GNNs suffer from sub-optimal generalization performance given limited training data, referred to as over-fitting. This paper proposes Topology Adaptive Edge Drop** (TADropEdge) method as an adaptive data augmentation technique to i… ▽ More Graph neural networks (GNNs) are processing architectures that exploit graph structural information to model representations from network data. Despite their success, GNNs suffer from sub-optimal generalization performance given limited training data, referred to as over-fitting. This paper proposes Topology Adaptive Edge Drop** (TADropEdge) method as an adaptive data augmentation technique to improve generalization performance and learn robust GNN models. We start by explicitly analyzing how random edge drop** increases the data diversity during training, while indicating i.i.d. edge drop** does not account for graph structural information and could result in noisy augmented data degrading performance. To overcome this issue, we consider graph connectivity as the key property that captures graph topology. TADropEdge incorporates this factor into random edge drop** such that the edge-dropped subgraphs maintain similar topology as the underlying graph, yielding more satisfactory data augmentation. In particular, TADropEdge first leverages the graph spectrum to assign proper weights to graph edges, which represent their criticality for establishing the graph connectivity. It then normalizes the edge weights and drops graph edges adaptively based on their normalized weights. Besides improving generalization performance, TADropEdge reduces variance for efficient training and can be applied as a generic method modular to different GNN models. Intensive experiments on real-life and synthetic datasets corroborate theory and verify the effectiveness of the proposed method. △ Less

Submitted 5 June, 2021; originally announced June 2021.

Inter-cluster Transmission Control Using Graph Modal Barriers

Authors: Leiming Zhang, Brian M. Sadler, Rick S. Blum, Subhrajit Bhattacharya

Abstract: In this paper we consider the problem of transmission across a graph and how to effectively control/restrict it with limited resources. Transmission can represent information transfer across a social network, spread of a malicious virus across a computer network, or spread of an infectious disease across communities. The key insight is to assign proper weights to bottleneck edges of the graph base… ▽ More In this paper we consider the problem of transmission across a graph and how to effectively control/restrict it with limited resources. Transmission can represent information transfer across a social network, spread of a malicious virus across a computer network, or spread of an infectious disease across communities. The key insight is to assign proper weights to bottleneck edges of the graph based on their role in reducing the connection between two or more strongly-connected clusters within the graph. Selectively reducing the weights (implying reduced transmission rate) on the critical edges helps limit the transmission from one cluster to another. We refer to these as barrier weights and their computation is based on the eigenvectors of the graph Laplacian. Unlike other work on graph partitioning and clustering, we completely circumvent the associated computational complexities by assigning weights to edges instead of performing discrete graph cuts. This allows us to provide strong theoretical results on our proposed methods. We also develop approximations that allow low complexity distributed computation of the barrier weights using only neighborhood communication on the graph. △ Less

Submitted 9 October, 2020; originally announced October 2020.

Comments: 16 pages

Secret key authentication capacity region, Part I: average authentication rate

Authors: Jake Perazzone, Eric Graves, Paul Yu, Rick Blum

Abstract: This paper investigates the secret key authentication capacity region. Specifically, the focus is on a model where a source must transmit information over an adversary controlled channel where the adversary, prior to the source's transmission, decides whether or not to replace the destination's observation with an arbitrary one of their choosing (done in hopes of having the destination accept a fa… ▽ More This paper investigates the secret key authentication capacity region. Specifically, the focus is on a model where a source must transmit information over an adversary controlled channel where the adversary, prior to the source's transmission, decides whether or not to replace the destination's observation with an arbitrary one of their choosing (done in hopes of having the destination accept a false message). To combat the adversary, the source and destination share a secret key which they may use to guarantee authenticated communications. The secret key authentication capacity region here is then defined as the region of jointly achievable message rate, authentication rate, and key consumption rate (i.e., how many bits of secret key are needed). This is the first of a two part study, with the parts differing in how the authentication rate is measured. In this first study the authenticated rate is measured by the traditional metric of the maximum expected probability of false authentication. For this metric, we provide an inner bound which improves on those existing in the literature. This is achieved by adopting and merging different classical techniques in novel ways. Within these classical techniques, one technique derives authentication capability directly from the noisy communications channel, and the other technique derives its' authentication capability directly from obscuring the source. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Comments: First of a two part paper. Large text overlap in introductions. Submitted for possible publication

Secret key authentication capacity region, Part II: typical authentication rate

Authors: Eric Graves, Jake Perazzone, Paul Yu, Rick Blum

Abstract: This paper investigates the secret key authentication capacity region. Specifically, the focus is on a model where a source must transmit information over an adversary controlled channel where the adversary, prior to the source's transmission, decides whether or not to replace the destination's observation with an arbitrary one of their choosing (done in hopes of having the destination accept a fa… ▽ More This paper investigates the secret key authentication capacity region. Specifically, the focus is on a model where a source must transmit information over an adversary controlled channel where the adversary, prior to the source's transmission, decides whether or not to replace the destination's observation with an arbitrary one of their choosing (done in hopes of having the destination accept a false message). To combat the adversary, the source and destination share a secret key which they may use to guarantee authenticated communications. The secret key authentication capacity region here is then defined as the region of jointly achievable message rate, authentication rate, and key consumption rate (i.e., how many bits of secret key are needed). This is the second of a two part study, with the studies separated by how the authentication rate is measured. Here, the authentication rate is measured by the minimum of the maximum probability of false acceptance where the minimization is over all highly probable subsets of observations at the adversary. That is, consider the maximum probability of false authentication as a function of the adversary's observation, and the adversary's observation as a random variable. The authentication rate is then measured as the smallest number for which the probability that the maximum probability of false authentication is greater than said number is arbitrary small. This is termed typical authentication, since it only needs to consider adversarial observations which are typical with the transmission. Under this measure of authentication matching inner and outer bounds are determined. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Comments: Second of a two part paper. Large text overlap in introductions. Submitted for possible publication

Superlight -- A Permissionless, Light-client Only Blockchain with Self-Contained Proofs and BLS Signatures

Authors: Roman Blum, Thomas Bocek

Abstract: Blockchain protocols are based on a distributed database where stored data is guaranteed to be immutable. The requirement that all nodes have to maintain their own local copy of the database ensures security while consensus mechanisms help deciding which data gets added to the database and keep powerful adversaries from derailing the system. However, since the database that forms the foundation of… ▽ More Blockchain protocols are based on a distributed database where stored data is guaranteed to be immutable. The requirement that all nodes have to maintain their own local copy of the database ensures security while consensus mechanisms help deciding which data gets added to the database and keep powerful adversaries from derailing the system. However, since the database that forms the foundation of a blockchain is a continuously growing list of blocks, scalability is an inherent problem of this technology. Some public blockchains need a few 100 GB to Terabytes of storage. In this work, we present the concept Superlight with self-contained proofs, which is designed to improve scalability of a public blockchain, while preserving security and decentralization. Instead of all nodes having a local copy of the whole blockchain to verify a transaction, nodes can derive the validity of a transaction by only using block headers. To keep the block headers compact, BLS signatures are used to combine signatures. We provide a definition of SCPs and show the required steps of a client to create a proof that is accepted by other nodes for transferring funds. The advantage of such a light-client-only blockchain is the lower storage requirement, while the drawback is an increased computational complexity due to BLS signatures, limited use-cases due to lack of a global state, and the requirement for an interactive protocol between sender, receiver, and miner to create a transaction. △ Less

Submitted 16 September, 2019; v1 submitted 8 January, 2019; originally announced January 2019.

Energy-efficient Decision Fusion for Distributed Detection in Wireless Sensor Networks

Authors: N. Sriranga, K. G. Nagananda, R. S. Blum, A. Saucan, P. K. Varshney

Abstract: This paper proposes an energy-efficient counting rule for distributed detection by ordering sensor transmissions in wireless sensor networks. In the counting rule-based detection in an $N-$sensor network, the local sensors transmit binary decisions to the fusion center, where the number of all $N$ local-sensor detections are counted and compared to a threshold. In the ordering scheme, sensors tran… ▽ More This paper proposes an energy-efficient counting rule for distributed detection by ordering sensor transmissions in wireless sensor networks. In the counting rule-based detection in an $N-$sensor network, the local sensors transmit binary decisions to the fusion center, where the number of all $N$ local-sensor detections are counted and compared to a threshold. In the ordering scheme, sensors transmit their unquantized statistics to the fusion center in a sequential manner; highly informative sensors enjoy higher priority for transmission. When sufficient evidence is collected at the fusion center for decision making, the transmissions from the sensors are stopped. The ordering scheme achieves the same error probability as the optimum unconstrained energy approach (which requires observations from all the $N$ sensors) with far fewer sensor transmissions. The scheme proposed in this paper improves the energy efficiency of the counting rule detector by ordering the sensor transmissions: each sensor transmits at a time inversely proportional to a function of its observation. The resulting scheme combines the advantages offered by the counting rule (efficient utilization of the network's communication bandwidth, since the local decisions are transmitted in binary form to the fusion center) and ordering sensor transmissions (bandwidth efficiency, since the fusion center need not wait for all the $N$ sensors to transmit their local decisions), thereby leading to significant energy savings. As a concrete example, the problem of target detection in large-scale wireless sensor networks is considered. Under certain conditions the ordering-based counting rule scheme achieves the same detection performance as that of the original counting rule detector with fewer than $N/2$ sensor transmissions; in some cases, the savings in transmission approaches $(N-1)$. △ Less

Submitted 10 September, 2018; originally announced September 2018.

Comments: 7 pages, 3 figures. Proceedings of FUSION 2018, Cambridge, UK

Multi-snapshot Newtonized Orthogonal Matching Pursuit for Line Spectrum Estimation with Multiple Measurement Vectors

Authors: Jiang Zhu, Lin Han, Rick S. Blum, Zhiwei Xu

Abstract: In this paper, multi-snapshot Newtonized orthogonal matching pursuit (MNOMP) algorithm is proposed to deal with the line spectrum estimation with multiple measurement vectors (MMVs). MNOMP has the low computation complexity and state-of-the-art performance advantage of NOMP, and also includes two key steps: Detecting a new sinusoid on an oversampled discrete Fourier transform (DFT) grid and refini… ▽ More In this paper, multi-snapshot Newtonized orthogonal matching pursuit (MNOMP) algorithm is proposed to deal with the line spectrum estimation with multiple measurement vectors (MMVs). MNOMP has the low computation complexity and state-of-the-art performance advantage of NOMP, and also includes two key steps: Detecting a new sinusoid on an oversampled discrete Fourier transform (DFT) grid and refining the parameters of already detected sinusoids to avoid the problem of basis mismatch. We provide a stop** criterion based on the overestimating probability of the model order. In addition, the convergence of the proposed algorithm is also proved. Finally, numerical results are conducted to show that the performance of MNOMP benefits from MMVs, and the effectiveness of MNOMP when compared against the state-of-the-art algorithms in terms of frequency estimation accuracy and computation complexity. △ Less

Submitted 8 May, 2019; v1 submitted 4 February, 2018; originally announced February 2018.

Inner Bound for the Capacity Region of Noisy Channels with an Authentication Requirement

Authors: Jake Perazzone, Eric Graves, Paul Yu, Rick Blum

Abstract: The rate regions of many variations of the standard and wire-tap channels have been thoroughly explored. Secrecy capacity characterizes the loss of rate required to ensure that the adversary gains no information about the transmissions. Authentication does not have a standard metric, despite being an important counterpart to secrecy. While some results have taken an information-theoretic approach… ▽ More The rate regions of many variations of the standard and wire-tap channels have been thoroughly explored. Secrecy capacity characterizes the loss of rate required to ensure that the adversary gains no information about the transmissions. Authentication does not have a standard metric, despite being an important counterpart to secrecy. While some results have taken an information-theoretic approach to the problem of authentication coding, the full rate region and accompanying trade-offs have yet to be characterized. In this paper, we provide an inner bound of achievable rates with an average authentication and reliability constraint. The bound is established by combining and analyzing two existing authentication schemes for both noisy and noiseless channels. We find that our coding scheme improves upon existing schemes. △ Less

Submitted 26 June, 2018; v1 submitted 11 January, 2018; originally announced January 2018.

Signal Amplitude Estimation and Detection from Unlabeled Binary Quantized Samples

Authors: Guanyu Wang, Jiang Zhu, Rick S. Blum, Peter Willett, Stefano Marano, Vincenzo Matta, Paolo Braca

Abstract: Signal amplitude estimation and detection from unlabeled quantized binary samples are studied, assuming that the order of the time indexes is completely unknown. First, maximum likelihood (ML) estimators are utilized to estimate both the permutation matrix and unknown signal amplitude under arbitrary, but known signal shape and quantizer thresholds. Sufficient conditions are provided under which a… ▽ More Signal amplitude estimation and detection from unlabeled quantized binary samples are studied, assuming that the order of the time indexes is completely unknown. First, maximum likelihood (ML) estimators are utilized to estimate both the permutation matrix and unknown signal amplitude under arbitrary, but known signal shape and quantizer thresholds. Sufficient conditions are provided under which an ML estimator can be found in polynomial time and an alternating maximization algorithm is proposed to solve the general problem via good initial estimates. In addition, the statistical identifiability of the model is studied. Furthermore, the generalized likelihood ratio test (GLRT) detector is adopted to detect the presence of signal. In addition, an accurate approximation to the probability of successful permutation matrix recovery is derived, and explicit expressions are provided to reveal the relationship between the number of signal samples and the number of quantizers. Finally, numerical simulations are performed to verify the theoretical results. △ Less

Submitted 8 March, 2018; v1 submitted 4 June, 2017; originally announced June 2017.

Attack Detection in Sensor Network Target Localization Systems with Quantized Data

Authors: Jiangfan Zhang, Xiaodong Wang, Rick S. Blum, Lance M. Kaplan

Abstract: We consider a sensor network focused on target localization, where sensors measure the signal strength emitted from the target. Each measurement is quantized to one bit and sent to the fusion center. A general attack is considered at some sensors that attempts to cause the fusion center to produce an inaccurate estimation of the target location with a large mean-square-error. The attack is a combi… ▽ More We consider a sensor network focused on target localization, where sensors measure the signal strength emitted from the target. Each measurement is quantized to one bit and sent to the fusion center. A general attack is considered at some sensors that attempts to cause the fusion center to produce an inaccurate estimation of the target location with a large mean-square-error. The attack is a combination of man-in-the-middle, hacking, and spoofing attacks that can effectively change both signals going into and coming out of the sensor nodes in a realistic manner. We show that the essential effect of attacks is to alter the estimated distance between the target and each attacked sensor to a different extent, giving rise to a geometric inconsistency among the attacked and unattacked sensors. Hence, with the help of two secure sensors, a class of detectors are proposed to detect the attacked sensors by scrutinizing the existence of the geometric inconsistency. We show that the false alarm and miss probabilities of the proposed detectors decrease exponentially as the number of measurement samples increases, which implies that for sufficiently large number of samples, the proposed detectors can identify the attacked and unattacked sensors with any required accuracy. △ Less

Submitted 15 May, 2017; originally announced May 2017.

Sparse Representation based Multi-sensor Image Fusion: A Review

Authors: Qiang Zhang, Yi Liu, Rick S. Blum, Jungong Han, Dacheng Tao

Abstract: As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion. Unlike the traditional multiscale transforms (MSTs) that presume the basis functions, SR learns an over-complete dictionary from a set of training images for image fusion, and it achieves more stable and meaningful repr… ▽ More As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion. Unlike the traditional multiscale transforms (MSTs) that presume the basis functions, SR learns an over-complete dictionary from a set of training images for image fusion, and it achieves more stable and meaningful representations of the source images. By doing so, the SR-based fusion methods generally outperform the traditional MST-based image fusion methods in both subjective and objective tests. In addition, they are less susceptible to mis-registration among the source images, thus facilitating the practical applications. This survey paper proposes a systematic review of the SR-based multi-sensor image fusion literature, highlighting the pros and cons of each category of approaches. Specifically, we start by performing a theoretical investigation of the entire system from three key algorithmic aspects, (1) sparse representation models; (2) dictionary learning methods; and (3) activity levels and fusion rules. Subsequently, we show how the existing works address these scientific problems and design the appropriate fusion rules for each application, such as multi-focus image fusion and multi-modality (e.g., infrared and visible) image fusion. At last, we carry out some experiments to evaluate the impact of these three algorithmic components on the fusion performance when dealing with different applications. This article is expected to serve as a tutorial and source of reference for researchers preparing to enter the field or who desire to employ the sparse representation theory in other fields. △ Less

Submitted 12 February, 2017; originally announced February 2017.

Comments: 19 pages

Suboptimum Low Complexity Joint Multi-target Detection and Localization for Noncoherent MIMO Radar with Widely Separated Antennas

Authors: Wei Yi, Tao Zhou, Mingchi Xie, Yue Ai, Rick S. Blum

Abstract: In this paper, the problems of simultaneously detecting and localizing multiple targets are considered for noncoherent multiple-input multiple-output (MIMO) radar with widely separated antennas. By assuming a prior knowledge of target number, an optimal solution to this problem is presented first. It is essentially a maximum-likelihood (ML) estimator searching parameters of interest in a high dime… ▽ More In this paper, the problems of simultaneously detecting and localizing multiple targets are considered for noncoherent multiple-input multiple-output (MIMO) radar with widely separated antennas. By assuming a prior knowledge of target number, an optimal solution to this problem is presented first. It is essentially a maximum-likelihood (ML) estimator searching parameters of interest in a high dimensional space. However, the complexity of this method increases exponentially with the number G of targets.Besides, without the prior information of the number of targets, a multi-hypothesis testing strategy to determine the number of targets is required, which further complicates this method. Therefore, we split the joint maximization into G disjoint optimization problems by clearing the interference from previously declared targets. In this way, we derive two fast and robust suboptimal solutions which allow trading performance for a much lower implementation complexity which is almost independent of the number of targets. In addition, the multi-hypothesis testing is no longer required when target number is unknown. Simulation results show the proposed algorithms can correctly detect and accurately localize multiple targets even when targets share common range bins in some paths. △ Less

Submitted 5 January, 2017; originally announced January 2017.

Estimation Theory Based Robust Phase Offset Estimation in the Presence of Delay Attacks

Authors: Anantha K. Karthik, Rick S. Blum

Abstract: This paper addresses the problem of robust clock phase offset estimation for the IEEE 1588 precision time protocol (PTP) in the presence of delay attacks. Delay attacks are one of the most effective cyber attacks in PTP, as they cannot be mitigated using typical security measures. In this paper, we consider the case where the slave node can exchange synchronization messages with multiple master no… ▽ More This paper addresses the problem of robust clock phase offset estimation for the IEEE 1588 precision time protocol (PTP) in the presence of delay attacks. Delay attacks are one of the most effective cyber attacks in PTP, as they cannot be mitigated using typical security measures. In this paper, we consider the case where the slave node can exchange synchronization messages with multiple master nodes synchronized to the same clock. We first provide lower bounds on the best achievable performance for any phase offset estimation scheme in the presence of delay attacks. We then present a novel phase offset estimation scheme that employs the Expectation-Maximization algorithm for detecting which of the master-slave communication links have been subject to delay attacks. After discarding information from the links identified as attacked, which we show to be optimal, the optimal vector location parameter estimator is employed to estimate the phase offset of the slave node. Simulation results are presented to show that the proposed phase offset estimation scheme exhibits performance close to the lower bounds in a wide variety of scenarios. △ Less

Submitted 15 November, 2016; originally announced November 2016.

Comments: 30 pages, 4 figures, Journal paper

Low-Rank Tensor Decomposition-Aided Channel Estimation for Millimeter Wave MIMO-OFDM Systems

Authors: Zhou Zhou, Jun Fang, Linxiao Yang, Hongbin Li, Zhi Chen, Rick S. Blum

Abstract: We consider the problem of downlink channel estimation for millimeter wave (mmWave) MIMO-OFDM systems, where both the base station (BS) and the mobile station (MS) employ large antenna arrays for directional precoding/beamforming. Hybrid analog and digital beamforming structures are employed in order to offer a compromise between hardware complexity and system performance. Different from most exis… ▽ More We consider the problem of downlink channel estimation for millimeter wave (mmWave) MIMO-OFDM systems, where both the base station (BS) and the mobile station (MS) employ large antenna arrays for directional precoding/beamforming. Hybrid analog and digital beamforming structures are employed in order to offer a compromise between hardware complexity and system performance. Different from most existing studies that are concerned with narrowband channels, we consider estimation of wideband mmWave channels with frequency selectivity, which is more appropriate for mmWave MIMO-OFDM systems. By exploiting the sparse scattering nature of mmWave channels, we propose a CANDECOMP/PARAFAC (CP) decomposition-based method for channel parameter estimation (including angles of arrival/departure, time delays, and fading coefficients). In our proposed method, the received signal at the BS is expressed as a third-order tensor. We show that the tensor has the form of a low-rank CP decomposition, and the channel parameters can be estimated from the associated factor matrices. Our analysis reveals that the uniqueness of the CP decomposition can be guaranteed even when the size of the tensor is small. Hence the proposed method has the potential to achieve substantial training overhead reduction. We also develop Cramer-Rao bound (CRB) results for channel parameters, and compare our proposed method with a compressed sensing-based method. Simulation results show that the proposed method attains mean square errors that are very close to their associated CRBs, and presents a clear advantage over the compressed sensing-based method in terms of both estimation accuracy and computational complexity. △ Less

Submitted 1 November, 2016; v1 submitted 12 September, 2016; originally announced September 2016.

Comments: arXiv admin note: text overlap with arXiv:1602.07955

Performance Analysis of Target Parameters Estimation Using Multiple Widely Separated Antenna Arrays

Authors: Peter Khomchuk, Igal Bilik, Rick S. Blum

Abstract: Target parameter estimation performance is investigated for a radar employing a set of widely separated transmitting and receiving antenna arrays. Cases with multiple extended targets are considered under two signal model assumptions: stochastic and deterministic. The general expressions for the corresponding Cramer-Rao lower bound (CRLB) and the asymptotic properties of the maximum-likelihood (ML… ▽ More Target parameter estimation performance is investigated for a radar employing a set of widely separated transmitting and receiving antenna arrays. Cases with multiple extended targets are considered under two signal model assumptions: stochastic and deterministic. The general expressions for the corresponding Cramer-Rao lower bound (CRLB) and the asymptotic properties of the maximum-likelihood (ML) estimator are derived for a radar with $M_t$ arrays of $L_t$ transmitting elements and $M_r$ arrays of $L_r$ receiving elements for both types of signal models. It is shown that for an infinitely large product $M_tM_r$, and a finite $L_r$, the ML estimator is consistent and efficient under the stochastic model, while the deterministic model requires $M_tM_r$ to be finite and $L_r$ to be infinitely large in order to guarantee consistency and efficiency. Monte Carlo simulations further investigate the estimation performance of the proposed radar configuration in practical scenarios with finite $M_tM_r$ and $L_r$, and a fixed total number of available receiving antenna elements, $M_r L_r$. The numerical results demonstrate that grou** receiving elements into properly sized arrays reduces the mean squared error (MSE) and decreases the threshold SNR. In the numerical examples considered, the preferred configurations employ $M_t M_r > 1$. In fact, when $M_t M_r$ becomes too small, due to the loss of the geometric gain, the estimation performance becomes strongly dependent on the particular scenario and can degrade significantly, while the CRLB may become a poor prediction of the MSE even for high SNR. This suggests it may be advantageous to employ approaches where neither $M_tM_r$ nor $L_r$ are too small. △ Less

Submitted 15 June, 2016; originally announced June 2016.

Journal ref: P. Khomchuk, R. S. Blum, I. Bilik, "Performance analysis of target parameters estimation using multiple widely separated antenna arrays", IEEE Transactions on Aerospace and Electronic System, 52.5 (2016): 2413-2435

Functional Forms of Optimum Spoofing Attacks for Vector Parameter Estimation in Quantized Sensor Networks

Authors: Jiangfan Zhang, Rick S. Blum, Lance Kaplan, Xuanxuan Lu

Abstract: Estimation of an unknown deterministic vector from quantized sensor data is considered in the presence of spoofing attacks which alter the data presented to several sensors. Contrary to previous work, a generalized attack model is employed which manipulates the data using transformations with arbitrary functional forms determined by some attack parameters whose values are unknown to the attacked s… ▽ More Estimation of an unknown deterministic vector from quantized sensor data is considered in the presence of spoofing attacks which alter the data presented to several sensors. Contrary to previous work, a generalized attack model is employed which manipulates the data using transformations with arbitrary functional forms determined by some attack parameters whose values are unknown to the attacked system. For the first time, necessary and sufficient conditions are provided under which the transformations provide a guaranteed attack performance in terms of Cramer-Rao Bound (CRB) regardless of the processing the estimation system employs, thus defining a highly desirable attack. Interestingly, these conditions imply that, for any such attack when the attacked sensors can be perfectly identified by the estimation system, either the Fisher Information Matrix (FIM) for jointly estimating the desired and attack parameters is singular or that the attacked system is unable to improve the CRB for the desired vector parameter through this joint estimation even though the joint FIM is nonsingular. It is shown that it is always possible to construct such a highly desirable attack by properly employing a sufficiently large dimension attack vector parameter relative to the number of quantization levels employed, which was not observed previously. To illustrate the theory in a concrete way, we also provide some numerical results which corroborate that under the highly desirable attack, attacked data is not useful in reducing the CRB. △ Less

Submitted 1 June, 2016; v1 submitted 24 May, 2016; originally announced May 2016.

Wireless-Powered Cooperative Communications: Power-Splitting Relaying with Energy Accumulation

Authors: Zheng Zhou, Mugen Peng, Zhongyuan Zhao, Wenbo Wang, Rick S. Blum

Abstract: A harvest-use-store power splitting (PS) relaying strategy with distributed beamforming is proposed for wirelesspowered multi-relay cooperative networks in this paper. Different from the conventional battery-free PS relaying strategy, harvested energy is prioritized to power information relaying while the remainder is accumulated and stored for future usage with the help of a battery in the propos… ▽ More A harvest-use-store power splitting (PS) relaying strategy with distributed beamforming is proposed for wirelesspowered multi-relay cooperative networks in this paper. Different from the conventional battery-free PS relaying strategy, harvested energy is prioritized to power information relaying while the remainder is accumulated and stored for future usage with the help of a battery in the proposed strategy, which supports an efficient utilization of harvested energy. However, PS affects throughput at subsequent time slots due to the battery operations including the charging and discharging. To this end, PS and battery operations are coupled with distributed beamforming. A throughput optimization problem to incorporate these coupled operations is formulated though it is intractable. To address the intractability of the optimization,a layered optimization method is proposed to achieve the optimal joint PS and battery operation design with non-causal channel state information (CSI), in which the PS and the battery operation can be analyzed in a decomposed manner. Then, a general case with causal CSI is considered, where the proposed layered optimization method is extended by utilizing the statistical properties of CSI. To reach a better tradeoff between performance and complexity, a greedy method that requires no information about subsequent time slots is proposed. Simulation results reveal the upper and lower bound on performance of the proposed strategy, which are reached by the layered optimization method with non-causal CSI and the greedy method, respectively. Moreover, the proposed strategy outperforms the conventional PS-based relaying without energy accumulation and time switching-based relaying strategy. △ Less

Submitted 3 January, 2016; originally announced January 2016.

Comments: 15 pages, 7 figures. Manuscript received Apr. 15, 2015 by IEEE Journal on Selected Areas in Communications, revised Sep. 5, 2015, accepted Dec. 11, 2015

Generalized Cramer-Rao Bound for Joint Estimation of Target Position and Velocity for Active and Passive Radar Networks

Authors: Qian He, Jianbin Hu, Rick S. Blum, Yonggang Wu

Abstract: In this paper, we derive the Cramer-Rao bound (CRB) for joint target position and velocity estimation using an active or passive distributed radar network under more general, and practically occurring, conditions than assumed in previous work. In particular, the presented results allow nonorthogonal signals, spatially dependent Gaussian reflection coefficients, and spatially dependent Gaussian clu… ▽ More In this paper, we derive the Cramer-Rao bound (CRB) for joint target position and velocity estimation using an active or passive distributed radar network under more general, and practically occurring, conditions than assumed in previous work. In particular, the presented results allow nonorthogonal signals, spatially dependent Gaussian reflection coefficients, and spatially dependent Gaussian clutter-plus-noise. These bounds allow designers to compare the performance of their developed approaches, which are deemed to be of acceptable complexity, to the best achievable performance. If their developed approaches lead to performance close to the bounds, these developed approaches can be deemed "good enough". A particular recent study where algorithms have been developed for a practical radar application which must involve nonorthogonal signals, for which the best performance is unknown, is a great example. The presented results in our paper do not make any assumptions about the approximate location of the target being known from previous target detection signal processing. In addition, for situations in which we do not know some parameters accurately, we also derive the mismatched CRB. Numerical investigations of the mean squared error of the maximum likelihood estimation are employed to support the validity of the CRBs. In order to demonstrate the utility of the provided results to a topic of great current interest, the numerical results focus on a passive radar system using the Global System for Mobile communication (GSM) cellar system. △ Less

Submitted 8 October, 2015; originally announced October 2015.

Colocated MIMO Radar Waveform Design for Transmit Beampattern Formation

Authors: Haisheng Xu, Rick S. Blum, Jian Wang, Jian Yuan

Abstract: In this paper, colocated MIMO radar waveform design is considered by minimizing the integrated side-lobe level to obtain beam patterns with lower side-lobe levels than competing methods. First, a quadratic programming problem is formulated to design beam patterns by using the criteria for a minimal integrated side-lobe level. A theorem is derived that provides a closed-form analytical optimal solu… ▽ More In this paper, colocated MIMO radar waveform design is considered by minimizing the integrated side-lobe level to obtain beam patterns with lower side-lobe levels than competing methods. First, a quadratic programming problem is formulated to design beam patterns by using the criteria for a minimal integrated side-lobe level. A theorem is derived that provides a closed-form analytical optimal solution that appears to be an extension of the Rayleigh quotient minimization for a possibly singular matrix in quadratic form. Such singularities are shown to occur in the problem of interest, but proofs for the optimum solution in these singular matrix cases could not be found in the literature. Next, an additional constraint is added to obtain beam patterns with desired 3 dB beamwidths, resulting in a nonconvex quadratically constrained quadratic program which is NP-hard. A semidefinite program and a Gaussian randomized semidefinite relaxation are used to determine feasible solutions arbitrarily close to the solution to the original problem. Theoretical and numerical analyses illustrate the impacts of changing the number of transmitters and orthogonal waveforms employed in the designs. Numerical comparisons are conducted to evaluate the proposed design approaches. △ Less

Submitted 5 May, 2015; originally announced May 2015.

Comments: 22 pages, 6 figures, Accepted by IEEE Transactions on Aerospace and Electronic Systems

Journal ref: IEEE Transactions on Aerospace and Electronic Systems 51(2015) 1558 - 1568

Minimax Optimum Estimators for Phase Synchronization in IEEE 1588

Authors: Anand Guruswamy, Rick S. Blum, Shalinee Kishore, Mark Bordogna

Abstract: The IEEE 1588 protocol has received recent interest as a means of delivering sub-microsecond level clock phase synchronization over packet-switched mobile backhaul networks. Due to the randomness of the end-to-end delays in packet networks, the recovery of clock phase from packet timestamps in IEEE 1588 must be treated as a statistical estimation problem. A number of estimators for this problem ha… ▽ More The IEEE 1588 protocol has received recent interest as a means of delivering sub-microsecond level clock phase synchronization over packet-switched mobile backhaul networks. Due to the randomness of the end-to-end delays in packet networks, the recovery of clock phase from packet timestamps in IEEE 1588 must be treated as a statistical estimation problem. A number of estimators for this problem have been suggested in the literature, but little is known about the best achievable performance. In this paper, we describe new minimax estimators for this problem, that are optimum in terms of minimizing the maximum mean squared error over all possible values of the unknown parameters. Minimax estimators that utilize information from past timestamps to improve accuracy are also introduced. Simulation results indicate that significant performance gains over conventional estimators can be obtained via such optimum processing techniques. These minimax estimators also provide fundamental limits on the performance of phase offset estimation schemes. △ Less

Submitted 28 February, 2015; originally announced March 2015.

Comments: 11 pages, 19 figures

Super-Resolution Compressed Sensing: A Generalized Iterative Reweighted L2 Approach

Authors: Jun Fang, Hui** Duan, **g Li, Hongbin Li, Rick S. Blum

Abstract: Conventional compressed sensing theory assumes signals have sparse representations in a known, finite dictionary. Nevertheless, in many practical applications such as direction-of-arrival (DOA) estimation and line spectral estimation, the sparsifying dictionary is usually characterized by a set of unknown parameters in a continuous domain. To apply the conventional compressed sensing technique to… ▽ More Conventional compressed sensing theory assumes signals have sparse representations in a known, finite dictionary. Nevertheless, in many practical applications such as direction-of-arrival (DOA) estimation and line spectral estimation, the sparsifying dictionary is usually characterized by a set of unknown parameters in a continuous domain. To apply the conventional compressed sensing technique to such applications, the continuous parameter space has to be discretized to a finite set of grid points, based on which a "presumed dictionary" is constructed for sparse signal recovery. Discretization, however, inevitably incurs errors since the true parameters do not necessarily lie on the discretized grid. This error, also referred to as grid mismatch, may lead to deteriorated recovery performance or even recovery failure. To address this issue, in this paper, we propose a generalized iterative reweighted L2 method which jointly estimates the sparse signals and the unknown parameters associated with the true dictionary. The proposed algorithm is developed by iteratively decreasing a surrogate function majorizing a given objective function, resulting in a gradual and interweaved iterative process to refine the unknown parameters and the sparse signal. A simple yet effective scheme is developed for adaptively updating the regularization parameter that controls the tradeoff between the sparsity of the solution and the data fitting error. Extension of the proposed algorithm to the multiple measurement vector scenario is also considered. Numerical results show that the proposed algorithm achieves a super-resolution accuracy and presents superiority over other existing methods. △ Less

Submitted 18 December, 2014; v1 submitted 8 December, 2014; originally announced December 2014.

Comments: arXiv admin note: text overlap with arXiv:1401.4312

A PMU Scheduling Scheme for Transmission of Synchrophasor Data in Electric Power Systems

Authors: K. G. Nagananda, Shalinee Kishore, Rick S. Blum

Abstract: With the proposition to install a large number of phasor measurement units (PMUs) in the future power grid, it is essential to provide robust communications infrastructure for phasor data across the network. We make progress in this direction by devising a simple time division multiplexing scheme for transmitting phasor data from the PMUs to a central server: Time is divided into frames and the PM… ▽ More With the proposition to install a large number of phasor measurement units (PMUs) in the future power grid, it is essential to provide robust communications infrastructure for phasor data across the network. We make progress in this direction by devising a simple time division multiplexing scheme for transmitting phasor data from the PMUs to a central server: Time is divided into frames and the PMUs take turns to transmit to the control center within the time frame. The main contribution of this work is a scheduling policy based on which PMU transmissions are ordered during a time frame. The scheduling scheme is independent of the approach taken to solve the PMU placement problem, and unlike strategies devised for conventional communications, it is intended for the power network since it is fully governed by the measure of electrical connectedness between buses in the grid. To quantify the performance of the scheduling scheme, we couple it with a fault detection algorithm used to detect changes in the susceptance parameters in the grid. Results demonstrate that scheduling the PMU transmissions leads to an improved performance of the fault detection scheme compared to PMUs transmitting at random. △ Less

Submitted 31 December, 2014; v1 submitted 27 May, 2014; originally announced May 2014.

Comments: 9 pages, 6 figures; an extra figure included in the published version. appears in IEEE Transactions on Smart Grid, Special Issue on Cyber Physical Systems and Security for Smart Grid, 2015

Target Localization Accuracy Gain in MIMO Radar Based Systems

Authors: Hana Godrich, Alexander M. Haimovich, Rick S. Blum

Abstract: This paper presents an analysis of target localization accuracy, attainable by the use of MIMO (Multiple-Input Multiple-Output) radar systems, configured with multiple transmit and receive sensors, widely distributed over a given area. The Cramer-Rao lower bound (CRLB) for target localization accuracy is developed for both coherent and non-coherent processing. Coherent processing requires a comm… ▽ More This paper presents an analysis of target localization accuracy, attainable by the use of MIMO (Multiple-Input Multiple-Output) radar systems, configured with multiple transmit and receive sensors, widely distributed over a given area. The Cramer-Rao lower bound (CRLB) for target localization accuracy is developed for both coherent and non-coherent processing. Coherent processing requires a common phase reference for all transmit and receive sensors. The CRLB is shown to be inversely proportional to the signal effective bandwidth in the non-coherent case, but is approximately inversely proportional to the carrier frequency in the coherent case. We further prove that optimization over the sensors' positions lowers the CRLB by a factor equal to the product of the number of transmitting and receiving sensors. The best linear unbiased estimator (BLUE) is derived for the MIMO target localization problem. The BLUE's utility is in providing a closed form localization estimate that facilitates the analysis of the relations between sensors locations, target location, and localization accuracy. Geometric dilution of precision (GDOP) contours are used to map the relative performance accuracy for a given layout of radars over a given geographic area. △ Less

Submitted 24 September, 2008; originally announced September 2008.

Comments: 36 pages, 5 figures, submitted to IEEE Transaction on Information Theory