Showing 1–15 of 15 results for author: Hero, A O

Community Detection in High-Dimensional Graph Ensembles

Authors: Robert Malinas, Dogyoon Song, Alfred O. Hero III

Abstract: Detecting communities in high-dimensional graphs can be achieved by applying random matrix theory where the adjacency matrix of the graph is modeled by a Stochastic Block Model (SBM). However, the SBM makes an unrealistic assumption that the edge probabilities are homogeneous within communities, i.e., the edges occur with the same probabilities. The Degree-Corrected SBM is a generalization of the… ▽ More Detecting communities in high-dimensional graphs can be achieved by applying random matrix theory where the adjacency matrix of the graph is modeled by a Stochastic Block Model (SBM). However, the SBM makes an unrealistic assumption that the edge probabilities are homogeneous within communities, i.e., the edges occur with the same probabilities. The Degree-Corrected SBM is a generalization of the SBM that allows these edge probabilities to be different, but existing results from random matrix theory are not directly applicable to this heterogeneous model. In this paper, we derive a transformation of the adjacency matrix that eliminates this heterogeneity and preserves the relevant eigenstructure for community detection. We propose a test based on the extreme eigenvalues of this transformed matrix and (1) provide a method for controlling the significance level, (2) formulate a conjecture that the test achieves power one for all positive significance levels in the limit as the number of nodes approaches infinity, and (3) provide empirical evidence and theory supporting these claims. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: 8 pages, 3 figures

Graph Sparsification by Approximate Matrix Multiplication

Authors: Neophytos Charalambides, Alfred O. Hero III

Abstract: Graphs arising in statistical problems, signal processing, large networks, combinatorial optimization, and data analysis are often dense, which causes both computational and storage bottlenecks. One way of \textit{sparsifying} a \textit{weighted} graph, while sharing the same vertices as the original graph but reducing the number of edges, is through \textit{spectral sparsification}. We study this… ▽ More Graphs arising in statistical problems, signal processing, large networks, combinatorial optimization, and data analysis are often dense, which causes both computational and storage bottlenecks. One way of \textit{sparsifying} a \textit{weighted} graph, while sharing the same vertices as the original graph but reducing the number of edges, is through \textit{spectral sparsification}. We study this problem through the perspective of RandNLA. Specifically, we utilize randomized matrix multiplication to give a clean and simple analysis of how sampling according to edge weights gives a spectral approximation to graph Laplacians. Through the $CR$-MM algorithm, we attain a simple and computationally efficient sparsifier whose resulting Laplacian estimate is unbiased and of minimum variance. Furthermore, we define a new notion of \textit{additive spectral sparsifiers}, which has not been considered in the literature. △ Less

Submitted 26 April, 2023; v1 submitted 17 April, 2023; originally announced April 2023.

MSC Class: 65F50 65F50; 65F55; 68W20; 68W25; 05C90; 05C50; 05C85 ACM Class: F.2.1; G.3; G.1.2; G.1.3; G.2.2

Orthonormal Sketches for Secure Coded Regression

Authors: Neophytos Charalambides, Hessam Mahdavifar, Mert Pilanci, Alfred O. Hero III

Abstract: In this work, we propose a method for speeding up linear regression distributively, while ensuring security. We leverage randomized sketching techniques, and improve straggler resilience in asynchronous systems. Specifically, we apply a random orthonormal matrix and then subsample in \textit{blocks}, to simultaneously secure the information and reduce the dimension of the regression problem. In ou… ▽ More In this work, we propose a method for speeding up linear regression distributively, while ensuring security. We leverage randomized sketching techniques, and improve straggler resilience in asynchronous systems. Specifically, we apply a random orthonormal matrix and then subsample in \textit{blocks}, to simultaneously secure the information and reduce the dimension of the regression problem. In our setup, the transformation corresponds to an encoded encryption in an \textit{approximate} gradient coding scheme, and the subsampling corresponds to the responses of the non-straggling workers; in a centralized coded computing network. We focus on the special case of the \textit{Subsampled Randomized Hadamard Transform}, which we generalize to block sampling; and discuss how it can be used to secure the data. We illustrate the performance through numerical experiments. △ Less

Submitted 22 February, 2022; v1 submitted 20 January, 2022; originally announced January 2022.

Comments: 3 figures, 5 pages excluding appendices

MSC Class: 65F10; 65F45; 68W15; 68W20; 68W25; 68P27; 68P30; ACM Class: E.3; E.4; G.1.2; G.1.3

Data Discovery Using Lossless Compression-Based Sparse Representation

Authors: Elyas Sabeti, Peter X. K. Song, Alfred O. Hero III

Abstract: Sparse representation has been widely used in data compression, signal and image denoising, dimensionality reduction and computer vision. While overcomplete dictionaries are required for sparse representation of multidimensional data, orthogonal bases represent one-dimensional data well. In this paper, we propose a data-driven sparse representation using orthonormal bases under the lossless compre… ▽ More Sparse representation has been widely used in data compression, signal and image denoising, dimensionality reduction and computer vision. While overcomplete dictionaries are required for sparse representation of multidimensional data, orthogonal bases represent one-dimensional data well. In this paper, we propose a data-driven sparse representation using orthonormal bases under the lossless compression constraint. We show that imposing such constraint under the Minimum Description Length (MDL) principle leads to a unique and optimal sparse representation for one-dimensional data, which results in discriminative features useful for data discovery. △ Less

Submitted 16 March, 2021; v1 submitted 15 March, 2021; originally announced March 2021.

Space-Time Adaptive Detection at Low Sample Support

Authors: Benjamin D. Robinson, Robert Malinas, Alfred O. Hero III

Abstract: An important problem in space-time adaptive detection is the estimation of the large p-by-p interference covariance matrix from training signals. When the number of training signals n is greater than 2p, existing estimators are generally considered to be adequate, as demonstrated by fixed-dimensional asymptotics. But in the low-sample-support regime (n < 2p or even n < p) fixed-dimensional asympto… ▽ More An important problem in space-time adaptive detection is the estimation of the large p-by-p interference covariance matrix from training signals. When the number of training signals n is greater than 2p, existing estimators are generally considered to be adequate, as demonstrated by fixed-dimensional asymptotics. But in the low-sample-support regime (n < 2p or even n < p) fixed-dimensional asymptotics are no longer applicable. The remedy undertaken in this paper is to consider the "large dimensional limit" in which n and p go to infinity together. In this asymptotic regime, a new type of estimator is defined (Definition 2), shown to exist (Theorem 1), and shown to be detection-theoretically ideal (Theorem 2). Further, asymptotic conditional detection and false-alarm rates of filters formed from this type of estimator are characterized (Theorems 3 and 4) and shown to depend only on data that is given, even for non-Gaussian interference statistics. The paper concludes with several Monte Carlo simulations that compare the performance of the estimator in Theorem 1 to the predictions of Theorems 2-4, showing in particular higher detection probability than Steiner and Gerlach's Fast Maximum Likelihood estimator. △ Less

Submitted 7 October, 2020; originally announced October 2020.

Comments: 13 pages, 3 figures

Journal ref: IEEE Transactions on Signal Processing (2021)

Adaptive multi-channel event segmentation and feature extraction for monitoring health outcomes

Authors: Xichen She, Yaya Zhai, Ricardo Henao, Christopher W. Woods, Christopher Chiu, Geoffrey S. Ginsburg, Peter X. K. Song, Alfred O. Hero

Abstract: $\textbf{Objective}$: To develop a multi-channel device event segmentation and feature extraction algorithm that is robust to changes in data distribution. $\textbf{Methods}… ▽ More $\textbf{Objective}$: To develop a multi-channel device event segmentation and feature extraction algorithm that is robust to changes in data distribution. $\textbf{Methods}$: We introduce an adaptive transfer learning algorithm to classify and segment events from non-stationary multi-channel temporal data. Using a multivariate hidden Markov model (HMM) and Fisher's linear discriminant analysis (FLDA) the algorithm adaptively adjusts to shifts in distribution over time. The proposed algorithm is unsupervised and learns to label events without requiring $\textit{a priori}$ information about true event states. The procedure is illustrated on experimental data collected from a cohort in a human viral challenge (HVC) study, where certain subjects have disrupted wake and sleep patterns after exposure to a H1N1 influenza pathogen. $\textbf{Results}$: Simulations establish that the proposed adaptive algorithm significantly outperforms other event classification methods. When applied to early time points in the HVC data the algorithm extracts sleep/wake features that are predictive of both infection and infection onset time. $\textbf{Conclusion}$: The proposed transfer learning event segmentation method is robust to temporal shifts in data distribution and can be used to produce highly discriminative event-labeled features for health monitoring. $\textbf{Significance}$: Our integrated multisensor signal processing and transfer learning method is applicable to many ambulatory monitoring applications. △ Less

Submitted 19 November, 2020; v1 submitted 20 August, 2020; originally announced August 2020.

Journal ref: IEEE Transactions on Biomedical Engineering, Nov. 17 2020

Pattern-Based Analysis of Time Series: Estimation

Authors: Elyas Sabeti, Peter X. K. Song, Alfred O. Hero

Abstract: While Internet of Things (IoT) devices and sensors create continuous streams of information, Big Data infrastructures are deemed to handle the influx of data in real-time. One type of such a continuous stream of information is time series data. Due to the richness of information in time series and inadequacy of summary statistics to encapsulate structures and patterns in such data, development of… ▽ More While Internet of Things (IoT) devices and sensors create continuous streams of information, Big Data infrastructures are deemed to handle the influx of data in real-time. One type of such a continuous stream of information is time series data. Due to the richness of information in time series and inadequacy of summary statistics to encapsulate structures and patterns in such data, development of new approaches to learn time series is of interest. In this paper, we propose a novel method, called pattern tree, to learn patterns in the times-series using a binary-structured tree. While a pattern tree can be used for many purposes such as lossless compression, prediction and anomaly detection, in this paper we focus on its application in time series estimation and forecasting. In comparison to other methods, our proposed pattern tree method improves the mean squared error of estimation. △ Less

Submitted 2 May, 2020; originally announced May 2020.

Numerically Stable Binary Gradient Coding

Authors: Neophytos Charalambides, Hessam Mahdavifar, Alfred O. Hero III

Abstract: A major hurdle in machine learning is scalability to massive datasets. One approach to overcoming this is to distribute the computational tasks among several workers. \textit{Gradient coding} has been recently proposed in distributed optimization to compute the gradient of an objective function using multiple, possibly unreliable, worker nodes. By designing distributed coded schemes, gradient code… ▽ More A major hurdle in machine learning is scalability to massive datasets. One approach to overcoming this is to distribute the computational tasks among several workers. \textit{Gradient coding} has been recently proposed in distributed optimization to compute the gradient of an objective function using multiple, possibly unreliable, worker nodes. By designing distributed coded schemes, gradient coded computations can be made resilient to \textit{stragglers}, nodes with longer response time comparing to other nodes in a distributed network. Most such schemes rely on operations over the real or complex numbers and are inherently numerically unstable. We present a binary scheme which avoids such operations, thereby enabling numerically stable distributed computation of the gradient. Also, some restricting assumptions in prior work are dropped, and a more efficient decoding is given. △ Less

Submitted 15 September, 2020; v1 submitted 30 January, 2020; originally announced January 2020.

Comments: Conference

MSC Class: 94Bxx; 94B60

Robust Distributed Fixed-Time Economic Dispatch under Time-Varying Topology

Authors: Mayank Baranwal, Kunal Garg, Dimitra Panagou, Alfred O. Hero

Abstract: The centralized power generation infrastructure that defines the North American electric grid is slowly moving to the distributed architecture due to the explosion in use of renewable generation and distributed energy resources (DERs), such as residential solar, wind turbines and battery storage. Furthermore, variable pricing policies and profusion of flexible loads entail frequent and severe chan… ▽ More The centralized power generation infrastructure that defines the North American electric grid is slowly moving to the distributed architecture due to the explosion in use of renewable generation and distributed energy resources (DERs), such as residential solar, wind turbines and battery storage. Furthermore, variable pricing policies and profusion of flexible loads entail frequent and severe changes in power outputs required from the individual generation units, requiring fast availability of power allocation. To this end, a fixed-time convergent, fully distributed economic dispatch algorithm for scheduling optimal power generation among a set of DERs is proposed. The proposed algorithm incorporates both load balance and generation capacity constraints. △ Less

Submitted 26 August, 2020; v1 submitted 30 October, 2019; originally announced October 2019.

Comments: 6 pages, 3 figures, to appear in L-CSS

Journal ref: IEEE Control Systems Letters, vol. 5, no. 4, pp. 1183-1188, Oct. 2021

Testing that a Local Optimum of the Likelihood is Globally Optimum using Reparameterized Embeddings

Authors: Joel W. LeBlanc, Brian J. Thelen, Alfred O. Hero

Abstract: Many mathematical imaging problems are posed as non-convex optimization problems. When numerically tractable global optimization procedures are not available, one is often interested in testing ex post facto whether or not a locally convergent algorithm has found the globally optimal solution. When the problem is formulated in terms of maximizing the likelihood function under a statistical model f… ▽ More Many mathematical imaging problems are posed as non-convex optimization problems. When numerically tractable global optimization procedures are not available, one is often interested in testing ex post facto whether or not a locally convergent algorithm has found the globally optimal solution. When the problem is formulated in terms of maximizing the likelihood function under a statistical model for the measurements, one can construct a statistical test that a local maximum is in fact the global maximum. A one-sided test is proposed for the case that the statistical model is a member of the generalized location family of probability distributions, a condition often satisfied in imaging and other inverse problems. We propose a general method for improving the accuracy of the test by reparameterizing the likelihood function to embed its domain into a higher dimensional parameter space. We show that the proposed global maximum testing method results in improved accuracy and reduced computation for a physically-motivated joint-inverse problem arising in camera-blur estimation. △ Less

Submitted 10 July, 2020; v1 submitted 31 May, 2019; originally announced June 2019.

A New Data-Driven Sparse-Learning Approach to Study Chemical Reaction Networks

Authors: Farshad Harirchi, Doohyun Kim, Omar A. Khalil, Sijia Liu, Paolo Elvati, Angela Violi, Alfred O. Hero

Abstract: Chemical kinetic mechanisms can be represented by sets of elementary reactions that are easily translated into mathematical terms using physicochemical relationships. The schematic representation of reactions captures the interactions between reacting species and products. Determining the minimal chemical interactions underlying the dynamic behavior of systems is a major task. In this paper, we in… ▽ More Chemical kinetic mechanisms can be represented by sets of elementary reactions that are easily translated into mathematical terms using physicochemical relationships. The schematic representation of reactions captures the interactions between reacting species and products. Determining the minimal chemical interactions underlying the dynamic behavior of systems is a major task. In this paper, we introduce a novel approach for the identification of the influential reactions in chemical reaction networks for combustion applications, using a data-driven sparse-learning technique. The proposed approach identifies a set of influential reactions using species concentrations and reaction rates, with minimal computational cost without requiring additional data or simulations. The new approach is applied to analyze the combustion chemistry of H2 and C3H8 in a constant-volume homogeneous reactor. The influential reactions identified by the sparse-learning method are consistent with the current kinetics knowledge of chemical mechanisms. Additionally, we show that a reduced version of the parent mechanism can be generated as a combination of the influential reactions identified at different times and conditions and that for both H2 and C3H8 this reduced mechanism performs closely to the parent mechanism as a function of ignition delay over a wide range of conditions. Our results demonstrate the potential of the sparse-learning approach as an effective and efficient tool for mechanism analysis and mechanism reduction. △ Less

Submitted 10 February, 2019; v1 submitted 18 December, 2017; originally announced December 2017.

Fundamental Limits on Data Acquisition: Trade-offs between Sample Complexity and Query Difficulty

Authors: Hye Won Chung, Ji Oon Lee, Alfred O. Hero

Abstract: We consider query-based data acquisition and the corresponding information recovery problem, where the goal is to recover $k$ binary variables (information bits) from parity measurements of those variables. The queries and the corresponding parity measurements are designed using the encoding rule of Fountain codes. By using Fountain codes, we can design potentially limitless number of queries, and… ▽ More We consider query-based data acquisition and the corresponding information recovery problem, where the goal is to recover $k$ binary variables (information bits) from parity measurements of those variables. The queries and the corresponding parity measurements are designed using the encoding rule of Fountain codes. By using Fountain codes, we can design potentially limitless number of queries, and corresponding parity measurements, and guarantee that the original $k$ information bits can be recovered with high probability from any sufficiently large set of measurements of size $n$. In the query design, the average number of information bits that is associated with one parity measurement is called query difficulty ($\bar{d}$) and the minimum number of measurements required to recover the $k$ information bits for a fixed $\bar{d}$ is called sample complexity ($n$). We analyze the fundamental trade-offs between the query difficulty and the sample complexity, and show that the sample complexity of $n=c\max\{k,(k\log k)/\bar{d}\}$ for some constant $c>0$ is necessary and sufficient to recover $k$ information bits with high probability as $k\to\infty$. △ Less

Submitted 2 January, 2018; v1 submitted 30 November, 2017; originally announced December 2017.

Learning Sparse Graphs Under Smoothness Prior

Authors: Sundeep Prabhakar Chepuri, Sijia Liu, Geert Leus, Alfred O. Hero III

Abstract: In this paper, we are interested in learning the underlying graph structure behind training data. Solving this basic problem is essential to carry out any graph signal processing or machine learning task. To realize this, we assume that the data is smooth with respect to the graph topology, and we parameterize the graph topology using an edge sampling function. That is, the graph Laplacian is expr… ▽ More In this paper, we are interested in learning the underlying graph structure behind training data. Solving this basic problem is essential to carry out any graph signal processing or machine learning task. To realize this, we assume that the data is smooth with respect to the graph topology, and we parameterize the graph topology using an edge sampling function. That is, the graph Laplacian is expressed in terms of a sparse edge selection vector, which provides an explicit handle to control the sparsity level of the graph. We solve the sparse graph learning problem given some training data in both the noiseless and noisy settings. Given the true smooth data, the posed sparse graph learning problem can be solved optimally and is based on simple rank ordering. Given the noisy data, we show that the joint sparse graph learning and denoising problem can be simplified to designing only the sparse edge selection vector, which can be solved using convex optimization. △ Less

Submitted 12 September, 2016; originally announced September 2016.

Comments: ICASSP 2017 conference paper

On Decentralized Estimation with Active Queries

Authors: Theodoros Tsiligkaridis, Brian M. Sadler, Alfred O. Hero III

Abstract: We consider the problem of decentralized 20 questions with noise for multiple players/agents under the minimum entropy criterion in the setting of stochastic search over a parameter space, with application to target localization. We propose decentralized extensions of the active query-based stochastic search strategy that combines elements from the 20 questions approach and social learning. We pro… ▽ More We consider the problem of decentralized 20 questions with noise for multiple players/agents under the minimum entropy criterion in the setting of stochastic search over a parameter space, with application to target localization. We propose decentralized extensions of the active query-based stochastic search strategy that combines elements from the 20 questions approach and social learning. We prove convergence to correct consensus on the value of the parameter. This framework provides a flexible and tractable mathematical model for decentralized parameter estimation systems based on active querying. We illustrate the effectiveness and robustness of the proposed decentralized collaborative 20 questions algorithm for random network topologies with information sharing. △ Less

Submitted 5 February, 2015; v1 submitted 30 December, 2013; originally announced December 2013.

Comments: 22 pages, to appear in IEEE Transactions on Signal Processing

Sensor Management: Past, Present, and Future

Authors: Alfred O. Hero III, Douglas Cochran

Abstract: Sensor systems typically operate under resource constraints that prevent the simultaneous use of all resources all of the time. Sensor management becomes relevant when the sensing system has the capability of actively managing these resources; i.e., changing its operating configuration during deployment in reaction to previous measurements. Examples of systems in which sensor management is current… ▽ More Sensor systems typically operate under resource constraints that prevent the simultaneous use of all resources all of the time. Sensor management becomes relevant when the sensing system has the capability of actively managing these resources; i.e., changing its operating configuration during deployment in reaction to previous measurements. Examples of systems in which sensor management is currently used or is likely to be used in the near future include autonomous robots, surveillance and reconnaissance networks, and waveform-agile radars. This paper provides an overview of the theory, algorithms, and applications of sensor management as it has developed over the past decades and as it stands today. △ Less

Submitted 11 September, 2011; originally announced September 2011.

Comments: 15 pages, 112 references

Journal ref: IEEE Sensors Journal, vol. 11, issue 12, pp. 3064-3075, December 2011