Showing 1–36 of 36 results for author: Spanias, A

An L2-Normalized Spatial Attention Network For Accurate And Fast Classification Of Brain Tumors In 2D T1-Weighted CE-MRI Images

Authors: Grace Billingsley, Julia Dietlmeier, Vivek Narayanaswamy, Andreas Spanias, Noel E. OConnor

Abstract: We propose an accurate and fast classification network for classification of brain tumors in MRI images that outperforms all lightweight methods investigated in terms of accuracy. We test our model on a challenging 2D T1-weighted CE-MRI dataset containing three types of brain tumors: Meningioma, Glioma and Pituitary. We introduce an l2-normalized spatial attention mechanism that acts as a regulari… ▽ More We propose an accurate and fast classification network for classification of brain tumors in MRI images that outperforms all lightweight methods investigated in terms of accuracy. We test our model on a challenging 2D T1-weighted CE-MRI dataset containing three types of brain tumors: Meningioma, Glioma and Pituitary. We introduce an l2-normalized spatial attention mechanism that acts as a regularizer against overfitting during training. We compare our results against the state-of-the-art on this dataset and show that by integrating l2-normalized spatial attention into a baseline network we achieve a performance gain of 1.79 percentage points. Even better accuracy can be attained by combining our model in an ensemble with the pretrained VGG16 at the expense of execution speed. Our code is publicly available at https://github.com/juliadietlmeier/MRI_image_classification △ Less

Submitted 1 August, 2023; originally announced August 2023.

Comments: Accepted to be published in: IEEE International Conference on Image Processing (ICIP), Kuala Lumpur October 8-11, 2023

Towards Live 3D Reconstruction from Wearable Video: An Evaluation of V-SLAM, NeRF, and Videogrammetry Techniques

Authors: David Ramirez, Suren Jayasuriya, Andreas Spanias

Abstract: Mixed reality (MR) is a key technology which promises to change the future of warfare. An MR hybrid of physical outdoor environments and virtual military training will enable engagements with long distance enemies, both real and simulated. To enable this technology, a large-scale 3D model of a physical environment must be maintained based on live sensor observations. 3D reconstruction algorithms s… ▽ More Mixed reality (MR) is a key technology which promises to change the future of warfare. An MR hybrid of physical outdoor environments and virtual military training will enable engagements with long distance enemies, both real and simulated. To enable this technology, a large-scale 3D model of a physical environment must be maintained based on live sensor observations. 3D reconstruction algorithms should utilize the low cost and pervasiveness of video camera sensors, from both overhead and soldier-level perspectives. Map** speed and 3D quality can be balanced to enable live MR training in dynamic environments. Given these requirements, we survey several 3D reconstruction algorithms for large-scale map** for military applications given only live video. We measure 3D reconstruction performance from common structure from motion, visual-SLAM, and photogrammetry techniques. This includes the open source algorithms COLMAP, ORB-SLAM3, and NeRF using Instant-NGP. We utilize the autonomous driving academic benchmark KITTI, which includes both dashboard camera video and lidar produced 3D ground truth. With the KITTI data, our primary contribution is a quantitative evaluation of 3D reconstruction computational speed when considering live video. △ Less

Submitted 21 November, 2022; originally announced November 2022.

Comments: Accepted to 2022 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), 13 pages

Know Your Space: Inlier and Outlier Construction for Calibrating Medical OOD Detectors

Authors: Vivek Narayanaswamy, Yamen Mubarka, Rushil Anirudh, Deepta Rajan, Andreas Spanias, Jayaraman J. Thiagarajan

Abstract: We focus on the problem of producing well-calibrated out-of-distribution (OOD) detectors, in order to enable safe deployment of medical image classifiers. Motivated by the difficulty of curating suitable calibration datasets, synthetic augmentations have become highly prevalent for inlier/outlier specification. While there have been rapid advances in data augmentation techniques, this paper makes… ▽ More We focus on the problem of producing well-calibrated out-of-distribution (OOD) detectors, in order to enable safe deployment of medical image classifiers. Motivated by the difficulty of curating suitable calibration datasets, synthetic augmentations have become highly prevalent for inlier/outlier specification. While there have been rapid advances in data augmentation techniques, this paper makes a striking finding that the space in which the inliers and outliers are synthesized, in addition to the type of augmentation, plays a critical role in calibrating OOD detectors. Using the popular energy-based OOD detection framework, we find that the optimal protocol is to synthesize latent-space inliers along with diverse pixel-space outliers. Based on empirical studies with multiple medical imaging benchmarks, we demonstrate that our approach consistently leads to superior OOD detection ($15\% - 35\%$ in AUROC) over the state-of-the-art in a variety of open-set recognition settings. △ Less

Submitted 22 April, 2023; v1 submitted 11 July, 2022; originally announced July 2022.

Adaptive Subsampling for ROI-based Visual Tracking: Algorithms and FPGA Implementation

Authors: Odrika Iqbal, Victor Isaac Torres Muro, Sameeksha Katoch, Andreas Spanias, Suren Jayasuriya

Abstract: There is tremendous scope for improving the energy efficiency of embedded vision systems by incorporating programmable region-of-interest (ROI) readout in the image sensor design. In this work, we study how ROI programmability can be leveraged for tracking applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region. We refer to this p… ▽ More There is tremendous scope for improving the energy efficiency of embedded vision systems by incorporating programmable region-of-interest (ROI) readout in the image sensor design. In this work, we study how ROI programmability can be leveraged for tracking applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region. We refer to this process of ROI prediction and corresponding sensor configuration as adaptive subsampling. Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking. To further facilitate the implementation of our adaptive algorithms in real life, we select a candidate algorithm and map it onto an FPGA. Leveraging Xilinx Vitis AI tools, we designed and accelerated a YOLO object detector-based adaptive subsampling algorithm. In order to further improve the algorithm post-deployment, we evaluated several competing baselines on the OTB100 and LaSOT datasets. We found that coupling the ECO tracker with the Kalman filter has a competitive AUC score of 0.4568 and 0.3471 on the OTB100 and LaSOT datasets respectively. Further, the power efficiency of this algorithm is on par with, and in a couple of instances superior to, the other baselines. The ECO-based algorithm incurs a power consumption of approximately 4 W averaged across both datasets while the YOLO-based approach requires power consumption of approximately 6 W (as per our power consumption model). In terms of accuracy-latency tradeoff, the ECO-based algorithm provides near-real-time performance (19.23 FPS) while managing to attain competitive tracking precision. △ Less

Submitted 17 January, 2022; v1 submitted 17 December, 2021; originally announced December 2021.

Designing Counterfactual Generators using Deep Model Inversion

Authors: Jayaraman J. Thiagarajan, Vivek Narayanaswamy, Deepta Rajan, Jason Liang, Akshay Chaudhari, Andreas Spanias

Abstract: Explanation techniques that synthesize small, interpretable changes to a given image while producing desired changes in the model prediction have become popular for introspecting black-box models. Commonly referred to as counterfactuals, the synthesized explanations are required to contain discernible changes (for easy interpretability) while also being realistic (consistency to the data manifold)… ▽ More Explanation techniques that synthesize small, interpretable changes to a given image while producing desired changes in the model prediction have become popular for introspecting black-box models. Commonly referred to as counterfactuals, the synthesized explanations are required to contain discernible changes (for easy interpretability) while also being realistic (consistency to the data manifold). In this paper, we focus on the case where we have access only to the trained deep classifier and not the actual training data. While the problem of inverting deep models to synthesize images from the training distribution has been explored, our goal is to develop a deep inversion approach to generate counterfactual explanations for a given query image. Despite their effectiveness in conditional image synthesis, we show that existing deep inversion methods are insufficient for producing meaningful counterfactuals. We propose DISC (Deep Inversion for Synthesizing Counterfactuals) that improves upon deep inversion by utilizing (a) stronger image priors, (b) incorporating a novel manifold consistency objective and (c) adopting a progressive optimization strategy. We find that, in addition to producing visually meaningful explanations, the counterfactuals from DISC are effective at learning classifier decision boundaries and are robust to unknown test-time corruptions. △ Less

Submitted 5 October, 2021; v1 submitted 29 September, 2021; originally announced September 2021.

Comments: Neurips 2021

On the Design of Deep Priors for Unsupervised Audio Restoration

Authors: Vivek Sivaraman Narayanaswamy, Jayaraman J. Thiagarajan, Andreas Spanias

Abstract: Unsupervised deep learning methods for solving audio restoration problems extensively rely on carefully tailored neural architectures that carry strong inductive biases for defining priors in the time or spectral domain. In this context, lot of recent success has been achieved with sophisticated convolutional network constructions that recover audio signals in the spectral domain. However, in prac… ▽ More Unsupervised deep learning methods for solving audio restoration problems extensively rely on carefully tailored neural architectures that carry strong inductive biases for defining priors in the time or spectral domain. In this context, lot of recent success has been achieved with sophisticated convolutional network constructions that recover audio signals in the spectral domain. However, in practice, audio priors require careful engineering of the convolutional kernels to be effective at solving ill-posed restoration tasks, while also being easy to train. To this end, in this paper, we propose a new U-Net based prior that does not impact either the network complexity or convergence behavior of existing convolutional architectures, yet leads to significantly improved restoration. In particular, we advocate the use of carefully designed dilation schedules and dense connections in the U-Net architecture to obtain powerful audio priors. Using empirical studies on standard benchmarks and a variety of ill-posed restoration tasks, such as audio denoising, in-painting and source separation, we demonstrate that our proposed approach consistently outperforms widely adopted audio prior architectures. △ Less

Submitted 14 April, 2021; originally announced April 2021.

Loss Estimators Improve Model Generalization

Authors: Vivek Narayanaswamy, Jayaraman J. Thiagarajan, Deepta Rajan, Andreas Spanias

Abstract: With increased interest in adopting AI methods for clinical diagnosis, a vital step towards safe deployment of such tools is to ensure that the models not only produce accurate predictions but also do not generalize to data regimes where the training data provide no meaningful evidence. Existing approaches for ensuring the distribution of model predictions to be similar to that of the true distrib… ▽ More With increased interest in adopting AI methods for clinical diagnosis, a vital step towards safe deployment of such tools is to ensure that the models not only produce accurate predictions but also do not generalize to data regimes where the training data provide no meaningful evidence. Existing approaches for ensuring the distribution of model predictions to be similar to that of the true distribution rely on explicit uncertainty estimators that are inherently hard to calibrate. In this paper, we propose to train a loss estimator alongside the predictive model, using a contrastive training objective, to directly estimate the prediction uncertainties. Interestingly, we find that, in addition to producing well-calibrated uncertainties, this approach improves the generalization behavior of the predictor. Using a dermatology use-case, we show the impact of loss estimators on model generalization, in terms of both its fidelity on in-distribution data and its ability to detect out of distribution samples or new classes unseen during training. △ Less

Submitted 5 March, 2021; originally announced March 2021.

Using Deep Image Priors to Generate Counterfactual Explanations

Authors: Vivek Narayanaswamy, Jayaraman J. Thiagarajan, Andreas Spanias

Abstract: Through the use of carefully tailored convolutional neural network architectures, a deep image prior (DIP) can be used to obtain pre-images from latent representation encodings. Though DIP inversion has been known to be superior to conventional regularized inversion strategies such as total variation, such an over-parameterized generator is able to effectively reconstruct even images that are not… ▽ More Through the use of carefully tailored convolutional neural network architectures, a deep image prior (DIP) can be used to obtain pre-images from latent representation encodings. Though DIP inversion has been known to be superior to conventional regularized inversion strategies such as total variation, such an over-parameterized generator is able to effectively reconstruct even images that are not in the original data distribution. This limitation makes it challenging to utilize such priors for tasks such as counterfactual reasoning, wherein the goal is to generate small, interpretable changes to an image that systematically leads to changes in the model prediction. To this end, we propose a novel regularization strategy based on an auxiliary loss estimator jointly trained with the predictor, which efficiently guides the prior to recover natural pre-images. Our empirical studies with a real-world ISIC skin lesion detection problem clearly evidence the effectiveness of the proposed approach in synthesizing meaningful counterfactuals. In comparison, we find that the standard DIP inversion often proposes visually imperceptible perturbations to irrelevant parts of the image, thus providing no additional insights into the model behavior. △ Less

Submitted 22 October, 2020; originally announced October 2020.

Uncertainty-Matching Graph Neural Networks to Defend Against Poisoning Attacks

Authors: Uday Shankar Shanthamallu, Jayaraman J. Thiagarajan, Andreas Spanias

Abstract: Graph Neural Networks (GNNs), a generalization of neural networks to graph-structured data, are often implemented using message passes between entities of a graph. While GNNs are effective for node classification, link prediction and graph classification, they are vulnerable to adversarial attacks, i.e., a small perturbation to the structure can lead to a non-trivial performance degradation. In th… ▽ More Graph Neural Networks (GNNs), a generalization of neural networks to graph-structured data, are often implemented using message passes between entities of a graph. While GNNs are effective for node classification, link prediction and graph classification, they are vulnerable to adversarial attacks, i.e., a small perturbation to the structure can lead to a non-trivial performance degradation. In this work, we propose Uncertainty Matching GNN (UM-GNN), that is aimed at improving the robustness of GNN models, particularly against poisoning attacks to the graph structure, by leveraging epistemic uncertainties from the message passing framework. More specifically, we propose to build a surrogate predictor that does not directly access the graph structure, but systematically extracts reliable knowledge from a standard GNN through a novel uncertainty-matching strategy. Interestingly, this uncoupling makes UM-GNN immune to evasion attacks by design, and achieves significantly improved robustness against poisoning attacks. Using empirical studies with standard benchmarks and a suite of global and target attacks, we demonstrate the effectiveness of UM-GNN, when compared to existing baselines including the state-of-the-art robust GCN. △ Less

Submitted 30 September, 2020; originally announced September 2020.

Accurate and Robust Feature Importance Estimation under Distribution Shifts

Authors: Jayaraman J. Thiagarajan, Vivek Narayanaswamy, Rushil Anirudh, Peer-Timo Bremer, Andreas Spanias

Abstract: With increasing reliance on the outcomes of black-box models in critical applications, post-hoc explainability tools that do not require access to the model internals are often used to enable humans understand and trust these models. In particular, we focus on the class of methods that can reveal the influence of input features on the predicted outputs. Despite their wide-spread adoption, existing… ▽ More With increasing reliance on the outcomes of black-box models in critical applications, post-hoc explainability tools that do not require access to the model internals are often used to enable humans understand and trust these models. In particular, we focus on the class of methods that can reveal the influence of input features on the predicted outputs. Despite their wide-spread adoption, existing methods are known to suffer from one or more of the following challenges: computational complexities, large uncertainties and most importantly, inability to handle real-world domain shifts. In this paper, we propose PRoFILE, a novel feature importance estimation method that addresses all these challenges. Through the use of a loss estimator jointly trained with the predictive model and a causal objective, PRoFILE can accurately estimate the feature importance scores even under complex distribution shifts, without any additional re-training. To this end, we also develop learning strategies for training the loss estimator, namely contrastive and dropout calibration, and find that it can effectively detect distribution shifts. Using empirical studies on several benchmark image and non-image data, we show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness. △ Less

Submitted 30 September, 2020; originally announced September 2020.

Unsupervised Audio Source Separation using Generative Priors

Authors: Vivek Narayanaswamy, Jayaraman J. Thiagarajan, Rushil Anirudh, Andreas Spanias

Abstract: State-of-the-art under-determined audio source separation systems rely on supervised end-end training of carefully tailored neural network architectures operating either in the time or the spectral domain. However, these methods are severely challenged in terms of requiring access to expensive source level labeled data and being specific to a given set of sources and the mixing process, which dema… ▽ More State-of-the-art under-determined audio source separation systems rely on supervised end-end training of carefully tailored neural network architectures operating either in the time or the spectral domain. However, these methods are severely challenged in terms of requiring access to expensive source level labeled data and being specific to a given set of sources and the mixing process, which demands complete re-training when those assumptions change. This strongly emphasizes the need for unsupervised methods that can leverage the recent advances in data-driven modeling, and compensate for the lack of labeled data through meaningful priors. To this end, we propose a novel approach for audio source separation based on generative priors trained on individual sources. Through the use of projected gradient descent optimization, our approach simultaneously searches in the source-specific latent spaces to effectively recover the constituent sources. Though the generative priors can be defined in the time domain directly, e.g. WaveGAN, we find that using spectral domain loss functions for our optimization leads to good-quality source estimates. Our empirical studies on standard spoken digit and instrument datasets clearly demonstrate the effectiveness of our approach over classical as well as state-of-the-art unsupervised baselines. △ Less

Submitted 27 May, 2020; originally announced May 2020.

Comments: 5 pages, 2 figures

Invenio: Discovering Hidden Relationships Between Tasks/Domains Using Structured Meta Learning

Authors: Sameeksha Katoch, Kowshik Thopalli, Jayaraman J. Thiagarajan, Pavan Turaga, Andreas Spanias

Abstract: Exploiting known semantic relationships between fine-grained tasks is critical to the success of recent model agnostic approaches. These approaches often rely on meta-optimization to make a model robust to systematic task or domain shifts. However, in practice, the performance of these methods can suffer, when there are no coherent semantic relationships between the tasks (or domains). We present… ▽ More Exploiting known semantic relationships between fine-grained tasks is critical to the success of recent model agnostic approaches. These approaches often rely on meta-optimization to make a model robust to systematic task or domain shifts. However, in practice, the performance of these methods can suffer, when there are no coherent semantic relationships between the tasks (or domains). We present Invenio, a structured meta-learning algorithm to infer semantic similarities between a given set of tasks and to provide insights into the complexity of transferring knowledge between different tasks. In contrast to existing techniques such as Task2Vec and Taskonomy, which measure similarities between pre-trained models, our approach employs a novel self-supervised learning strategy to discover these relationships in the training loop and at the same time utilizes them to update task-specific models in the meta-update step. Using challenging task and domain databases, under few-shot learning settings, we show that Invenio can discover intricate dependencies between tasks or domains, and can provide significant gains over existing approaches in terms of generalization performance. The learned semantic structure between tasks/domains from Invenio is interpretable and can be used to construct meaningful priors for tasks or domains. △ Less

Submitted 1 February, 2020; v1 submitted 24 November, 2019; originally announced November 2019.

Comments: Semantic structure development for tasks/domains essential for efficient knowledge transfer

Audio Source Separation via Multi-Scale Learning with Dilated Dense U-Nets

Authors: Vivek Sivaraman Narayanaswamy, Sameeksha Katoch, Jayaraman J. Thiagarajan, Huan Song, Andreas Spanias

Abstract: Modern audio source separation techniques rely on optimizing sequence model architectures such as, 1D-CNNs, on mixture recordings to generalize well to unseen mixtures. Specifically, recent focus is on time-domain based architectures such as Wave-U-Net which exploit temporal context by extracting multi-scale features. However, the optimality of the feature extraction process in these architectures… ▽ More Modern audio source separation techniques rely on optimizing sequence model architectures such as, 1D-CNNs, on mixture recordings to generalize well to unseen mixtures. Specifically, recent focus is on time-domain based architectures such as Wave-U-Net which exploit temporal context by extracting multi-scale features. However, the optimality of the feature extraction process in these architectures has not been well investigated. In this paper, we examine and recommend critical architectural changes that forge an optimal multi-scale feature extraction process. To this end, we replace regular $1-$D convolutions with adaptive dilated convolutions that have innate capability of capturing increased context by using large temporal receptive fields. We also investigate the impact of dense connections on the extraction process that encourage feature reuse and better gradient flow. The dense connections between the downsampling and upsampling paths of a U-Net architecture capture multi-resolution information leading to improved temporal modelling. We evaluate the proposed approaches on the MUSDB test dataset. In addition to providing an improved performance over the state-of-the-art, we also provide insights on the impact of different architectural choices on complex data-driven solutions for source separation. △ Less

Submitted 8 April, 2019; originally announced April 2019.

Location Estimation and Detection in Wireless Sensor Networks in the Presence of Fading

Authors: Xue Zhang, Cihan Tepedelenlioglu, Mahesh K. Banavar, Andreas Spanias, Gowtham Muniraju

Abstract: In this paper, localization using narrowband communication signals are considered in the presence of fading channels with time of arrival measurements. When narrowband signals are used for localization, due to existing hardware constraints, fading channels play a crucial role in localization accuracy. In a location estimation formulation, the Cramer-Rao lower bound for localization error is derive… ▽ More In this paper, localization using narrowband communication signals are considered in the presence of fading channels with time of arrival measurements. When narrowband signals are used for localization, due to existing hardware constraints, fading channels play a crucial role in localization accuracy. In a location estimation formulation, the Cramer-Rao lower bound for localization error is derived under different assumptions on fading coefficients. For the same level of localization accuracy, the loss in performance due to Rayleigh fading with known phase is shown to be about 5dB compared to the case with no fading. Unknown phase causes an additional 1dB loss. The maximum likelihood estimators are also derived. In an alternative distributed detection formulation, each anchor receives a noisy signal from a node with known location if the node is active. Each anchor makes a decision as to whether the node is active or not and transmits a bit to a fusion center once a decision is made. The fusion center combines all the decisions and uses a design parameter to make the final decision. We derive optimal thresholds and calculate the probabilities of false alarm and detection under different assumptions on the knowledge of channel information. Simulations corroborate our analytical results. △ Less

Submitted 2 November, 2018; originally announced November 2018.

Comments: Accepted in Physical Communications (PHYCOM)

Designing an Effective Metric Learning Pipeline for Speaker Diarization

Authors: Vivek Sivaraman Narayanaswamy, Jayaraman J. Thiagarajan, Huan Song, Andreas Spanias

Abstract: State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained $i-$vectors to representations learned via different sequence modeling architectures (e.g.… ▽ More State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained $i-$vectors to representations learned via different sequence modeling architectures (e.g. 1D-CNNs, LSTMs, attention models), while adopting off-the-shelf metric learning solutions. In this paper, we argue that, regardless of the feature extractor, it is crucial to carefully design a metric learning pipeline, namely the loss function, the sampling strategy and the discrimnative margin parameter, for building robust diarization systems. Furthermore, we propose to adopt a fine-grained validation process to obtain a comprehensive evaluation of the generalization power of metric learning pipelines. To this end, we measure diarization performance across different language speakers, and variations in the number of speakers in a recording. Using empirical studies, we provide interesting insights into the effectiveness of different design choices and make recommendations. △ Less

Submitted 31 October, 2018; originally announced November 2018.

A Regularized Attention Mechanism for Graph Attention Networks

Authors: Uday Shankar Shanthamallu, Jayaraman J. Thiagarajan, Andreas Spanias

Abstract: Machine learning models that can exploit the inherent structure in data have gained prominence. In particular, there is a surge in deep learning solutions for graph-structured data, due to its wide-spread applicability in several fields. Graph attention networks (GAT), a recent addition to the broad class of feature learning models in graphs, utilizes the attention mechanism to efficiently learn c… ▽ More Machine learning models that can exploit the inherent structure in data have gained prominence. In particular, there is a surge in deep learning solutions for graph-structured data, due to its wide-spread applicability in several fields. Graph attention networks (GAT), a recent addition to the broad class of feature learning models in graphs, utilizes the attention mechanism to efficiently learn continuous vector representations for semi-supervised learning problems. In this paper, we perform a detailed analysis of GAT models, and present interesting insights into their behavior. In particular, we show that the models are vulnerable to heterogeneous rogue nodes and hence propose novel regularization strategies to improve the robustness of GAT models. Using benchmark datasets, we demonstrate performance improvements on semi-supervised learning, using the proposed robust variant of GAT. △ Less

Submitted 10 February, 2020; v1 submitted 31 October, 2018; originally announced November 2018.

GrAMME: Semi-Supervised Learning using Multi-layered Graph Attention Models

Authors: Uday Shankar Shanthamallu, Jayaraman J. Thiagarajan, Huan Song, Andreas Spanias

Abstract: Modern data analysis pipelines are becoming increasingly complex due to the presence of multi-view information sources. While graphs are effective in modeling complex relationships, in many scenarios a single graph is rarely sufficient to succinctly represent all interactions, and hence multi-layered graphs have become popular. Though this leads to richer representations, extending solutions from… ▽ More Modern data analysis pipelines are becoming increasingly complex due to the presence of multi-view information sources. While graphs are effective in modeling complex relationships, in many scenarios a single graph is rarely sufficient to succinctly represent all interactions, and hence multi-layered graphs have become popular. Though this leads to richer representations, extending solutions from the single-graph case is not straightforward. Consequently, there is a strong need for novel solutions to solve classical problems, such as node classification, in the multi-layered case. In this paper, we consider the problem of semi-supervised learning with multi-layered graphs. Though deep network embeddings, e.g. DeepWalk, are widely adopted for community discovery, we argue that feature learning with random node attributes, using graph neural networks, can be more effective. To this end, we propose to use attention models for effective feature learning, and develop two novel architectures, GrAMME-SG and GrAMME-Fusion, that exploit the inter-layer dependencies for building multi-layered graph embeddings. Using empirical studies on several benchmark datasets, we evaluate the proposed approaches and demonstrate significant performance improvements in comparison to state-of-the-art network embedding strategies. The results also show that using simple random features is an effective choice, even in cases where explicit node attributes are not available. △ Less

Submitted 27 March, 2019; v1 submitted 2 October, 2018; originally announced October 2018.

Coverage-Based Designs Improve Sample Mining and Hyper-Parameter Optimization

Authors: Gowtham Muniraju, Bhavya Kailkhura, Jayaraman J. Thiagarajan, Peer-Timo Bremer, Cihan Tepedelenlioglu, Andreas Spanias

Abstract: Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning, and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling and hyper-parameter optimization. Existing solutions attempt to adaptively trade-off between global exploration and local exploitation, wherein the ini… ▽ More Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning, and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling and hyper-parameter optimization. Existing solutions attempt to adaptively trade-off between global exploration and local exploitation, wherein the initial exploratory sample is critical to their success. While discrepancy-based samples have become the de facto approach for exploration, results from computer graphics suggest that coverage-based designs, e.g. Poisson disk sampling, can be a superior alternative. In order to successfully adopt coverage-based sample designs to ML applications, which were originally developed for 2-d image analysis, we propose fundamental advances by constructing a parameterized family of designs with provably improved coverage characteristics, and by develo** algorithms for effective sample synthesis. Using experiments in sample mining and hyper-parameter optimization for supervised learning, we show that our approach consistently outperforms existing exploratory sampling methods in both blind exploration, and sequential search with Bayesian optimization. △ Less

Submitted 4 April, 2019; v1 submitted 5 September, 2018; originally announced September 2018.

Triplet Network with Attention for Speaker Diarization

Authors: Huan Song, Megan Willi, Jayaraman J. Thiagarajan, Visar Berisha, Andreas Spanias

Abstract: In automatic speech processing systems, speaker diarization is a crucial front-end component to separate segments from different speakers. Inspired by the recent success of deep neural networks (DNNs) in semantic inferencing, triplet loss-based architectures have been successfully used for this problem. However, existing work utilizes conventional i-vectors as the input representation and builds s… ▽ More In automatic speech processing systems, speaker diarization is a crucial front-end component to separate segments from different speakers. Inspired by the recent success of deep neural networks (DNNs) in semantic inferencing, triplet loss-based architectures have been successfully used for this problem. However, existing work utilizes conventional i-vectors as the input representation and builds simple fully connected networks for metric learning, thus not fully leveraging the modeling power of DNN architectures. This paper investigates the importance of learning effective representations from the sequences directly in metric learning pipelines for speaker diarization. More specifically, we propose to employ attention models to learn embeddings and the metric jointly in an end-to-end fashion. Experiments are conducted on the CALLHOME conversational speech corpus. The diarization results demonstrate that, besides providing a unified model, the proposed approach achieves improved performance when compared against existing approaches. △ Less

Submitted 4 August, 2018; originally announced August 2018.

Comments: Interspeech2018

Consensus-based Distributed Quantile Estimation in Sensor Networks

Authors: Jongmin Lee, Cihan Tepedelenlioglu, Andreas Spanias

Abstract: A quantile is defined as a value below which random draws from a given distribution falls with a given probability. In a centralized setting where the cumulative distribution function (CDF) is unknown, the empirical CDF (ECDF) can be used to estimate such quantiles after aggregating the data. In a fully distributed sensor network, however, it is challenging to estimate quantiles. This is because e… ▽ More A quantile is defined as a value below which random draws from a given distribution falls with a given probability. In a centralized setting where the cumulative distribution function (CDF) is unknown, the empirical CDF (ECDF) can be used to estimate such quantiles after aggregating the data. In a fully distributed sensor network, however, it is challenging to estimate quantiles. This is because each sensor node observes local measurement data with limited storage and data transmission power which make it difficult to obtain the global ECDF. This paper proposes consensus-based quantile estimation for such a distributed network. The states of the proposed algorithm are recursively updated with two-steps at each iteration: one is a local update based on the measurement data and the current state, and the other is averaging the updated states with neighboring nodes. We consider the realistic case of communication links between nodes being corrupted by independent random noise. It is shown that the estimated state sequence is asymptotically unbiased and converges toward the sample quantile in the mean-square sense. The two step-size sequences corresponding to the averaging and local update steps result in a mixed-time scale algorithm with proper decay rates in order to achieve convergence. We also provide applications to distributed estimation of trimmed mean, computation of median, maximum, or minimum values and identification of outliers through simulation. △ Less

Submitted 30 April, 2018; originally announced May 2018.

Optimizing Kernel Machines using Deep Learning

Authors: Huan Song, Jayaraman J. Thiagarajan, Prasanna Sattigeri, Andreas Spanias

Abstract: Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are… ▽ More Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task-specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this article, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the DKMO (Deep Kernel Machine Optimization) framework, that creates an ensemble of dense embeddings using Nystrom kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pre-trained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques. △ Less

Submitted 14 November, 2017; originally announced November 2017.

Attend and Diagnose: Clinical Time Series Analysis using Attention Models

Authors: Huan Song, Deepta Rajan, Jayaraman J. Thiagarajan, Andreas Spanias

Abstract: With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art results in several clinical prediction tasks. Despite the success of RNNs, its seque… ▽ More With widespread adoption of electronic health records, there is an increased emphasis for predictive models that can effectively deal with clinical time-series data. Powered by Recurrent Neural Network (RNN) architectures with Long Short-Term Memory (LSTM) units, deep neural networks have achieved state-of-the-art results in several clinical prediction tasks. Despite the success of RNNs, its sequential nature prohibits parallelized computing, thus making it inefficient particularly when processing long sequences. Recently, architectures which are based solely on attention mechanisms have shown remarkable success in transduction tasks in NLP, while being computationally superior. In this paper, for the first time, we utilize attention models for clinical time-series modeling, thereby dispensing recurrence entirely. We develop the \textit{SAnD} (Simply Attend and Diagnose) architecture, which employs a masked, self-attention mechanism, and uses positional encoding and dense interpolation strategies for incorporating temporal order. Furthermore, we develop a multi-task variant of \textit{SAnD} to jointly infer models with multiple diagnosis tasks. Using the recent MIMIC-III benchmark datasets, we demonstrate that the proposed approach achieves state-of-the-art performance in all tasks, outperforming LSTM models and classical baselines with hand-engineered features. △ Less

Submitted 19 November, 2017; v1 submitted 10 November, 2017; originally announced November 2017.

Comments: AAAI 2018

Direct estimation of density functionals using a polynomial basis

Authors: Alan Wisler, Visar Berisha, Andreas Spanias, Alfred O. Hero

Abstract: A number of fundamental quantities in statistical signal processing and information theory can be expressed as integral functions of two probability density functions. Such quantities are called density functionals as they map density functions onto the real line. For example, information divergence functions measure the dissimilarity between two probability density functions and are useful in a n… ▽ More A number of fundamental quantities in statistical signal processing and information theory can be expressed as integral functions of two probability density functions. Such quantities are called density functionals as they map density functions onto the real line. For example, information divergence functions measure the dissimilarity between two probability density functions and are useful in a number of applications. Typically, estimating these quantities requires complete knowledge of the underlying distribution followed by multi-dimensional integration. Existing methods make parametric assumptions about the data distribution or use non-parametric density estimation followed by high-dimensional integration. In this paper, we propose a new alternative. We introduce the concept of "data-driven basis functions" - functions of distributions whose value we can estimate given only samples from the underlying distributions without requiring distribution fitting or direct integration. We derive a new data-driven complete basis that is similar to the deterministic Bernstein polynomial basis and develop two methods for performing basis expansions of functionals of two distributions. We also show that the new basis set allows us to approximate functions of distributions as closely as desired. Finally, we evaluate the methodology by develo** data driven estimators for the Kullback-Leibler divergences and the Hellinger distance and by constructing empirical estimates of tight bounds on the Bayes error rate. △ Less

Submitted 20 November, 2017; v1 submitted 21 February, 2017; originally announced February 2017.

Comments: Under review for IEEE Transactions on Signal Processing

A Deep Learning Approach To Multiple Kernel Fusion

Authors: Huan Song, Jayaraman J. Thiagarajan, Prasanna Sattigeri, Karthikeyan Natesan Ramamurthy, Andreas Spanias

Abstract: Kernel fusion is a popular and effective approach for combining multiple features that characterize different aspects of data. Traditional approaches for Multiple Kernel Learning (MKL) attempt to learn the parameters for combining the kernels through sophisticated optimization procedures. In this paper, we propose an alternative approach that creates dense embeddings for data using the kernel simi… ▽ More Kernel fusion is a popular and effective approach for combining multiple features that characterize different aspects of data. Traditional approaches for Multiple Kernel Learning (MKL) attempt to learn the parameters for combining the kernels through sophisticated optimization procedures. In this paper, we propose an alternative approach that creates dense embeddings for data using the kernel similarities and adopts a deep neural network architecture for fusing the embeddings. In order to improve the effectiveness of this network, we introduce the kernel dropout regularization strategy coupled with the use of an expanded set of composition kernels. Experiment results on a real-world activity recognition dataset show that the proposed architecture is effective in fusing kernels and achieves state-of-the-art performance. △ Less

Submitted 28 December, 2016; originally announced December 2016.

Undergraduate Signal Processing Laboratories for the Android Operating System

Authors: Suhas Ranganath, JJ Thiagarajan, KN Ramamurthy, Shuang Hu, Mahesh Banavar, Andreas Spanias

Abstract: We present a DSP simulation environment that will enable students to perform laboratory exercises using Android mobile devices and tablets. Due to the pervasive nature of the mobile technology, education applications designed for mobile devices have the potential to stimulate student interest in addition to offering convenient access and interaction capabilities. This paper describes a portable si… ▽ More We present a DSP simulation environment that will enable students to perform laboratory exercises using Android mobile devices and tablets. Due to the pervasive nature of the mobile technology, education applications designed for mobile devices have the potential to stimulate student interest in addition to offering convenient access and interaction capabilities. This paper describes a portable signal processing laboratory for the Android platform. This software is intended to be an educational tool for students and instructors in DSP, and signals and systems courses. The development of Android JDSP (A-JDSP) is carried out using the Android SDK, which is a Java-based open source development platform. The proposed application contains basic DSP functions for convolution, sampling, FFT, filtering and frequency domain analysis, with a convenient graphical user interface. A description of the architecture, functions and planned assessments are presented in this paper. △ Less

Submitted 24 February, 2015; originally announced February 2015.

Comments: ASEE (American Society for Engineering Education) 2012

Empirically Estimable Classification Bounds Based on a New Divergence Measure

Authors: Visar Berisha, Alan Wisler, Alfred O. Hero, Andreas Spanias

Abstract: Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the same distribution and for the case where there exists some mismatch between traini… ▽ More Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the same distribution and for the case where there exists some mismatch between training and test distributions. We confirm the theoretical results by designing feature selection algorithms using the criteria from these bounds and by evaluating the algorithms on a series of pathological speech classification tasks. △ Less

Submitted 10 February, 2015; v1 submitted 19 December, 2014; originally announced December 2014.

Comments: 12 pages, 5 figures

Robust Consensus in the Presence of Impulsive Channel Noise

Authors: Sivaraman Dasarathan, Cihan Tepedelenlioglu, Mahesh Banavar, Andreas Spanias

Abstract: A distributed average consensus algorithm robust to a wide range of impulsive channel noise distributions is proposed. This work is the first of its kind in the literature to propose a consensus algorithm which relaxes the requirement of finite moments on the communication noise. It is shown that the nodes reach consensus asymptotically to a finite random variable whose expectation is the desired… ▽ More A distributed average consensus algorithm robust to a wide range of impulsive channel noise distributions is proposed. This work is the first of its kind in the literature to propose a consensus algorithm which relaxes the requirement of finite moments on the communication noise. It is shown that the nodes reach consensus asymptotically to a finite random variable whose expectation is the desired sample average of the initial observations with a variance that depends on the step size of the algorithm and the receiver nonlinear function. The asymptotic performance is characterized by deriving the asymptotic covariance matrix using results from stochastic approximation theory. Simulations corroborate our analytical findings and highlight the robustness of the proposed algorithm. △ Less

Submitted 16 August, 2014; originally announced August 2014.

Comments: 24 pages, 7 figures, Submitted to Transactions on Signal Processing, Apr 2014 (Submitted, currently in review)

Recovering Non-negative and Combined Sparse Representations

Authors: Karthikeyan Natesan Ramamurthy, Jayaraman J. Thiagarajan, Andreas Spanias

Abstract: The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we develop the paradigm of combined sparse representations, where only a part of the co… ▽ More The non-negative solution to an underdetermined linear system can be uniquely recovered sometimes, even without imposing any additional sparsity constraints. In this paper, we derive conditions under which a unique non-negative solution for such a system can exist, based on the theory of polytopes. Furthermore, we develop the paradigm of combined sparse representations, where only a part of the coefficient vector is constrained to be non-negative, and the rest is unconstrained (general). We analyze the recovery of the unique, sparsest solution, for combined representations, under three different cases of coefficient support knowledge: (a) the non-zero supports of non-negative and general coefficients are known, (b) the non-zero support of general coefficients alone is known, and (c) both the non-zero supports are unknown. For case (c), we propose the combined orthogonal matching pursuit algorithm for coefficient recovery and derive the deterministic sparsity threshold under which recovery of the unique, sparsest coefficient vector is possible. We quantify the order complexity of the algorithms, and examine their performance in exact and approximate recovery of coefficients under various conditions of noise. Furthermore, we also obtain their empirical phase transition characteristics. We show that the basis pursuit algorithm, with partial non-negative constraints, and the proposed greedy algorithm perform better in recovering the unique sparse representation when compared to their unconstrained counterparts. Finally, we demonstrate the utility of the proposed methods in recovering images corrupted by saturation noise. △ Less

Submitted 20 September, 2013; v1 submitted 12 March, 2013; originally announced March 2013.

Kernel Sparse Models for Automated Tumor Segmentation

Authors: Jayaraman J. Thiagarajan, Karthikeyan Natesan Ramamurthy, Deepta Rajan, Anup Puri, David Frakes, Andreas Spanias

Abstract: In this paper, we propose sparse coding-based approaches for segmentation of tumor regions from MR images. Sparse coding with data-adapted dictionaries has been successfully employed in several image recovery and vision problems. The proposed approaches obtain sparse codes for each pixel in brain magnetic resonance images considering their intensity values and location information. Since it is tri… ▽ More In this paper, we propose sparse coding-based approaches for segmentation of tumor regions from MR images. Sparse coding with data-adapted dictionaries has been successfully employed in several image recovery and vision problems. The proposed approaches obtain sparse codes for each pixel in brain magnetic resonance images considering their intensity values and location information. Since it is trivial to obtain pixel-wise sparse codes, and combining multiple features in the sparse coding setup is not straightforward, we propose to perform sparse coding in a high-dimensional feature space where non-linear similarities can be effectively modeled. We use the training data from expert-segmented images to obtain kernel dictionaries with the kernel K-lines clustering procedure. For a test image, sparse codes are computed with these kernel dictionaries, and they are used to identify the tumor regions. This approach is completely automated, and does not require user intervention to initialize the tumor regions in a test image. Furthermore, a low complexity segmentation approach based on kernel sparse codes, which allows the user to initialize the tumor region, is also presented. Results obtained with both the proposed approaches are validated against manual segmentation by an expert radiologist, and the proposed methods lead to accurate tumor identification. △ Less

Submitted 11 March, 2013; originally announced March 2013.

Multiple Kernel Sparse Representations for Supervised and Unsupervised Learning

Authors: Jayaraman J. Thiagarajan, Karthikeyan Natesan Ramamurthy, Andreas Spanias

Abstract: In complex visual recognition tasks it is typical to adopt multiple descriptors, that describe different aspects of the images, for obtaining an improved recognition performance. Descriptors that have diverse forms can be fused into a unified feature space in a principled manner using kernel methods. Sparse models that generalize well to the test data can be learned in the unified kernel space, an… ▽ More In complex visual recognition tasks it is typical to adopt multiple descriptors, that describe different aspects of the images, for obtaining an improved recognition performance. Descriptors that have diverse forms can be fused into a unified feature space in a principled manner using kernel methods. Sparse models that generalize well to the test data can be learned in the unified kernel space, and appropriate constraints can be incorporated for application in supervised and unsupervised learning. In this paper, we propose to perform sparse coding and dictionary learning in the multiple kernel space, where the weights of the ensemble kernel are tuned based on graph-embedding principles such that class discrimination is maximized. In our proposed algorithm, dictionaries are inferred using multiple levels of 1-D subspace clustering in the kernel space, and the sparse codes are obtained using a simple levelwise pursuit scheme. Empirical results for object recognition and image clustering show that our algorithm outperforms existing sparse coding based approaches, and compares favorably to other state-of-the-art methods. △ Less

Submitted 4 October, 2013; v1 submitted 3 March, 2013; originally announced March 2013.

Learning Stable Multilevel Dictionaries for Sparse Representations

Authors: Jayaraman J. Thiagarajan, Karthikeyan Natesan Ramamurthy, Andreas Spanias

Abstract: Sparse representations using learned dictionaries are being increasingly used with success in several data processing and machine learning applications. The availability of abundant training data necessitates the development of efficient, robust and provably good dictionary learning algorithms. Algorithmic stability and generalization are desirable characteristics for dictionary learning algorithm… ▽ More Sparse representations using learned dictionaries are being increasingly used with success in several data processing and machine learning applications. The availability of abundant training data necessitates the development of efficient, robust and provably good dictionary learning algorithms. Algorithmic stability and generalization are desirable characteristics for dictionary learning algorithms that aim to build global dictionaries which can efficiently model any test data similar to the training samples. In this paper, we propose an algorithm to learn dictionaries for sparse representations from large scale data, and prove that the proposed learning algorithm is stable and generalizable asymptotically. The algorithm employs a 1-D subspace clustering procedure, the K-hyperline clustering, in order to learn a hierarchical dictionary with multiple levels. We also propose an information-theoretic scheme to estimate the number of atoms needed in each level of learning and develop an ensemble approach to learn robust dictionaries. Using the proposed dictionaries, the sparse code for novel test data can be computed using a low-complexity pursuit procedure. We demonstrate the stability and generalization characteristics of the proposed algorithm using simulations. We also evaluate the utility of the multilevel dictionaries in compressed recovery and subspace learning applications. △ Less

Submitted 25 September, 2013; v1 submitted 2 March, 2013; originally announced March 2013.

Ensemble Sparse Models for Image Analysis

Authors: Karthikeyan Natesan Ramamurthy, Jayaraman J. Thiagarajan, Prasanna Sattigeri, Andreas Spanias

Abstract: Sparse representations with learned dictionaries have been successful in several image analysis applications. In this paper, we propose and analyze the framework of ensemble sparse models, and demonstrate their utility in image restoration and unsupervised clustering. The proposed ensemble model approximates the data as a linear combination of approximations from multiple \textit{weak} sparse mode… ▽ More Sparse representations with learned dictionaries have been successful in several image analysis applications. In this paper, we propose and analyze the framework of ensemble sparse models, and demonstrate their utility in image restoration and unsupervised clustering. The proposed ensemble model approximates the data as a linear combination of approximations from multiple \textit{weak} sparse models. Theoretical analysis of the ensemble model reveals that even in the worst-case, the ensemble can perform better than any of its constituent individual models. The dictionaries corresponding to the individual sparse models are obtained using either random example selection or boosted approaches. Boosted approaches learn one dictionary per round such that the dictionary learned in a particular round is optimized for the training examples having high reconstruction error in the previous round. Results with compressed recovery show that the ensemble representations lead to a better performance compared to using a single dictionary obtained with the conventional alternating minimization approach. The proposed ensemble models are also used for single image superresolution, and we show that they perform comparably to the recent approaches. In unsupervised clustering, experiments show that the proposed model performs better than baseline approaches in several standard datasets. △ Less

Submitted 27 February, 2013; originally announced February 2013.

Non-Linear Distributed Average Consensus using Bounded Transmissions

Authors: Sivaraman Dasarathan, Cihan Tepedelenlioglu, Mahesh Banavar, Andreas Spanias

Abstract: A distributed average consensus algorithm in which every sensor transmits with bounded peak power is proposed. In the presence of communication noise, it is shown that the nodes reach consensus asymptotically to a finite random variable whose expectation is the desired sample average of the initial observations with a variance that depends on the step size of the algorithm and the variance of the… ▽ More A distributed average consensus algorithm in which every sensor transmits with bounded peak power is proposed. In the presence of communication noise, it is shown that the nodes reach consensus asymptotically to a finite random variable whose expectation is the desired sample average of the initial observations with a variance that depends on the step size of the algorithm and the variance of the communication noise. The asymptotic performance is characterized by deriving the asymptotic covariance matrix using results from stochastic approximation theory. It is shown that using bounded transmissions results in slower convergence compared to the linear consensus algorithm based on the Laplacian heuristic. Simulations corroborate our analytical findings. △ Less

Submitted 21 February, 2013; originally announced February 2013.

Comments: 24 pages, 8 figures, Submitted to IEEE Transactions on Signal Processing

Distributed SNR Estimation using Constant Modulus Signaling over Gaussian Multiple-Access Channels

Authors: Mahesh K. Banavar, Cihan Tepedelenlioglu, Andreas Spanias

Abstract: A sensor network is used for distributed joint mean and variance estimation, in a single time snapshot. Sensors observe a signal embedded in noise, which are phase modulated using a constant-modulus scheme and transmitted over a Gaussian multiple-access channel to a fusion center, where the mean and variance are estimated jointly, using an asymptotically minimum-variance estimator, which is shown… ▽ More A sensor network is used for distributed joint mean and variance estimation, in a single time snapshot. Sensors observe a signal embedded in noise, which are phase modulated using a constant-modulus scheme and transmitted over a Gaussian multiple-access channel to a fusion center, where the mean and variance are estimated jointly, using an asymptotically minimum-variance estimator, which is shown to decouple into simple individual estimators of the mean and the variance. The constant-modulus phase modulation scheme ensures a fixed transmit power, robust estimation across several sensing noise distributions, as well as an SNR estimate that requires a single set of transmissions from the sensors to the fusion center, unlike the amplify-and-forward approach. The performance of the estimators of the mean and variance are evaluated in terms of asymptotic variance, which is used to evaluate the performance of the SNR estimator in the case of Gaussian, Laplace and Cauchy sensing noise distributions. For each sensing noise distribution, the optimal phase transmission parameters are also determined. The asymptotic relative efficiency of the mean and variance estimators is evaluated. It is shown that among the noise distributions considered, the estimators are asymptotically efficient only when the noise distribution is Gaussian. Simulation results corroborate analytical results. △ Less

Submitted 23 February, 2011; originally announced February 2011.

Comments: Submitted to the IEEE Transactions on Signal Processing. 18 pages, 8 figures

On Inequalities Relating the Characteristic Function and Fisher Information

Authors: Cihan Tepedelenlioglu, Mahesh K. Banavar, Andreas Spanias

Abstract: A relationship between the Fisher information and the characteristic function is established with the help of two inequalities. A necessary and sufficient condition for equality is found. These results are used to determine the asymptotic efficiency of a distributed estimation algorithm that uses constant modulus transmissions over Gaussian multiple access channels. The loss in efficiency of the d… ▽ More A relationship between the Fisher information and the characteristic function is established with the help of two inequalities. A necessary and sufficient condition for equality is found. These results are used to determine the asymptotic efficiency of a distributed estimation algorithm that uses constant modulus transmissions over Gaussian multiple access channels. The loss in efficiency of the distributed estimation scheme relative to the centralized approach is quantified for different sensing noise distributions. It is shown that the distributed estimator does not incur an efficiency loss if and only if the sensing noise distribution is Gaussian. △ Less

Submitted 8 July, 2010; originally announced July 2010.

Comments: Submitted to IEEE Transactions on Information Theory; 8 pages; 3 figures

Distributed Detection over Fading MACs with Multiple Antennas at the Fusion Center

Authors: Mahesh K. Banavar, Anthony D. Smith, Cihan Tepedelenlioglu, Andreas Spanias

Abstract: A distributed detection problem over fading Gaussian multiple-access channels is considered. Sensors observe a phenomenon and transmit their observations to a fusion center using the amplify and forward scheme. The fusion center has multiple antennas with different channel models considered between the sensors and the fusion center, and different cases of channel state information are assumed at t… ▽ More A distributed detection problem over fading Gaussian multiple-access channels is considered. Sensors observe a phenomenon and transmit their observations to a fusion center using the amplify and forward scheme. The fusion center has multiple antennas with different channel models considered between the sensors and the fusion center, and different cases of channel state information are assumed at the sensors. The performance is evaluated in terms of the error exponent for each of these cases, where the effect of multiple antennas at the fusion center is studied. It is shown that for zero-mean channels between the sensors and the fusion center when there is no channel information at the sensors, arbitrarily large gains in the error exponent can be obtained with sufficient increase in the number of antennas at the fusion center. In stark contrast, when there is channel information at the sensors, the gain in error exponent due to having multiple antennas at the fusion center is shown to be no more than a factor of (8/pi) for Rayleigh fading channels between the sensors and the fusion center, independent of the number of antennas at the fusion center, or correlation among noise samples across sensors. Scaling laws for such gains are also provided when both sensors and antennas are increased simultaneously. Simple practical schemes and a numerical method using semidefinite relaxation techniques are presented that utilize the limited possible gains available. Simulations are used to establish the accuracy of the results. △ Less

Submitted 2 September, 2010; v1 submitted 18 January, 2010; originally announced January 2010.

Comments: 21 pages, 9 figures, submitted to the IEEE Transactions on Signal Processing