-
On Counterfactual Interventions in Vector Autoregressive Models
Authors:
Kurt Butler,
Marija Iloska,
Petar M. Djuric
Abstract:
Counterfactual reasoning allows us to explore hypothetical scenarios in order to explain the impacts of our decisions. However, addressing such inquires is impossible without establishing the appropriate mathematical framework. In this work, we introduce the problem of counterfactual reasoning in the context of vector autoregressive (VAR) processes. We also formulate the inference of a causal mode…
▽ More
Counterfactual reasoning allows us to explore hypothetical scenarios in order to explain the impacts of our decisions. However, addressing such inquires is impossible without establishing the appropriate mathematical framework. In this work, we introduce the problem of counterfactual reasoning in the context of vector autoregressive (VAR) processes. We also formulate the inference of a causal model as a joint regression task where for inference we use both data with and without interventions. After learning the model, we exploit linearity of the VAR model to make exact predictions about the effects of counterfactual interventions. Furthermore, we quantify the total causal effects of past counterfactual interventions. The source code for this project is freely available at https://github.com/KurtButler/counterfactual_interventions.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
A Gaussian Process-based Streaming Algorithm for Prediction of Time Series With Regimes and Outliers
Authors:
Daniel Waxman,
Petar M. Djurić
Abstract:
Online prediction of time series under regime switching is a widely studied problem in the literature, with many celebrated approaches. Using the non-parametric flexibility of Gaussian processes, the recently proposed INTEL algorithm provides a product of experts approach to online prediction of time series under possible regime switching, including the special case of outliers. This is achieved b…
▽ More
Online prediction of time series under regime switching is a widely studied problem in the literature, with many celebrated approaches. Using the non-parametric flexibility of Gaussian processes, the recently proposed INTEL algorithm provides a product of experts approach to online prediction of time series under possible regime switching, including the special case of outliers. This is achieved by adaptively combining several candidate models, each reporting their predictive distribution at time $t$. However, the INTEL algorithm uses a finite context window approximation to the predictive distribution, the computation of which scales cubically with the maximum lag, or otherwise scales quartically with exact predictive distributions. We introduce LINTEL, which uses the exact filtering distribution at time $t$ with constant-time updates, making the time complexity of the streaming algorithm optimal. We additionally note that the weighting mechanism of INTEL is better suited to a mixture of experts approach, and propose a fusion policy based on arithmetic averaging for LINTEL. We show experimentally that our proposed approach is over five times faster than INTEL under reasonable settings with better quality predictions.
△ Less
Submitted 1 June, 2024;
originally announced June 2024.
-
Dynamic Online Ensembles of Basis Expansions
Authors:
Daniel Waxman,
Petar M. Djurić
Abstract:
Practical Bayesian learning often requires (1) online inference, (2) dynamic models, and (3) ensembling over multiple different models. Recent advances have shown how to use random feature approximations to achieve scalable, online ensembling of Gaussian processes with desirable theoretical properties and fruitful applications. One key to these methods' success is the inclusion of a random walk on…
▽ More
Practical Bayesian learning often requires (1) online inference, (2) dynamic models, and (3) ensembling over multiple different models. Recent advances have shown how to use random feature approximations to achieve scalable, online ensembling of Gaussian processes with desirable theoretical properties and fruitful applications. One key to these methods' success is the inclusion of a random walk on the model parameters, which makes models dynamic. We show that these methods can be generalized easily to any basis expansion model and that using alternative basis expansions, such as Hilbert space Gaussian processes, often results in better performance. To simplify the process of choosing a specific basis expansion, our method's generality also allows the ensembling of several entirely different models, for example, a Gaussian process and polynomial regression. Finally, we propose a novel method to ensemble static and dynamic models together.
△ Less
Submitted 2 May, 2024;
originally announced May 2024.
-
Explainable Learning with Gaussian Processes
Authors:
Kurt Butler,
Guanchao Feng,
Petar M. Djuric
Abstract:
The field of explainable artificial intelligence (XAI) attempts to develop methods that provide insight into how complicated machine learning methods make predictions. Many methods of explanation have focused on the concept of feature attribution, a decomposition of the model's prediction into individual contributions corresponding to each input feature. In this work, we explore the problem of fea…
▽ More
The field of explainable artificial intelligence (XAI) attempts to develop methods that provide insight into how complicated machine learning methods make predictions. Many methods of explanation have focused on the concept of feature attribution, a decomposition of the model's prediction into individual contributions corresponding to each input feature. In this work, we explore the problem of feature attribution in the context of Gaussian process regression (GPR). We take a principled approach to defining attributions under model uncertainty, extending the existing literature. We show that although GPR is a highly flexible and non-parametric approach, we can derive interpretable, closed-form expressions for the feature attributions. When using integrated gradients as an attribution method, we show that the attributions of a GPR model also follow a Gaussian process distribution, which quantifies the uncertainty in attribution arising from uncertainty in the model. We demonstrate, both through theory and experimentation, the versatility and robustness of this approach. We also show that, when applicable, the exact expressions for GPR attributions are both more accurate and less computationally expensive than the approximations currently used in practice. The source code for this project is freely available under MIT license at https://github.com/KurtButler/2024_attributions_paper.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
Fusion of Gaussian Processes Predictions with Monte Carlo Sampling
Authors:
Marzieh Ajirak,
Daniel Waxman,
Fernando Llorente,
Petar M. Djuric
Abstract:
In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and integrate their outcomes. In this paper, we operate within the Bayesian paradigm, relying on Gaussian processes as our models. These models generate predictive…
▽ More
In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and integrate their outcomes. In this paper, we operate within the Bayesian paradigm, relying on Gaussian processes as our models. These models generate predictive probability density functions (pdfs), and the objective is to integrate them systematically, employing both linear and log-linear pooling. We introduce novel approaches for log-linear pooling, determining input-dependent weights for the predictive pdfs of the Gaussian processes. The aggregation of the pdfs is realized through Monte Carlo sampling, drawing samples of weights from their posterior. The performance of these methods, as well as those based on linear pooling, is demonstrated using a synthetic dataset.
△ Less
Submitted 2 March, 2024;
originally announced March 2024.
-
Dagma-DCE: Interpretable, Non-Parametric Differentiable Causal Discovery
Authors:
Daniel Waxman,
Kurt Butler,
Petar M. Djuric
Abstract:
We introduce Dagma-DCE, an interpretable and model-agnostic scheme for differentiable causal discovery. Current non- or over-parametric methods in differentiable causal discovery use opaque proxies of ``independence'' to justify the inclusion or exclusion of a causal relationship. We show theoretically and empirically that these proxies may be arbitrarily different than the actual causal strength.…
▽ More
We introduce Dagma-DCE, an interpretable and model-agnostic scheme for differentiable causal discovery. Current non- or over-parametric methods in differentiable causal discovery use opaque proxies of ``independence'' to justify the inclusion or exclusion of a causal relationship. We show theoretically and empirically that these proxies may be arbitrarily different than the actual causal strength. Juxtaposed to existing differentiable causal discovery algorithms, \textsc{Dagma-DCE} uses an interpretable measure of causal strength to define weighted adjacency matrices. In a number of simulated datasets, we show our method achieves state-of-the-art level performance. We additionally show that \textsc{Dagma-DCE} allows for principled thresholding and sparsity penalties by domain-experts. The code for our method is available open-source at https://github.com/DanWaxman/DAGMA-DCE, and can easily be adapted to arbitrary differentiable models.
△ Less
Submitted 5 January, 2024;
originally announced January 2024.
-
Fusion of Probability Density Functions
Authors:
Günther Koliander,
Yousef El-Laham,
Petar M. Djurić,
Franz Hlawatsch
Abstract:
Fusing probabilistic information is a fundamental task in signal and data processing with relevance to many fields of technology and science. In this work, we investigate the fusion of multiple probability density functions (pdfs) of a continuous random variable or vector. Although the case of continuous random variables and the problem of pdf fusion frequently arise in multisensor signal processi…
▽ More
Fusing probabilistic information is a fundamental task in signal and data processing with relevance to many fields of technology and science. In this work, we investigate the fusion of multiple probability density functions (pdfs) of a continuous random variable or vector. Although the case of continuous random variables and the problem of pdf fusion frequently arise in multisensor signal processing, statistical inference, and machine learning, a universally accepted method for pdf fusion does not exist. The diversity of approaches, perspectives, and solutions related to pdf fusion motivates a unified presentation of the theory and methodology of the field. We discuss three different approaches to fusing pdfs. In the axiomatic approach, the fusion rule is defined indirectly by a set of properties (axioms). In the optimization approach, it is the result of minimizing an objective function that involves an information-theoretic divergence or a distance measure. In the supra-Bayesian approach, the fusion center interprets the pdfs to be fused as random observations. Our work is partly a survey, reviewing in a structured and coherent fashion many of the concepts and methods that have been developed in the literature. In addition, we present new results for each of the three approaches. Our original contributions include new fusion rules, axioms, and axiomatic and optimization-based characterizations; a new formulation of supra-Bayesian fusion in terms of finite-dimensional parametrizations; and a study of supra-Bayesian fusion of posterior pdfs for linear Gaussian models.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Networks of UAVs of Low-Complexity for Time-Critical Localization
Authors:
Anna Guerra,
Francesco Guidi,
Davide Dardari,
Petar M. Djuric
Abstract:
Future networks of unmanned aerial vehicles (UAVs) will be tasked to carry out ever-increasing complex operations that are time-critical and that require accurate localization performance (e.g., tracking the state of a malicious user). Since there is the need to preserve low UAV complexity while tackling the challenging goals of missions in effective ways, one key aspect is the UAV intelligence (U…
▽ More
Future networks of unmanned aerial vehicles (UAVs) will be tasked to carry out ever-increasing complex operations that are time-critical and that require accurate localization performance (e.g., tracking the state of a malicious user). Since there is the need to preserve low UAV complexity while tackling the challenging goals of missions in effective ways, one key aspect is the UAV intelligence (UAV-I). The UAV's intelligence includes the UAV's capability to process information and to make decisions, e.g., to decide where to sense and whether to delegate some tasks to other network entities. In this paper, we provide an overview of possible solutions for the design of UAVs of low complexity, showing some of the needs of the UAVs for running efficient localization operations, performed either as a team or individually. Further, we focus on different network configurations, which possibly include assistance with edge computing. We also discuss open problems and future perspectives for these settings.
△ Less
Submitted 7 August, 2021;
originally announced August 2021.
-
Near-field Tracking with Large Antenna Arrays: Fundamental Limits and Practical Algorithms
Authors:
Anna Guerra,
Francesco Guidi,
Davide Dardari,
Petar M. Djuric
Abstract:
Applications towards 6G have brought a huge interest towards arrays with a high number of antennas and operating within the millimeter and sub-THz bandwidths for joint communication and localization. With such large arrays, the plane wave approximation is often not accurate because the system may operate in the near-field propagation region (Fresnel region) where the electromagnetic field wavefron…
▽ More
Applications towards 6G have brought a huge interest towards arrays with a high number of antennas and operating within the millimeter and sub-THz bandwidths for joint communication and localization. With such large arrays, the plane wave approximation is often not accurate because the system may operate in the near-field propagation region (Fresnel region) where the electromagnetic field wavefront is spherical. In this case, the curvature of arrival (CoA) is a measure of the spherical wavefront that can be used to infer the source position using only a single large array. In this paper, we study a near-field tracking problem for inferring the state (i.e., the position and velocity) of a moving source with an ad-hoc observation model that accounts for the phase profile of a large receiving array. For this tracking problem, we derive the posterior Cramér-Rao Lower Bound (P-CRLB) and show the effects when the source moves inside and outside the Fresnel region. We provide insights on how the loss of positioning information outside Fresnel comes from an increase of the ranging error rather than from inaccuracies of angular estimation. Then, we investigate the performance of different Bayesian tracking algorithms in the presence of model mismatches and abrupt trajectory changes. Our results demonstrate the feasibility and high accuracy for most of the tracking approaches without the need of wideband signals and of any synchronization scheme. signals and of any synchronization scheme.
△ Less
Submitted 11 February, 2021;
originally announced February 2021.
-
Reinforcement Learning for UAV Autonomous Navigation, Map** and Target Detection
Authors:
Anna Guerra,
Francesco Guidi,
Davide Dardari,
Petar M. Djuric
Abstract:
In this paper, we study a joint detection, map** and navigation problem for a single unmanned aerial vehicle (UAV) equipped with a low complexity radar and flying in an unknown environment. The goal is to optimize its trajectory with the purpose of maximizing the map** accuracy and, at the same time, to avoid areas where measurements might not be sufficiently informative from the perspective o…
▽ More
In this paper, we study a joint detection, map** and navigation problem for a single unmanned aerial vehicle (UAV) equipped with a low complexity radar and flying in an unknown environment. The goal is to optimize its trajectory with the purpose of maximizing the map** accuracy and, at the same time, to avoid areas where measurements might not be sufficiently informative from the perspective of a target detection. This problem is formulated as a Markov decision process (MDP) where the UAV is an agent that runs either a state estimator for target detection and for environment map**, and a reinforcement learning (RL) algorithm to infer its own policy of navigation (i.e., the control law). Numerical results show the feasibility of the proposed idea, highlighting the UAV's capability of autonomously exploring areas with high probability of target detection while reconstructing the surrounding environment.
△ Less
Submitted 5 May, 2020;
originally announced May 2020.
-
Dynamic Radar Network of UAVs: A Joint Navigation and Tracking Approach
Authors:
Anna Guerra,
Davide Dardari,
Petar M. Djuric
Abstract:
Nowadays there is a growing research interest on the possibility of enriching small flying robots with autonomous sensing and online navigation capabilities. This will enable a large number of applications spanning from remote surveillance to logistics, smarter cities and emergency aid in hazardous environments. In this context, an emerging problem is to track unauthorized small unmanned aerial ve…
▽ More
Nowadays there is a growing research interest on the possibility of enriching small flying robots with autonomous sensing and online navigation capabilities. This will enable a large number of applications spanning from remote surveillance to logistics, smarter cities and emergency aid in hazardous environments. In this context, an emerging problem is to track unauthorized small unmanned aerial vehicles (UAVs) hiding behind buildings or concealing in large UAV networks. In contrast with current solutions mainly based on static and on-ground radars, this paper proposes the idea of a dynamic radar network of UAVs for real-time and high-accuracy tracking of malicious targets. To this end, we describe a solution for real-time navigation of UAVs to track a dynamic target using heterogeneously sensed information. Such information is shared by the UAVs with their neighbors via multi-hops, allowing tracking the target by a local Bayesian estimator running at each agent. Since not all the paths are equal in terms of information gathering point-of-view, the UAVs plan their own trajectory by minimizing the posterior covariance matrix of the target state under UAV kinematic and anti-collision constraints. Our results show how a dynamic network of radars attains better localization results compared to a fixed configuration and how the on-board sensor technology impacts the accuracy in tracking a target with different radar cross sections, especially in non line-of-sight (NLOS) situations.
△ Less
Submitted 13 January, 2020;
originally announced January 2020.
-
Joint Indoor Localization and Navigation of UAVs for Network Formation Control
Authors:
Anna Guerra,
Davide Dardari,
Petar M. Djuric
Abstract:
In this paper, we propose a joint indoor localization and navigation algorithm to enable a swarm of unmanned aerial vehicles (UAVs) to deploy in a specific spatial formation in indoor environments. In the envisioned scenario, we consider a static user acting as a central unit whose main task is to acquire all the UAV measurements carrying position-dependent information and to estimate the UAV posi…
▽ More
In this paper, we propose a joint indoor localization and navigation algorithm to enable a swarm of unmanned aerial vehicles (UAVs) to deploy in a specific spatial formation in indoor environments. In the envisioned scenario, we consider a static user acting as a central unit whose main task is to acquire all the UAV measurements carrying position-dependent information and to estimate the UAV positions when there is no existing infrastructure for positioning. Subsequently, the user exploits the estimated positions as inputs for the navigation control with the aim of deploying the UAVs in a desired formation in space (formation sha**). The user plans the trajectory of each UAV in real time, guaranteeing a safe navigation in the presence of obstacles. The proposed algorithm guides the UAVs to their desired final locations with good accuracy.
△ Less
Submitted 28 October, 2019;
originally announced October 2019.
-
Communications with decode-and-forward relays in mesh networks
Authors:
Angel Bravo-Santos,
Petar M. Djuric
Abstract:
We consider mesh networks composed of groups of relaying nodes which operate in decode-and-forward mode, where each node from a group relays information to all the nodes in the next group. We study these networks in two setups, one where the nodes have complete channel state information from the nodes that transmit to them, and another when they only have the statistics of the channel. We derive r…
▽ More
We consider mesh networks composed of groups of relaying nodes which operate in decode-and-forward mode, where each node from a group relays information to all the nodes in the next group. We study these networks in two setups, one where the nodes have complete channel state information from the nodes that transmit to them, and another when they only have the statistics of the channel. We derive recursive expressions for the probabilities of errors of the nodes and present several implementations of detectors used in these networks. We compare the mesh networks with multihop networks, the latter being formed by a set of parallel sections of multiple relaying nodes. We demonstrate with numerous simulations that there are significant improvements in performance of mesh over multihop networks in various scenarios.
△ Less
Submitted 16 October, 2012;
originally announced October 2012.
-
Likelihood Consensus-Based Distributed Particle Filtering with Distributed Proposal Density Adaptation
Authors:
Ondrej Hlinka,
Franz Hlawatsch,
Petar M. Djuric
Abstract:
We present a consensus-based distributed particle filter (PF) for wireless sensor networks. Each sensor runs a local PF to compute a global state estimate that takes into account the measurements of all sensors. The local PFs use the joint (all-sensors) likelihood function, which is calculated in a distributed way by a novel generalization of the likelihood consensus scheme. A performance improvem…
▽ More
We present a consensus-based distributed particle filter (PF) for wireless sensor networks. Each sensor runs a local PF to compute a global state estimate that takes into account the measurements of all sensors. The local PFs use the joint (all-sensors) likelihood function, which is calculated in a distributed way by a novel generalization of the likelihood consensus scheme. A performance improvement (or a reduction of the required number of particles) is achieved by a novel distributed, consensus-based method for adapting the proposal densities of the local PFs. The performance of the proposed distributed PF is demonstrated for a target tracking problem.
△ Less
Submitted 28 September, 2011;
originally announced September 2011.
-
Likelihood Consensus and Its Application to Distributed Particle Filtering
Authors:
Ondrej Hlinka,
Ondrej Sluciak,
Franz Hlawatsch,
Petar M. Djuric,
Markus Rupp
Abstract:
We consider distributed state estimation in a wireless sensor network without a fusion center. Each sensor performs a global estimation task---based on the past and current measurements of all sensors---using only local processing and local communications with its neighbors. In this estimation task, the joint (all-sensors) likelihood function (JLF) plays a central role as it epitomizes the measure…
▽ More
We consider distributed state estimation in a wireless sensor network without a fusion center. Each sensor performs a global estimation task---based on the past and current measurements of all sensors---using only local processing and local communications with its neighbors. In this estimation task, the joint (all-sensors) likelihood function (JLF) plays a central role as it epitomizes the measurements of all sensors. We propose a distributed method for computing, at each sensor, an approximation of the JLF by means of consensus algorithms. This "likelihood consensus" method is applicable if the local likelihood functions of the various sensors (viewed as conditional probability density functions of the local measurements) belong to the exponential family of distributions. We then use the likelihood consensus method to implement a distributed particle filter and a distributed Gaussian particle filter. Each sensor runs a local particle filter, or a local Gaussian particle filter, that computes a global state estimate. The weight update in each local (Gaussian) particle filter employs the JLF, which is obtained through the likelihood consensus scheme. For the distributed Gaussian particle filter, the number of particles can be significantly reduced by means of an additional consensus scheme. Simulation results are presented to assess the performance of the proposed distributed particle filters for a multiple target tracking problem.
△ Less
Submitted 1 August, 2012; v1 submitted 31 August, 2011;
originally announced August 2011.
