-
Causal Inference on Process Graphs, Part II: Causal Structure and Effect Identification
Authors:
Nicolas-Domenic Reiter,
Jonas Wahl,
Andreas Gerhardus,
Jakob Runge
Abstract:
A structural vector autoregressive (SVAR) process is a linear causal model for variables that evolve over a discrete set of time points and between which there may be lagged and instantaneous effects. The qualitative causal structure of an SVAR process can be represented by its finite and directed process graph, in which a directed link connects two processes whenever there is a lagged or instanta…
▽ More
A structural vector autoregressive (SVAR) process is a linear causal model for variables that evolve over a discrete set of time points and between which there may be lagged and instantaneous effects. The qualitative causal structure of an SVAR process can be represented by its finite and directed process graph, in which a directed link connects two processes whenever there is a lagged or instantaneous effect between them. At the process graph level, the causal structure of SVAR processes is compactly parameterised in the frequency domain. In this paper, we consider the problem of causal discovery and causal effect estimation from the spectral density, the frequency domain analogue of the auto covariance, of the SVAR process. Causal discovery concerns the recovery of the process graph and causal effect estimation concerns the identification and estimation of causal effects in the frequency domain.
We show that information about the process graph, in terms of $d$- and $t$-separation statements, can be identified by verifying algebraic constraints on the spectral density. Furthermore, we introduce a notion of rational identifiability for frequency causal effects that may be confounded by exogenous latent processes, and show that the recent graphical latent factor half-trek criterion can be used on the process graph to assess whether a given (confounded) effect can be identified by rational operations on the entries of the spectral density.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Projecting infinite time series graphs to finite marginal graphs using number theory
Authors:
Andreas Gerhardus,
Jonas Wahl,
Sofia Faltenbacher,
Urmi Ninad,
Jakob Runge
Abstract:
In recent years, a growing number of method and application works have adapted and applied the causal-graphical-model framework to time series data. Many of these works employ time-resolved causal graphs that extend infinitely into the past and future and whose edges are repetitive in time, thereby reflecting the assumption of stationary causal relationships. However, most results and algorithms f…
▽ More
In recent years, a growing number of method and application works have adapted and applied the causal-graphical-model framework to time series data. Many of these works employ time-resolved causal graphs that extend infinitely into the past and future and whose edges are repetitive in time, thereby reflecting the assumption of stationary causal relationships. However, most results and algorithms from the causal-graphical-model framework are not designed for infinite graphs. In this work, we develop a method for projecting infinite time series graphs with repetitive edges to marginal graphical models on a finite time window. These finite marginal graphs provide the answers to $m$-separation queries with respect to the infinite graph, a task that was previously unresolved. Moreover, we argue that these marginal graphs are useful for causal discovery and causal effect estimation in time series, effectively enabling to apply results developed for finite graphs to the infinite graphs. The projection procedure relies on finding common ancestors in the to-be-projected graph and is, by itself, not new. However, the projection procedure has not yet been algorithmically implemented for time series graphs since in these infinite graphs there can be infinite sets of paths that might give rise to common ancestors. We solve the search over these possibly infinite sets of paths by an intriguing combination of path-finding techniques for finite directed graphs and solution theory for linear Diophantine equations. By providing an algorithm that carries out the projection, our paper makes an important step towards a theoretically-grounded and method-agnostic generalization of a range of causal inference methods and results to time series.
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
-
Bootstrap aggregation and confidence measures to improve time series causal discovery
Authors:
Kevin Debeire,
Jakob Runge,
Andreas Gerhardus,
Veronika Eyring
Abstract:
Learning causal graphs from multivariate time series is a ubiquitous challenge in all application domains dealing with time-dependent systems, such as in Earth sciences, biology, or engineering, to name a few. Recent developments for this causal discovery learning task have shown considerable skill, notably the specific time-series adaptations of the popular conditional independence-based learning…
▽ More
Learning causal graphs from multivariate time series is a ubiquitous challenge in all application domains dealing with time-dependent systems, such as in Earth sciences, biology, or engineering, to name a few. Recent developments for this causal discovery learning task have shown considerable skill, notably the specific time-series adaptations of the popular conditional independence-based learning framework. However, uncertainty estimation is challenging for conditional independence-based methods. Here, we introduce a novel bootstrap approach designed for time series causal discovery that preserves the temporal dependencies and lag structure. It can be combined with a range of time series causal discovery methods and provides a measure of confidence for the links of the time series graphs. Furthermore, next to confidence estimation, an aggregation, also called bagging, of the bootstrapped graphs by majority voting results in bagged causal discovery methods. In this work, we combine this approach with the state-of-the-art conditional-independence-based algorithm PCMCI+. With extensive numerical experiments we empirically demonstrate that, in addition to providing confidence measures for links, Bagged-PCMCI+ improves in precision and recall as compared to its base algorithm PCMCI+, at the cost of higher computational demands. These statistical performance improvements are especially pronounced in the more challenging settings (short time sample size, large number of variables, high autocorrelation). Our bootstrap approach can also be combined with other time series causal discovery algorithms and can be of considerable use in many real-world applications.
△ Less
Submitted 22 February, 2024; v1 submitted 15 June, 2023;
originally announced June 2023.
-
Discovering Causal Relations and Equations from Data
Authors:
Gustau Camps-Valls,
Andreas Gerhardus,
Urmi Ninad,
Gherardo Varando,
Georg Martius,
Emili Balaguer-Ballester,
Ricardo Vinuesa,
Emiliano Diaz,
Laure Zanna,
Jakob Runge
Abstract:
Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing t…
▽ More
Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems.
△ Less
Submitted 21 May, 2023;
originally announced May 2023.
-
Causal Inference on Process Graphs, Part I: The Structural Equation Process Representation
Authors:
Nicolas-Domenic Reiter,
Andreas Gerhardus,
Jonas Wahl,
Jakob Runge
Abstract:
When dealing with time series data, causal inference methods often employ structural vector autoregressive (SVAR) processes to model time-evolving random systems. In this work, we rephrase recursive SVAR processes with possible latent component processes as a linear Structural Causal Model (SCM) of stochastic processes on a simple causal graph, the \emph{process graph}, that models every process a…
▽ More
When dealing with time series data, causal inference methods often employ structural vector autoregressive (SVAR) processes to model time-evolving random systems. In this work, we rephrase recursive SVAR processes with possible latent component processes as a linear Structural Causal Model (SCM) of stochastic processes on a simple causal graph, the \emph{process graph}, that models every process as a single node. Using this reformulation, we generalise Wright's well-known path-rule for linear Gaussian SCMs to the newly introduced process SCMs and we express the auto-covariance sequence of an SVAR process by means of a generalised trek-rule. Employing the Fourier-Transformation, we derive compact expressions for causal effects in the frequency domain that allow us to efficiently visualise the causal interactions in a multivariate SVAR process. Finally, we observe that the process graph can be used to formulate graphical criteria for identifying causal effects and to derive algebraic relations with which these frequency domain causal effects can be recovered from the observed spectral density.
△ Less
Submitted 24 June, 2024; v1 submitted 19 May, 2023;
originally announced May 2023.
-
Selecting Robust Features for Machine Learning Applications using Multidata Causal Discovery
Authors:
Saranya Ganesh S.,
Tom Beucler,
Frederick Iat-Hin Tam,
Milton S. Gomez,
Jakob Runge,
Andreas Gerhardus
Abstract:
Robust feature selection is vital for creating reliable and interpretable Machine Learning (ML) models. When designing statistical prediction models in cases where domain knowledge is limited and underlying interactions are unknown, choosing the optimal set of features is often difficult. To mitigate this issue, we introduce a Multidata (M) causal feature selection approach that simultaneously pro…
▽ More
Robust feature selection is vital for creating reliable and interpretable Machine Learning (ML) models. When designing statistical prediction models in cases where domain knowledge is limited and underlying interactions are unknown, choosing the optimal set of features is often difficult. To mitigate this issue, we introduce a Multidata (M) causal feature selection approach that simultaneously processes an ensemble of time series datasets and produces a single set of causal drivers. This approach uses the causal discovery algorithms PC1 or PCMCI that are implemented in the Tigramite Python package. These algorithms utilize conditional independence tests to infer parts of the causal graph. Our causal feature selection approach filters out causally-spurious links before passing the remaining causal features as inputs to ML models (Multiple linear regression, Random Forest) that predict the targets. We apply our framework to the statistical intensity prediction of Western Pacific Tropical Cyclones (TC), for which it is often difficult to accurately choose drivers and their dimensionality reduction (time lags, vertical levels, and area-averaging). Using more stringent significance thresholds in the conditional independence tests helps eliminate spurious causal relationships, thus hel** the ML model generalize better to unseen TC cases. M-PC1 with a reduced number of features outperforms M-PCMCI, non-causal ML, and other feature selection methods (lagged correlation, random), even slightly outperforming feature selection based on eXplainable Artificial Intelligence. The optimal causal drivers obtained from our causal feature selection help improve our understanding of underlying relationships and suggest new potential drivers of TC intensification.
△ Less
Submitted 30 June, 2023; v1 submitted 11 April, 2023;
originally announced April 2023.
-
Causal inference for temporal patterns
Authors:
Nicolas-Domenic Reiter,
Andreas Gerhardus,
Jakob Runge
Abstract:
Complex dynamical systems are prevalent in many scientific disciplines. In the analysis of such systems two aspects are of particular interest: 1) the temporal patterns along which they evolve and 2) the underlying causal mechanisms. Time-series representations like discrete Fourier and wavelet transforms have been widely applied in order to obtain insights on the temporal structure of complex dyn…
▽ More
Complex dynamical systems are prevalent in many scientific disciplines. In the analysis of such systems two aspects are of particular interest: 1) the temporal patterns along which they evolve and 2) the underlying causal mechanisms. Time-series representations like discrete Fourier and wavelet transforms have been widely applied in order to obtain insights on the temporal structure of complex dynamical systems. Questions of cause and effect can be formalized in the causal inference framework. We propose an elementary and systematic approach to combine time-series representations with causal inference. Our method is based on a notion of causal effects from a cause on an effect process with respect to a pair of temporal patterns. In particular, our framework can be used to study causal effects in the frequency domain. We will see how our approach compares to the well known Granger Causality in the frequency domain. Furthermore, using a singular value decomposition we establish a representation of how one process drives another over a time-window of specified length in terms of temporal impulse-response patterns. To these we will refer to as Causal Orthogonal Functions (COF), a causal analogue of the temporal patterns derived with covariance-based multivariate Singular Spectrum Analysis (mSSA).
△ Less
Submitted 30 May, 2022;
originally announced May 2022.
-
Characterization of causal ancestral graphs for time series with latent confounders
Authors:
Andreas Gerhardus
Abstract:
In this paper, we introduce a novel class of graphical models for representing time lag specific causal relationships and independencies of multivariate time series with unobserved confounders. We completely characterize these graphs and show that they constitute proper subsets of the currently employed model classes. As we show, from the novel graphs one can thus draw stronger causal inferences -…
▽ More
In this paper, we introduce a novel class of graphical models for representing time lag specific causal relationships and independencies of multivariate time series with unobserved confounders. We completely characterize these graphs and show that they constitute proper subsets of the currently employed model classes. As we show, from the novel graphs one can thus draw stronger causal inferences -- without additional assumptions. We further introduce a graphical representation of Markov equivalence classes of the novel graphs. This graphical representation contains more causal knowledge than what current state-of-the-art causal discovery algorithms learn.
△ Less
Submitted 5 October, 2023; v1 submitted 15 December, 2021;
originally announced December 2021.
-
High-recall causal discovery for autocorrelated time series with latent confounders
Authors:
Andreas Gerhardus,
Jakob Runge
Abstract:
We present a new method for linear and nonlinear, lagged and contemporaneous constraint-based causal discovery from observational time series in the presence of latent confounders. We show that existing causal discovery methods such as FCI and variants suffer from low recall in the autocorrelated time series case and identify low effect size of conditional independence tests as the main reason. In…
▽ More
We present a new method for linear and nonlinear, lagged and contemporaneous constraint-based causal discovery from observational time series in the presence of latent confounders. We show that existing causal discovery methods such as FCI and variants suffer from low recall in the autocorrelated time series case and identify low effect size of conditional independence tests as the main reason. Information-theoretical arguments show that effect size can often be increased if causal parents are included in the conditioning sets. To identify parents early on, we suggest an iterative procedure that utilizes novel orientation rules to determine ancestral relationships already during the edge removal phase. We prove that the method is order-independent, and sound and complete in the oracle case. Extensive simulation studies for different numbers of variables, time lags, sample sizes, and further cases demonstrate that our method indeed achieves much higher recall than existing methods for the case of autocorrelated continuous variables while kee** false positives at the desired level. This performance gain grows with stronger autocorrelation. At https://github.com/jakobrunge/tigramite we provide Python code for all methods involved in the simulation studies.
△ Less
Submitted 1 February, 2021; v1 submitted 3 July, 2020;
originally announced July 2020.
