-
Kernel Methods and their derivatives: Concept and perspectives for the Earth system sciences
Authors:
J. Emmanuel Johnson,
Valero Laparra,
Adrián Pérez-Suay,
Miguel D. Mahecha,
Gustau Camps-Valls
Abstract:
Kernel methods are powerful machine learning techniques which implement generic non-linear functions to solve complex tasks in a simple way. They Have a solid mathematical background and exhibit excellent performance in practice. However, kernel machines are still considered black-box models as the feature map** is not directly accessible and difficult to interpret.The aim of this work is to sho…
▽ More
Kernel methods are powerful machine learning techniques which implement generic non-linear functions to solve complex tasks in a simple way. They Have a solid mathematical background and exhibit excellent performance in practice. However, kernel machines are still considered black-box models as the feature map** is not directly accessible and difficult to interpret.The aim of this work is to show that it is indeed possible to interpret the functions learned by various kernel methods is intuitive despite their complexity. Specifically, we show that derivatives of these functions have a simple mathematical formulation, are easy to compute, and can be applied to many different problems. We note that model function derivatives in kernel machines is proportional to the kernel function derivative. We provide the explicit analytic form of the first and second derivatives of the most common kernel functions with regard to the inputs as well as generic formulas to compute higher order derivatives. We use them to analyze the most used supervised and unsupervised kernel learning methods: Gaussian Processes for regression, Support Vector Machines for classification, Kernel Entropy Component Analysis for density estimation, and the Hilbert-Schmidt Independence Criterion for estimating the dependency between random variables. For all cases we expressed the derivative of the learned function as a linear combination of the kernel function derivative. Moreover we provide intuitive explanations through illustrative toy examples and show how to improve the interpretation of real applications in the context of spatiotemporal Earth system data cubes. This work reflects on the observation that function derivatives may play a crucial role in kernel methods analysis and understanding.
△ Less
Submitted 5 October, 2020; v1 submitted 29 July, 2020;
originally announced July 2020.
-
Towards Causal Inference for Spatio-Temporal Data: Conflict and Forest Loss in Colombia
Authors:
Rune Christiansen,
Matthias Baumann,
Tobias Kuemmerle,
Miguel D. Mahecha,
Jonas Peters
Abstract:
In many data scientific problems, we are interested in inferring causal relationships in the data generating mechanism. Here, we consider the following real-world question: how has the Colombian conflict influenced tropical forest loss? There is evidence for both enhancing and reducing impacts. Answering such questions requires the use of causal models. In this work, we propose a class of causal m…
▽ More
In many data scientific problems, we are interested in inferring causal relationships in the data generating mechanism. Here, we consider the following real-world question: how has the Colombian conflict influenced tropical forest loss? There is evidence for both enhancing and reducing impacts. Answering such questions requires the use of causal models. In this work, we propose a class of causal models for spatio-temporal stochastic processes. It allows us to formally define and quantify the causal effect of a vector of covariates $X$ on a real-valued response $Y$, even if the causal background knowledge is incomplete. We introduce a procedure for estimating causal effects, and a non-parametric hypothesis test for these effects being zero. The proposed methods do not make strong distributional assumptions, and allow for arbitrarily many latent confounders, given that these confounders do not vary across time (or, alternatively, they do not vary across space). When applying our causal methodology to the problem of conflict and forest loss, using data from 2000 to 2018, we find a reducing but insignificant causal effect of conflict on forest loss. Regionally, both enhancing and reducing effects can be identified. Our theoretical findings are supported by simulations, and code is available online.
△ Less
Submitted 29 August, 2022; v1 submitted 18 May, 2020;
originally announced May 2020.
-
Maximally Divergent Intervals for Anomaly Detection
Authors:
Erik Rodner,
Björn Barz,
Yanira Guanche,
Milan Flach,
Miguel Mahecha,
Paul Bodesheim,
Markus Reichstein,
Joachim Denzler
Abstract:
We present new methods for batch anomaly detection in multivariate time series. Our methods are based on maximizing the Kullback-Leibler divergence between the data distribution within and outside an interval of the time series. An empirical analysis shows the benefits of our algorithms compared to methods that treat each time step independently from each other without optimizing with respect to a…
▽ More
We present new methods for batch anomaly detection in multivariate time series. Our methods are based on maximizing the Kullback-Leibler divergence between the data distribution within and outside an interval of the time series. An empirical analysis shows the benefits of our algorithms compared to methods that treat each time step independently from each other without optimizing with respect to all possible intervals.
△ Less
Submitted 21 October, 2016;
originally announced October 2016.
