Uncertain Trees: Dealing with Uncertain Inputs in Regression Trees
Authors:
Myriam Tami,
Marianne Clausel,
Emilie Devijver,
Adrien Dulac,
Eric Gaussier,
Stefan Janaqi,
Meriam Chebre
Abstract:
Tree-based ensemble methods, as Random Forests and Gradient Boosted Trees, have been successfully used for regression in many applications and research studies. Furthermore, these methods have been extended in order to deal with uncertainty in the output variable, using for example a quantile loss in Random Forests (Meinshausen, 2006). To the best of our knowledge, no extension has been provided y…
▽ More
Tree-based ensemble methods, as Random Forests and Gradient Boosted Trees, have been successfully used for regression in many applications and research studies. Furthermore, these methods have been extended in order to deal with uncertainty in the output variable, using for example a quantile loss in Random Forests (Meinshausen, 2006). To the best of our knowledge, no extension has been provided yet for dealing with uncertainties in the input variables, even though such uncertainties are common in practical situations. We propose here such an extension by showing how standard regression trees optimizing a quadratic loss can be adapted and learned while taking into account the uncertainties in the inputs. By doing so, one no longer assumes that an observation lies into a single region of the regression tree, but rather that it belongs to each region with a certain probability. Experiments conducted on several data sets illustrate the good behavior of the proposed extension.
△ Less
Submitted 18 November, 2018; v1 submitted 27 October, 2018;
originally announced October 2018.
Sparse hierarchical interaction learning with epigraphical projection
Authors:
Mingyuan Jiu,
Nelly Pustelnik,
Stefan Janaqi,
Mériam Chebre,
Lin Qi,
Philippe Ricoux
Abstract:
This work focuses on learning optimization problems with quadratical interactions between variables, which go beyond the additive models of traditional linear learning. We investigate more specifically two different methods encountered in the literature to deal with this problem: "hierNet" and structured-sparsity regularization, and study their connections. We propose a primal-dual proximal algori…
▽ More
This work focuses on learning optimization problems with quadratical interactions between variables, which go beyond the additive models of traditional linear learning. We investigate more specifically two different methods encountered in the literature to deal with this problem: "hierNet" and structured-sparsity regularization, and study their connections. We propose a primal-dual proximal algorithm based on an epigraphical projection to optimize a general formulation of these learning problems. The experimental setting first highlights the improvement of the proposed procedure compared to state-of-the-art methods based on fast iterative shrinkage-thresholding algorithm (i.e. FISTA) or alternating direction method of multipliers (i.e. ADMM), and then, using the proposed flexible optimization framework, we provide fair comparisons between the different hierarchical penalizations and their improvement over the standard $\ell_1$-norm penalization. The experiments are conducted both on synthetic and real data, and they clearly show that the proposed primal-dual proximal algorithm based on epigraphical projection is efficient and effective to solve and investigate the problem of hierarchical interaction learning.
△ Less
Submitted 9 February, 2021; v1 submitted 22 May, 2017;
originally announced May 2017.
