Showing 1–27 of 27 results for author: Verbeke, W

Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimation

Authors: Christopher Bockel-Rickermann, Toon Vanderschueren, Tim Verdonck, Wouter Verbeke

Abstract: Estimating conditional average dose responses (CADR) is an important but challenging problem. Estimators must correctly model the potentially complex relationships between covariates, interventions, doses, and outcomes. In recent years, the machine learning community has shown great interest in develo** tailored CADR estimators that target specific challenges. Their performance is typically eval… ▽ More Estimating conditional average dose responses (CADR) is an important but challenging problem. Estimators must correctly model the potentially complex relationships between covariates, interventions, doses, and outcomes. In recent years, the machine learning community has shown great interest in develo** tailored CADR estimators that target specific challenges. Their performance is typically evaluated against other methods on (semi-) synthetic benchmark datasets. Our paper analyses this practice and shows that using popular benchmark datasets without further analysis is insufficient to judge model performance. Established benchmarks entail multiple challenges, whose impacts must be disentangled. Therefore, we propose a novel decomposition scheme that allows the evaluation of the impact of five distinct components contributing to CADR estimator performance. We apply this scheme to eight popular CADR estimators on four widely-used benchmark datasets, running nearly 1,500 individual experiments. Our results reveal that most established benchmarks are challenging for reasons different from their creators' claims. Notably, confounding, the key challenge tackled by most estimators, is not an issue in any of the considered datasets. We discuss the major implications of our findings and present directions for future research. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 25 pages, 9 figures

MSC Class: 62D20

Network Analytics for Anti-Money Laundering -- A Systematic Literature Review and Experimental Evaluation

Authors: Bruno Deprez, Toon Vanderschueren, Bart Baesens, Tim Verdonck, Wouter Verbeke

Abstract: Money laundering presents a pervasive challenge, burdening society by financing illegal activities. To more effectively combat and detect money laundering, the use of network information is increasingly being explored, exploiting that money laundering necessarily involves interconnected parties. This has lead to a surge in literature on network analytics (NA) for anti-money laundering (AML). The l… ▽ More Money laundering presents a pervasive challenge, burdening society by financing illegal activities. To more effectively combat and detect money laundering, the use of network information is increasingly being explored, exploiting that money laundering necessarily involves interconnected parties. This has lead to a surge in literature on network analytics (NA) for anti-money laundering (AML). The literature, however, is fragmented and a comprehensive overview of existing work is missing. This results in limited understanding of the methods that may be applied and their comparative detection power. Therefore, this paper presents an extensive and systematic review of the literature. We identify and analyse 97 papers in the Web of Science and Scopus databases, resulting in a taxonomy of approaches following the fraud analytics framework of Bockel-Rickermann et al.. Moreover, this paper presents a comprehensive experimental framework to evaluate and compare the performance of prominent NA methods in a uniform setup. The framework is applied on the publicly available Elliptic data set and implements manual feature engineering, random walk-based methods, and deep learning GNNs. We conclude from the results that network analytics increases the predictive power of the AML model with graph neural networks giving the best results. An open source implementation of the experimental framework is provided to facilitate researchers and practitioners to extend upon these results and experiment on proprietary data. As such, we aim to promote a standardised approach towards the analysis and evaluation of network analytics for AML. △ Less

Submitted 31 May, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Metalearners for Ranking Treatment Effects

Authors: Toon Vanderschueren, Wouter Verbeke, Felipe Moraes, Hugo Manuel Proença

Abstract: Efficiently allocating treatments with a budget constraint constitutes an important challenge across various domains. In marketing, for example, the use of promotions to target potential customers and boost conversions is limited by the available budget. While much research focuses on estimating causal effects, there is relatively limited work on learning to allocate treatments while considering t… ▽ More Efficiently allocating treatments with a budget constraint constitutes an important challenge across various domains. In marketing, for example, the use of promotions to target potential customers and boost conversions is limited by the available budget. While much research focuses on estimating causal effects, there is relatively limited work on learning to allocate treatments while considering the operational context. Existing methods for uplift modeling or causal inference primarily estimate treatment effects, without considering how this relates to a profit maximizing allocation policy that respects budget constraints. The potential downside of using these methods is that the resulting predictive model is not aligned with the operational context. Therefore, prediction errors are propagated to the optimization of the budget allocation problem, subsequently leading to a suboptimal allocation policy. We propose an alternative approach based on learning to rank. Our proposed methodology directly learns an allocation policy by prioritizing instances in terms of their incremental profit. We propose an efficient sampling procedure for the optimization of the ranking model to scale our methodology to large-scale data sets. Theoretically, we show how learning to rank can maximize the area under a policy's incremental profit curve. Empirically, we validate our methodology and show its effectiveness in practice through a series of experiments on both synthetic and real-world data. △ Less

Submitted 3 May, 2024; originally announced May 2024.

Uplift vs. predictive modeling: a theoretical analysis

Authors: Théo Verhelst, Robin Petit, Wouter Verbeke, Gianluca Bontempi

Abstract: Despite the growing popularity of machine-learning techniques in decision-making, the added value of causal-oriented strategies with respect to pure machine-learning approaches has rarely been quantified in the literature. These strategies are crucial for practitioners in various domains, such as marketing, telecommunications, health care and finance. This paper presents a comprehensive treatment… ▽ More Despite the growing popularity of machine-learning techniques in decision-making, the added value of causal-oriented strategies with respect to pure machine-learning approaches has rarely been quantified in the literature. These strategies are crucial for practitioners in various domains, such as marketing, telecommunications, health care and finance. This paper presents a comprehensive treatment of the subject, starting from firm theoretical foundations and highlighting the parameters that influence the performance of the uplift and predictive approaches. The focus of the paper is on a binary outcome case and a binary action, and the paper presents a theoretical analysis of uplift modeling, comparing it with the classical predictive approach. The main research contributions of the paper include a new formulation of the measure of profit, a formal proof of the convergence of the uplift curve to the measure of profit ,and an illustration, through simulations, of the conditions under which predictive approaches still outperform uplift modeling. We show that the mutual information between the features and the outcome plays a significant role, along with the variance of the estimators, the distribution of the potential outcomes and the underlying costs and benefits of the treatment and the outcome. △ Less

Submitted 21 September, 2023; originally announced September 2023.

Comments: 46 pages, 6 figures

Learning continuous-valued treatment effects through representation balancing

Authors: Christopher Bockel-Rickermann, Toon Vanderschueren, Jeroen Berrevoets, Tim Verdonck, Wouter Verbeke

Abstract: Estimating the effects of treatments with an associated dose on an instance's outcome, the "dose response", is relevant in a variety of domains, from healthcare to business, economics, and beyond. Such effects, also known as continuous-valued treatment effects, are typically estimated from observational data, which may be subject to dose selection bias. This means that the allocation of doses depe… ▽ More Estimating the effects of treatments with an associated dose on an instance's outcome, the "dose response", is relevant in a variety of domains, from healthcare to business, economics, and beyond. Such effects, also known as continuous-valued treatment effects, are typically estimated from observational data, which may be subject to dose selection bias. This means that the allocation of doses depends on pre-treatment covariates. Previous studies have shown that conventional machine learning approaches fail to learn accurate individual estimates of dose responses under the presence of dose selection bias. In this work, we propose CBRNet, a causal machine learning approach to estimate an individual dose response from observational data. CBRNet adopts the Neyman-Rubin potential outcome framework and extends the concept of balanced representation learning for overcoming selection bias to continuous-valued treatments. Our work is the first to apply representation balancing in a continuous-valued treatment setting. We evaluate our method on a newly proposed benchmark. Our experiments demonstrate CBRNet's ability to accurately learn treatment effects under selection bias and competitive performance with respect to other state-of-the-art methods. △ Less

Submitted 7 September, 2023; originally announced September 2023.

Comments: 24 pages, 8 figures

MSC Class: 62D20

A Causal Perspective on Loan Pricing: Investigating the Impacts of Selection Bias on Identifying Bid-Response Functions

Authors: Christopher Bockel-Rickermann, Sam Verboven, Tim Verdonck, Wouter Verbeke

Abstract: In lending, where prices are specific to both customers and products, having a well-functioning personalized pricing policy in place is essential to effective business making. Typically, such a policy must be derived from observational data, which introduces several challenges. While the problem of ``endogeneity'' is prominently studied in the established pricing literature, the problem of selecti… ▽ More In lending, where prices are specific to both customers and products, having a well-functioning personalized pricing policy in place is essential to effective business making. Typically, such a policy must be derived from observational data, which introduces several challenges. While the problem of ``endogeneity'' is prominently studied in the established pricing literature, the problem of selection bias (or, more precisely, bid selection bias) is not. We take a step towards understanding the effects of selection bias by posing pricing as a problem of causal inference. Specifically, we consider the reaction of a customer to price a treatment effect. In our experiments, we simulate varying levels of selection bias on a semi-synthetic dataset on mortgage loan applications in Belgium. We investigate the potential of parametric and nonparametric methods for the identification of individual bid-response functions. Our results illustrate how conventional methods such as logistic regression and neural networks suffer adversely from selection bias. In contrast, we implement state-of-the-art methods from causal machine learning and show their capability to overcome selection bias in pricing data. △ Less

Submitted 7 September, 2023; originally announced September 2023.

Comments: 24 pages, 5 figures

Timing Process Interventions with Causal Inference and Reinforcement Learning

Authors: Hans Weytjens, Wouter Verbeke, Jochen De Weerdt

Abstract: The shift from the understanding and prediction of processes to their optimization offers great benefits to businesses and other organizations. Precisely timed process interventions are the cornerstones of effective optimization. Prescriptive process monitoring (PresPM) is the sub-field of process mining that concentrates on process optimization. The emerging PresPM literature identifies state-of-… ▽ More The shift from the understanding and prediction of processes to their optimization offers great benefits to businesses and other organizations. Precisely timed process interventions are the cornerstones of effective optimization. Prescriptive process monitoring (PresPM) is the sub-field of process mining that concentrates on process optimization. The emerging PresPM literature identifies state-of-the-art methods, causal inference (CI) and reinforcement learning (RL), without presenting a quantitative comparison. Most experiments are carried out using historical data, causing problems with the accuracy of the methods' evaluations and preempting online RL. Our contribution consists of experiments on timed process interventions with synthetic data that renders genuine online RL and the comparison to CI possible, and allows for an accurate evaluation of the results. Our experiments reveal that RL's policies outperform those from CI and are more robust at the same time. Indeed, the RL policies approach perfect policies. Unlike CI, the unaltered online RL approach can be applied to other, more generic PresPM problems such as next best activity recommendations. Nonetheless, CI has its merits in settings where online learning is not an option. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Accounting For Informative Sampling When Learning to Forecast Treatment Outcomes Over Time

Authors: Toon Vanderschueren, Alicia Curth, Wouter Verbeke, Mihaela van der Schaar

Abstract: Machine learning (ML) holds great potential for accurately forecasting treatment outcomes over time, which could ultimately enable the adoption of more individualized treatment strategies in many practical applications. However, a significant challenge that has been largely overlooked by the ML literature on this topic is the presence of informative sampling in observational data. When instances a… ▽ More Machine learning (ML) holds great potential for accurately forecasting treatment outcomes over time, which could ultimately enable the adoption of more individualized treatment strategies in many practical applications. However, a significant challenge that has been largely overlooked by the ML literature on this topic is the presence of informative sampling in observational data. When instances are observed irregularly over time, sampling times are typically not random, but rather informative -- depending on the instance's characteristics, past outcomes, and administered treatments. In this work, we formalize informative sampling as a covariate shift problem and show that it can prohibit accurate estimation of treatment outcomes if not properly accounted for. To overcome this challenge, we present a general framework for learning treatment outcomes in the presence of informative sampling using inverse intensity-weighting, and propose a novel method, TESAR-CDE, that instantiates this framework using Neural CDEs. Using a simulation environment based on a clinical use case, we demonstrate the effectiveness of our approach in learning under informative sampling. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Comments: To appear in the Proceedings of the 40th International Conference on Machine Learning, Honolulu, Hawaii, USA. PMLR 202, 2023

Client Recruitment for Federated Learning in ICU Length of Stay Prediction

Authors: Vincent Scheltjens, Lyse Naomi Wamba Momo, Wouter Verbeke, Bart De Moor

Abstract: Machine and deep learning methods for medical and healthcare applications have shown significant progress and performance improvement in recent years. These methods require vast amounts of training data which are available in the medical sector, albeit decentralized. Medical institutions generate vast amounts of data for which sharing and centralizing remains a challenge as the result of data and… ▽ More Machine and deep learning methods for medical and healthcare applications have shown significant progress and performance improvement in recent years. These methods require vast amounts of training data which are available in the medical sector, albeit decentralized. Medical institutions generate vast amounts of data for which sharing and centralizing remains a challenge as the result of data and privacy regulations. The federated learning technique is well-suited to tackle these challenges. However, federated learning comes with a new set of open problems related to communication overhead, efficient parameter aggregation, client selection strategies and more. In this work, we address the step prior to the initiation of a federated network for model training, client recruitment. By intelligently recruiting clients, communication overhead and overall cost of training can be reduced without sacrificing predictive performance. Client recruitment aims at pre-excluding potential clients from partaking in the federation based on a set of criteria indicative of their eventual contributions to the federation. In this work, we propose a client recruitment approach using only the output distribution and sample size at the client site. We show how a subset of clients can be recruited without sacrificing model performance whilst, at the same time, significantly improving computation time. By applying the recruitment approach to the training of federated models for accurate patient Length of Stay prediction using data from 189 Intensive Care Units, we show how the models trained in federations made up from recruited clients significantly outperform federated models trained with the standard procedure in terms of predictive power and training time. △ Less

Submitted 28 April, 2023; originally announced April 2023.

Fraud Analytics: A Decade of Research -- Organizing Challenges and Solutions in the Field

Authors: Christopher Bockel-Rickermann, Tim Verdonck, Wouter Verbeke

Abstract: The literature on fraud analytics and fraud detection has seen a substantial increase in output in the past decade. This has led to a wide range of research topics and overall little organization of the many aspects of fraud analytical research. The focus of academics ranges from identifying fraudulent credit card payments to spotting illegitimate insurance claims. In addition, there is a wide ran… ▽ More The literature on fraud analytics and fraud detection has seen a substantial increase in output in the past decade. This has led to a wide range of research topics and overall little organization of the many aspects of fraud analytical research. The focus of academics ranges from identifying fraudulent credit card payments to spotting illegitimate insurance claims. In addition, there is a wide range of methods and research objectives. This paper aims to provide an overview of fraud analytics in research and aims to more narrowly organize the discipline and its many subfields. We analyze a sample of almost 300 records on fraud analytics published between 2011 and 2020. In a systematic way, we identify the most prominent domains of application, challenges faced, performance metrics, and methods used. In addition, we build a framework for fraud analytical methods and propose a keywording strategy for future research. One of the key challenges in fraud analytics is access to public datasets. To further aid the community, we provide eight requirements for suitable data sets in research motivated by our research. We structure our sample of the literature in an online database. The database is available online for fellow researchers to investigate and potentially build upon. △ Less

Submitted 7 December, 2022; originally announced December 2022.

Prescriptive maintenance with causal machine learning

Authors: Toon Vanderschueren, Robert Boute, Tim Verdonck, Bart Baesens, Wouter Verbeke

Abstract: Machine maintenance is a challenging operational problem, where the goal is to plan sufficient preventive maintenance to avoid machine failures and overhauls. Maintenance is often imperfect in reality and does not make the asset as good as new. Although a variety of imperfect maintenance policies have been proposed in the literature, these rely on strong assumptions regarding the effect of mainten… ▽ More Machine maintenance is a challenging operational problem, where the goal is to plan sufficient preventive maintenance to avoid machine failures and overhauls. Maintenance is often imperfect in reality and does not make the asset as good as new. Although a variety of imperfect maintenance policies have been proposed in the literature, these rely on strong assumptions regarding the effect of maintenance on the machine's condition, assuming the effect is (1) deterministic or governed by a known probability distribution, and (2) machine-independent. This work proposes to relax both assumptions by learning the effect of maintenance conditional on a machine's characteristics from observational data on similar machines using existing methodologies for causal inference. By predicting the maintenance effect, we can estimate the number of overhauls and failures for different levels of maintenance and, consequently, optimize the preventive maintenance frequency to minimize the total estimated cost. We validate our proposed approach using real-life data on more than 4,000 maintenance contracts from an industrial partner. Empirical results show that our novel, causal approach accurately predicts the maintenance effect and results in individualized maintenance schedules that are more accurate and cost-effective than supervised or non-individualized approaches. △ Less

Submitted 3 June, 2022; originally announced June 2022.

A new perspective on classification: optimally allocating limited resources to uncertain tasks

Authors: Toon Vanderschueren, Bart Baesens, Tim Verdonck, Wouter Verbeke

Abstract: A central problem in business concerns the optimal allocation of limited resources to a set of available tasks, where the payoff of these tasks is inherently uncertain. In credit card fraud detection, for instance, a bank can only assign a small subset of transactions to their fraud investigations team. Typically, such problems are solved using a classification framework, where the focus is on pre… ▽ More A central problem in business concerns the optimal allocation of limited resources to a set of available tasks, where the payoff of these tasks is inherently uncertain. In credit card fraud detection, for instance, a bank can only assign a small subset of transactions to their fraud investigations team. Typically, such problems are solved using a classification framework, where the focus is on predicting task outcomes given a set of characteristics. Resources are then allocated to the tasks that are predicted to be the most likely to succeed. However, we argue that using classification to address task uncertainty is inherently suboptimal as it does not take into account the available capacity. Therefore, we first frame the problem as a type of assignment problem. Then, we present a novel solution using learning to rank by directly optimizing the assignment's expected profit given limited, stochastic capacity. This is achieved by optimizing a specific instance of the net discounted cumulative gain, a commonly used class of metrics in learning to rank. Empirically, we demonstrate that our new method achieves higher expected profit and expected precision compared to a classification approach for a wide variety of application areas and data sets. This illustrates the benefit of an integrated approach and of explicitly considering the available resources when learning a predictive model. △ Less

Submitted 9 February, 2022; originally announced February 2022.

Weight-of-evidence 2.0 with shrinkage and spline-binning

Authors: Jakob Raymaekers, Wouter Verbeke, Tim Verdonck

Abstract: In many practical applications, such as fraud detection, credit risk modeling or medical decision making, classification models for assigning instances to a predefined set of classes are required to be both precise as well as interpretable. Linear modeling methods such as logistic regression are often adopted, since they offer an acceptable balance between precision and interpretability. Linear me… ▽ More In many practical applications, such as fraud detection, credit risk modeling or medical decision making, classification models for assigning instances to a predefined set of classes are required to be both precise as well as interpretable. Linear modeling methods such as logistic regression are often adopted, since they offer an acceptable balance between precision and interpretability. Linear methods, however, are not well equipped to handle categorical predictors with high-cardinality or to exploit non-linear relations in the data. As a solution, data preprocessing methods such as weight-of-evidence are typically used for transforming the predictors. The binning procedure that underlies the weight-of-evidence approach, however, has been little researched and typically relies on ad-hoc or expert driven procedures. The objective in this paper, therefore, is to propose a formalized, data-driven and powerful method. To this end, we explore the discretization of continuous variables through the binning of spline functions, which allows for capturing non-linear effects in the predictor variables and yields highly interpretable predictors taking only a small number of discrete values. Moreover, we extend upon the weight-of-evidence approach and propose to estimate the proportions using shrinkage estimators. Together, this offers an improved ability to exploit both non-linear and categorical predictors for achieving increased classification precision, while maintaining interpretability of the resulting model and decreasing the risk of overfitting. We present the results of a series of experiments in a fraud detection setting, which illustrate the effectiveness of the presented approach. We facilitate reproduction of the presented results and adoption of the proposed approaches by providing both the dataset and the code for implementing the experiments and the presented approach. △ Less

Submitted 24 September, 2021; v1 submitted 5 January, 2021; originally announced January 2021.

Comments: New version: duplicate paragraph omitted

To do or not to do: cost-sensitive causal decision-making

Authors: Diego Olaya, Wouter Verbeke, Jente Van Belle, Marie-Anne Guerry

Abstract: Causal classification models are adopted across a variety of operational business processes to predict the effect of a treatment on a categorical business outcome of interest depending on the process instance characteristics. This allows optimizing operational decision-making and selecting the optimal treatment to apply in each specific instance, with the aim of maximizing the positive outcome rat… ▽ More Causal classification models are adopted across a variety of operational business processes to predict the effect of a treatment on a categorical business outcome of interest depending on the process instance characteristics. This allows optimizing operational decision-making and selecting the optimal treatment to apply in each specific instance, with the aim of maximizing the positive outcome rate. While various powerful approaches have been presented in the literature for learning causal classification models, no formal framework has been elaborated for optimal decision-making based on the estimated individual treatment effects, given the cost of the various treatments and the benefit of the potential outcomes. In this article, we therefore extend upon the expected value framework and formally introduce a cost-sensitive decision boundary for double binary causal classification, which is a linear function of the estimated individual treatment effect, the positive outcome probability and the cost and benefit parameters of the problem setting. The boundary allows causally classifying instances in the positive and negative treatment class to maximize the expected causal profit, which is introduced as the objective at hand in cost-sensitive causal classification. We introduce the expected causal profit ranker which ranks instances for maximizing the expected causal profit at each possible threshold for causally classifying instances and differs from the conventional ranking approach based on the individual treatment effect. The proposed ranking approach is experimentally evaluated on synthetic and marketing campaign data sets. The results indicate that the presented ranking method effectively outperforms the cost-insensitive ranking approach and allows boosting profitability. △ Less

Submitted 5 January, 2021; originally announced January 2021.

HydaLearn: Highly Dynamic Task Weighting for Multi-task Learning with Auxiliary Tasks

Authors: Sam Verboven, Muhammad Hafeez Chaudhary, Jeroen Berrevoets, Wouter Verbeke

Abstract: Multi-task learning (MTL) can improve performance on a task by sharing representations with one or more related auxiliary-tasks. Usually, MTL-networks are trained on a composite loss function formed by a constant weighted combination of the separate task losses. In practice, constant loss weights lead to poor results for two reasons: (i) the relevance of the auxiliary tasks can gradually drift thr… ▽ More Multi-task learning (MTL) can improve performance on a task by sharing representations with one or more related auxiliary-tasks. Usually, MTL-networks are trained on a composite loss function formed by a constant weighted combination of the separate task losses. In practice, constant loss weights lead to poor results for two reasons: (i) the relevance of the auxiliary tasks can gradually drift throughout the learning process; (ii) for mini-batch based optimisation, the optimal task weights vary significantly from one update to the next depending on mini-batch sample composition. We introduce HydaLearn, an intelligent weighting algorithm that connects main-task gain to the individual task gradients, in order to inform dynamic loss weighting at the mini-batch level, addressing i and ii. Using HydaLearn, we report performance increases on synthetic data, as well as on two supervised learning domains. △ Less

Submitted 26 August, 2020; originally announced August 2020.

The foundations of cost-sensitive causal classification

Authors: Wouter Verbeke, Diego Olaya, Jeroen Berrevoets, Sam Verboven, Sebastián Maldonado

Abstract: Classification is a well-studied machine learning task which concerns the assignment of instances to a set of outcomes. Classification models support the optimization of managerial decision-making across a variety of operational business processes. For instance, customer churn prediction models are adopted to increase the efficiency of retention campaigns by optimizing the selection of customers t… ▽ More Classification is a well-studied machine learning task which concerns the assignment of instances to a set of outcomes. Classification models support the optimization of managerial decision-making across a variety of operational business processes. For instance, customer churn prediction models are adopted to increase the efficiency of retention campaigns by optimizing the selection of customers that are to be targeted. Cost-sensitive and causal classification methods have independently been proposed to improve the performance of classification models. The former considers the benefits and costs of correct and incorrect classifications, such as the benefit of a retained customer, whereas the latter estimates the causal effect of an action, such as a retention campaign, on the outcome of interest. This study integrates cost-sensitive and causal classification by elaborating a unifying evaluation framework. The framework encompasses a range of existing and novel performance measures for evaluating both causal and conventional classification models in a cost-sensitive as well as a cost-insensitive manner. We proof that conventional classification is a specific case of causal classification in terms of a range of performance measures when the number of actions is equal to one. The framework is shown to instantiate to application-specific cost-sensitive performance measures that have been recently proposed for evaluating customer retention and response uplift models, and allows to maximize profitability when adopting a causal classification model for optimizing decision-making. The proposed framework paves the way toward the development of cost-sensitive causal learning methods and opens a range of opportunities for improving data-driven business decision-making. △ Less

Submitted 20 April, 2021; v1 submitted 24 July, 2020; originally announced July 2020.

Comments: New version: overall language edit - switch in notation (class 0: negatives, class 1: positive) - typos corrected - proofs moved to appendix

Misclassification cost-sensitive ensemble learning: A unifying framework

Authors: George Petrides, Wouter Verbeke

Abstract: Over the years, a plethora of cost-sensitive methods have been proposed for learning on data when different types of misclassification errors incur different costs. Our contribution is a unifying framework that provides a comprehensive and insightful overview on cost-sensitive ensemble methods, pinpointing their differences and similarities via a fine-grained categorization. Our framework contains… ▽ More Over the years, a plethora of cost-sensitive methods have been proposed for learning on data when different types of misclassification errors incur different costs. Our contribution is a unifying framework that provides a comprehensive and insightful overview on cost-sensitive ensemble methods, pinpointing their differences and similarities via a fine-grained categorization. Our framework contains natural extensions and generalisations of ideas across methods, be it AdaBoost, Bagging or Random Forest, and as a result not only yields all methods known to date but also some not previously considered. △ Less

Submitted 14 July, 2020; originally announced July 2020.

Instance-Dependent Cost-Sensitive Learning for Detecting Transfer Fraud

Authors: Sebastiaan Höppner, Bart Baesens, Wouter Verbeke, Tim Verdonck

Abstract: Card transaction fraud is a growing problem affecting card holders worldwide. Financial institutions increasingly rely upon data-driven methods for develo** fraud detection systems, which are able to automatically detect and block fraudulent transactions. From a machine learning perspective, the task of detecting fraudulent transactions is a binary classification problem. Classification models a… ▽ More Card transaction fraud is a growing problem affecting card holders worldwide. Financial institutions increasingly rely upon data-driven methods for develo** fraud detection systems, which are able to automatically detect and block fraudulent transactions. From a machine learning perspective, the task of detecting fraudulent transactions is a binary classification problem. Classification models are commonly trained and evaluated in terms of statistical performance measures, such as likelihood and AUC, respectively. These measures, however, do not take into account the actual business objective, which is to minimize the financial losses due to fraud. Fraud detection is to be acknowledged as an instance-dependent cost-sensitive classification problem, where the costs due to misclassification vary between instances, and requiring adapted approaches for learning a classification model. In this article, an instance-dependent threshold is derived, based on the instance-dependent cost matrix for transfer fraud detection, that allows for making the optimal cost-based decision for each transaction. Two novel classifiers are presented, based on lasso-regularized logistic regression and gradient tree boosting, which directly minimize the proposed instance-dependent cost measure when learning a classification model. The proposed methods are implemented in the R packages cslogit and csboost, and compared against state-of-the-art methods on a publicly available data set from the machine learning competition website Kaggle and a proprietary card transaction data set. The results of the experiments highlight the potential of reducing fraud losses by adopting the proposed methods. △ Less

Submitted 5 May, 2020; originally announced May 2020.

Comments: 24 pages, 4 figures, submitted

A study of the U.S. domestic air transportation network: Temporal evolution of network topology and robustness from 2001 to 2016

Authors: Leonidas Siozos-Rousoulis, Dimitri Robert, Wouter Verbeke

Abstract: The U.S. air transportation network (ATN) is critical to the mobility and the functioning of the United States. It is thus necessary to ensure that it is well-connected, efficient, and robust. Despite extensive research on its topology, the temporal evolution of the network's robustness and tolerance remains largely unexplored. In the present paper, a temporal study of the domestic U.S. ATN was pe… ▽ More The U.S. air transportation network (ATN) is critical to the mobility and the functioning of the United States. It is thus necessary to ensure that it is well-connected, efficient, and robust. Despite extensive research on its topology, the temporal evolution of the network's robustness and tolerance remains largely unexplored. In the present paper, a temporal study of the domestic U.S. ATN was performed based on annual flight data from 1996 to 2016 and network analytics were used to examine the effects of restructuring that followed the 9/11 events along with the current state of the system. Centrality measures were computed to assess the system's topology and its global robustness. A node deletion method was applied to assess the network's tolerance by simulating a targeted attack scenario. The study showed that the 9/11 terrorist attacks triggered vast restructuring of the network, in terms of efficiency and security. Air traffic expanded, as new airports and air routes were introduced. Airlines reconsidered their strategy and optimized their operations, thus allowing the network to recover rapidly and become even more efficient. Security concerns resulted in significant improvement of the network's robustness. Since 2001, the global traffic and topological properties of the U.S. ATN have displayed continuous growth, due to the network's expansion. On the other hand, the robustness of the system has not shown an improving tendency. Findings suggest that although the system's ability to sustain its operational level under extreme circumstances has lately improved, its tolerance to targeted attacks has deteriorated. The presented methodology may be applied on different network levels or different transportation networks, to provide a general perspective of the system's vulnerabilities. △ Less

Submitted 3 May, 2020; originally announced May 2020.

Autoencoders for strategic decision support

Authors: Sam Verboven, Jeroen Berrevoets, Chris Wuytens, Bart Baesens, Wouter Verbeke

Abstract: In the majority of executive domains, a notion of normality is involved in most strategic decisions. However, few data-driven tools that support strategic decision-making are available. We introduce and extend the use of autoencoders to provide strategically relevant granular feedback. A first experiment indicates that experts are inconsistent in their decision making, highlighting the need for st… ▽ More In the majority of executive domains, a notion of normality is involved in most strategic decisions. However, few data-driven tools that support strategic decision-making are available. We introduce and extend the use of autoencoders to provide strategically relevant granular feedback. A first experiment indicates that experts are inconsistent in their decision making, highlighting the need for strategic decision support. Furthermore, using two large industry-provided human resources datasets, the proposed solution is evaluated in terms of ranking accuracy, synergy with human experts, and dimension-level feedback. This three-point scheme is validated using (a) synthetic data, (b) the perspective of data quality, (c) blind expert validation, and (d) transparent expert evaluation. Our study confirms several principal weaknesses of human decision-making and stresses the importance of synergy between a model and humans. Moreover, unsupervised learning and in particular the autoencoder are shown to be valuable tools for strategic decision-making. △ Less

Submitted 3 May, 2020; originally announced May 2020.

Learning to rank for uplift modeling

Authors: Floris Devriendt, Tias Guns, Wouter Verbeke

Abstract: Uplift modeling has effectively been used in fields such as marketing and customer retention, to target those customers that are most likely to respond due to the campaign or treatment. Uplift models produce uplift scores which are then used to essentially create a ranking. We instead investigate to learn to rank directly by looking into the potential of learning-to-rank techniques in the context… ▽ More Uplift modeling has effectively been used in fields such as marketing and customer retention, to target those customers that are most likely to respond due to the campaign or treatment. Uplift models produce uplift scores which are then used to essentially create a ranking. We instead investigate to learn to rank directly by looking into the potential of learning-to-rank techniques in the context of uplift modeling. We propose a unified formalisation of different global uplift modeling measures in use today and explore how these can be integrated into the learning-to-rank framework. Additionally, we introduce a new metric for learning-to-rank that focusses on optimizing the area under the uplift curve called the promoted cumulative gain (PCG). We employ the learning-to-rank technique LambdaMART to optimize the ranking according to PCG and show improved results over standard learning-to-rank metrics and equal to improved results when compared with state-of-the-art uplift modeling. Finally, we show how learning-to-rank models can learn to optimize a certain targeting depth, however, these results do not generalize on the test set. △ Less

Submitted 14 February, 2020; originally announced February 2020.

Social Network Analytics for Churn Prediction in Telco: Model Building, Evaluation and Network Architecture

Authors: María Óskarsdóttir, Cristián Bravo, Wouter Verbeke, Carlos Sarraute, Bart Baesens, Jan Vanthienen

Abstract: Social network analytics methods are being used in the telecommunication industry to predict customer churn with great success. In particular it has been shown that relational learners adapted to this specific problem enhance the performance of predictive models. In the current study we benchmark different strategies for constructing a relational learner by applying them to a total of eight dist… ▽ More Social network analytics methods are being used in the telecommunication industry to predict customer churn with great success. In particular it has been shown that relational learners adapted to this specific problem enhance the performance of predictive models. In the current study we benchmark different strategies for constructing a relational learner by applying them to a total of eight distinct call-detail record datasets, originating from telecommunication organizations across the world. We statistically evaluate the effect of relational classifiers and collective inference methods on the predictive power of relational learners, as well as the performance of models where relational learners are combined with traditional methods of predicting customer churn in the telecommunication industry. Finally we investigate the effect of network construction on model performance; our findings imply that the definition of edges and weights in the network does have an impact on the results of the predictive models. As a result of the study, the best configuration is a non-relational learner enriched with network variables, without collective inference, using binary weights and undirected networks. In addition, we provide guidelines on how to apply social networks analytics for churn prediction in the telecommunication industry in an optimal way, ranging from network architecture to model building and evaluation. △ Less

Submitted 18 January, 2020; originally announced January 2020.

Journal ref: Expert Systems with Applications, Volume 85, 1 November 2017, Pages 204-220

A Comparative Study of Social Network Classifiers for Predicting Churn in the Telecommunication Industry

Authors: Maria Óskarsdóttir, Cristián Bravo, Wouter Verbeke, Carlos Sarraute, Bart Baesens, Jan Vanthienen

Abstract: Relational learning in networked data has been shown to be effective in a number of studies. Relational learners, composed of relational classifiers and collective inference methods, enable the inference of nodes in a network given the existence and strength of links to other nodes. These methods have been adapted to predict customer churn in telecommunication companies showing that incorporating… ▽ More Relational learning in networked data has been shown to be effective in a number of studies. Relational learners, composed of relational classifiers and collective inference methods, enable the inference of nodes in a network given the existence and strength of links to other nodes. These methods have been adapted to predict customer churn in telecommunication companies showing that incorporating them may give more accurate predictions. In this research, the performance of a variety of relational learners is compared by applying them to a number of CDR datasets originating from the telecommunication industry, with the goal to rank them as a whole and investigate the effects of relational classifiers and collective inference methods separately. Our results show that collective inference methods do not improve the performance of relational classifiers and the best performing relational classifier is the network-only link-based classifier, which builds a logistic model using link-based measures for the nodes in the network. △ Less

Submitted 18 January, 2020; originally announced January 2020.

Comments: 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)

Optimising Individual-Treatment-Effect Using Bandits

Authors: Jeroen Berrevoets, Sam Verboven, Wouter Verbeke

Abstract: Applying causal inference models in areas such as economics, healthcare and marketing receives great interest from the machine learning community. In particular, estimating the individual-treatment-effect (ITE) in settings such as precision medicine and targeted advertising has peaked in application. Optimising this ITE under the strong-ignorability-assumption -- meaning all confounders expressing… ▽ More Applying causal inference models in areas such as economics, healthcare and marketing receives great interest from the machine learning community. In particular, estimating the individual-treatment-effect (ITE) in settings such as precision medicine and targeted advertising has peaked in application. Optimising this ITE under the strong-ignorability-assumption -- meaning all confounders expressing influence on the outcome of a treatment are registered in the data -- is often referred to as uplift modeling (UM). While these techniques have proven useful in many settings, they suffer vividly in a dynamic environment due to concept drift. Take for example the negative influence on a marketing campaign when a competitor product is released. To counter this, we propose the uplifted contextual multi-armed bandit (U-CMAB), a novel approach to optimise the ITE by drawing upon bandit literature. Experiments on real and simulated data indicate that our proposed approach compares favourably against the state-of-the-art. All our code can be found online at https://github.com/vub-dl/u-cmab. △ Less

Submitted 16 October, 2019; originally announced October 2019.

Single top quark and rare top quark production at ATLAS and CMS

Authors: Willem Verbeke

Abstract: The latest results from ATLAS and CMS on single top quark production and rare production channels of top quarks at the LHC are presented. The latest results from ATLAS and CMS on single top quark production and rare production channels of top quarks at the LHC are presented. △ Less

Submitted 20 August, 2019; originally announced August 2019.

Comments: Proceedings of the 54th Rencontres de Moriond on QCD and high energy interactions

Causal Simulations for Uplift Modeling

Authors: Jeroen Berrevoets, Wouter Verbeke

Abstract: Uplift modeling requires experimental data, preferably collected in random fashion. This places a logistical and financial burden upon any organisation aspiring such models. Once deployed, uplift models are subject to effects from concept drift. Hence, methods are being developed that are able to learn from newly gained experience, as well as handle drifting environments. As these new methods atte… ▽ More Uplift modeling requires experimental data, preferably collected in random fashion. This places a logistical and financial burden upon any organisation aspiring such models. Once deployed, uplift models are subject to effects from concept drift. Hence, methods are being developed that are able to learn from newly gained experience, as well as handle drifting environments. As these new methods attempt to eliminate the need for experimental data, another approach to test such methods must be formulated. Therefore, we propose a method to simulate environments that offer causal relationships in their parameters. △ Less

Submitted 1 February, 2019; originally announced February 2019.

Search for heavy neutral leptons with the CMS detector

Authors: Willem Verbeke

Abstract: The smallness of neutrino masses provides a tantalizing allusion to physics beyond the standard model. Heavy neutral leptons ($N$), such as hypothetical sterile neutrinos, accommodate a way to explain this observation, through the see-saw mechanism. If they exist, $N$ could also provide answers about the nature of dark matter, and the baryon asymmetry of the universe. A search for the production o… ▽ More The smallness of neutrino masses provides a tantalizing allusion to physics beyond the standard model. Heavy neutral leptons ($N$), such as hypothetical sterile neutrinos, accommodate a way to explain this observation, through the see-saw mechanism. If they exist, $N$ could also provide answers about the nature of dark matter, and the baryon asymmetry of the universe. A search for the production of $N$ at the LHC, originating from leptonic W boson decays through the mixing of $N$ with SM neutrinos, is presented. The search focuses on signatures with three leptons, providing a clean signal for probing the production of $N$ in a wide mass range never explored before at the LHC: down to 1 GeV, and up to 1.2 TeV. The sample of proton-proton collisions collected by the CMS detector throughout 2016 is used, amounting to a volume of 35.9 $\text{fb}^{-1}$. The results are presented in the plane of the mixing parameter of $N$ to their SM counterparts, versus their mass, and are the first such result at a hadron collider for masses below 40 GeV and the first direct result for masses above 500 GeV, more than doubling the probed mass range. △ Less

Submitted 14 May, 2018; originally announced May 2018.

Comments: 4 pages, 3 figures, proceedings for the 53rd Rencontres de Moriond 2018: Electroweak Interactions and Unified Theories

Report number: CMS CR-2018/037