-
Forecast reconciliation for vaccine supply chain optimization
Authors:
Bhanu Angam,
Alessandro Beretta,
Eli De Poorter,
Matthieu Duvinage,
Daniel Peralta
Abstract:
Vaccine supply chain optimization can benefit from hierarchical time series forecasting, when grou** the vaccines by type or location. However, forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts, which can be addressed by reconciliation methods. In this paper, we tackle the vaccine sale forecasting problem by modeling s…
▽ More
Vaccine supply chain optimization can benefit from hierarchical time series forecasting, when grou** the vaccines by type or location. However, forecasts of different hierarchy levels become incoherent when higher levels do not match the sum of the lower levels forecasts, which can be addressed by reconciliation methods. In this paper, we tackle the vaccine sale forecasting problem by modeling sales data from GSK between 2010 and 2021 as a hierarchical time series. After forecasting future values with several ARIMA models, we systematically compare the performance of various reconciliation methods, using statistical tests. We also compare the performance of the forecast before and after COVID. The results highlight Minimum Trace and Weighted Least Squares with Structural scaling as the best performing methods, which provided a coherent forecast while reducing the forecast error of the baseline ARIMA.
△ Less
Submitted 2 May, 2023;
originally announced May 2023.
-
Human Response to an AI-Based Decision Support System: A User Study on the Effects of Accuracy and Bias
Authors:
David Solans,
Andrea Beretta,
Manuel Portela,
Carlos Castillo,
Anna Monreale
Abstract:
Artificial Intelligence (AI) is increasingly used to build Decision Support Systems (DSS) across many domains. This paper describes a series of experiments designed to observe human response to different characteristics of a DSS such as accuracy and bias, particularly the extent to which participants rely on the DSS, and the performance they achieve. In our experiments, participants play a simple…
▽ More
Artificial Intelligence (AI) is increasingly used to build Decision Support Systems (DSS) across many domains. This paper describes a series of experiments designed to observe human response to different characteristics of a DSS such as accuracy and bias, particularly the extent to which participants rely on the DSS, and the performance they achieve. In our experiments, participants play a simple online game inspired by so-called "wildcat" (i.e., exploratory) drilling for oil. The landscape has two layers: a visible layer describing the costs (terrain), and a hidden layer describing the reward (oil yield). Participants in the control group play the game without receiving any assistance, while in treatment groups they are assisted by a DSS suggesting places to drill. For certain treatments, the DSS does not consider costs, but only rewards, which introduces a bias that is observable by users. Between subjects, we vary the accuracy and bias of the DSS, and observe the participants' total score, time to completion, the extent to which they follow or ignore suggestions. We also measure the acceptability of the DSS in an exit survey. Our results show that participants tend to score better with the DSS, that the score increase is due to users following the DSS advice, and related to the difficulty of the game and the accuracy of the DSS. We observe that this setting elicits mostly rational behavior from participants, who place a moderate amount of trust in the DSS and show neither algorithmic aversion (under-reliance) nor automation bias (over-reliance).However, their stated willingness to accept the DSS in the exit survey seems less sensitive to the accuracy of the DSS than their behavior, suggesting that users are only partially aware of the (lack of) accuracy of the DSS.
△ Less
Submitted 24 March, 2022;
originally announced March 2022.
-
Explainable Deep Image Classifiers for Skin Lesion Diagnosis
Authors:
Carlo Metta,
Andrea Beretta,
Riccardo Guidotti,
Yuan Yin,
Patrick Gallinari,
Salvatore Rinzivillo,
Fosca Giannotti
Abstract:
A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we ana…
▽ More
A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we analyze a case study on skin lesion images where we customize an existing XAI approach for explaining a deep learning model able to recognize different types of skin lesions. The explanation is formed by synthetic exemplar and counter-exemplar images of skin lesion and offers the practitioner a way to highlight the crucial traits responsible for the classification decision. A survey conducted with domain experts, beginners and unskilled people proof that the usage of explanations increases the trust and confidence in the automatic decision system. Also, an analysis of the latent space adopted by the explainer unveils that some of the most frequent skin lesion classes are distinctly separated. This phenomenon could derive from the intrinsic characteristics of each class and, hopefully, can provide support in the resolution of the most frequent misclassifications by human experts.
△ Less
Submitted 22 November, 2021;
originally announced November 2021.
-
Following wrong suggestions: self-blame in human and computer scenarios
Authors:
Andrea Beretta,
Massimo Zancanaro,
Bruno Lepri
Abstract:
This paper investigates the specific experience of following a suggestion by an intelligent machine that has a wrong outcome and the emotions people feel. By adopting a typical task employed in studies on decision-making, we presented participants with two scenarios in which they follow a suggestion and have a wrong outcome by either an expert human being or an intelligent machine. We found a sign…
▽ More
This paper investigates the specific experience of following a suggestion by an intelligent machine that has a wrong outcome and the emotions people feel. By adopting a typical task employed in studies on decision-making, we presented participants with two scenarios in which they follow a suggestion and have a wrong outcome by either an expert human being or an intelligent machine. We found a significant decrease in the perceived responsibility on the wrong choice when the machine offers the suggestion. At present, few studies have investigated the negative emotions that could arise from a bad outcome after following the suggestion given by an intelligent system, and how to cope with the potential distrust that could affect the long-term use of the system and the cooperation. This preliminary research has implications in the study of cooperation and decision making with intelligent machines. Further research may address how to offer the suggestion in order to better cope with user's self-blame.
△ Less
Submitted 1 July, 2019;
originally announced July 2019.
-
The Stochastic complexity of spin models: Are pairwise models really simple?
Authors:
Alberto Beretta,
Claudia Battistin,
Clélia de Mulatier,
Iacopo Mastromatteo,
Matteo Marsili
Abstract:
Models can be simple for different reasons: because they yield a simple and computationally efficient interpretation of a generic dataset (e.g. in terms of pairwise dependences) - as in statistical learning - or because they capture the essential ingredients of a specific phenomenon - as e.g. in physics - leading to non-trivial falsifiable predictions. In information theory and Bayesian inference,…
▽ More
Models can be simple for different reasons: because they yield a simple and computationally efficient interpretation of a generic dataset (e.g. in terms of pairwise dependences) - as in statistical learning - or because they capture the essential ingredients of a specific phenomenon - as e.g. in physics - leading to non-trivial falsifiable predictions. In information theory and Bayesian inference, the simplicity of a model is precisely quantified in the stochastic complexity, which measures the number of bits needed to encode its parameters. In order to understand how simple models look like, we study the stochastic complexity of spin models with interactions of arbitrary order. We highlight the existence of invariances with respect to bijections within the space of operators, which allow us to partition the space of all models into equivalence classes, in which models share the same complexity. We thus found that the complexity (or simplicity) of a model is not determined by the order of the interactions, but rather by their mutual arrangements. Models where statistical dependencies are localized on non-overlap** groups of few variables (and that afford predictions on independencies that are easy to falsify) are simple. On the contrary, fully connected pairwise models, which are often used in statistical learning, appear to be highly complex, because of their extended set of interactions.
△ Less
Submitted 11 April, 2018; v1 submitted 24 February, 2017;
originally announced February 2017.
-
Analysis and Optimal Targets Setup of a Multihead Weighing Machine
Authors:
Enrique del Castillo,
Alessia Beretta,
Quirico Semeraro
Abstract:
Multihead weighing machines (MWMs) are ubiquitous in industry for fast and accurate packaging of a wide variety of foods and vegetables, small hardware items and office supplies. A MWM consists of a system of multiple hoppers that are filled with product which when discharged through a funnel fills a package to a desired weight. Operating this machine requires first to specify the product weight t…
▽ More
Multihead weighing machines (MWMs) are ubiquitous in industry for fast and accurate packaging of a wide variety of foods and vegetables, small hardware items and office supplies. A MWM consists of a system of multiple hoppers that are filled with product which when discharged through a funnel fills a package to a desired weight. Operating this machine requires first to specify the product weight targets or setpoints that each hopper should contain on average in each cycle, which do not need to be identical. The selection of these setpoints has a major impact on the performance of a MWM. Each cycle, the machine fills a package running a built-in knapsack algorithm that opens --or leaves shut-- different combinations of hoppers releasing their content such that the total weight of each package is near to its target, minimizing the amount of product ``given away". In this paper, we address the practical open problem for industry of how to determine the setpoint weights for each of the hoppers given a desired total package weight for a widely used type of MWM. An order statistic formulation based on a characterization of near-optimal solutions is presented. This is shown to be computationally intractable, and a faster heuristic that utilizes a lower bound approximation of the expected smallest order statistic is proposed instead. The setup solutions obtained with the proposed methods can result in substantial savings for MWM users. Alternatively, the analysis presented could be used by management to justify the acquisition of new MWM machines.
△ Less
Submitted 23 November, 2015;
originally announced November 2015.
