Showing 1–23 of 23 results for author: Goude, Y

Adaptive probabilistic forecasting of French electricity spot prices

Authors: Grégoire Dutot, Margaux Zaffran, Olivier Féron, Yannig Goude

Abstract: Electricity price forecasting (EPF) plays a major role for electricity companies as a fundamental entry for trading decisions or energy management operations. As electricity can not be stored, electricity prices are highly volatile which make EPF a particularly difficult task. This is all the more true when dramatic fortuitous events disrupt the markets. Trading and more generally energy managemen… ▽ More Electricity price forecasting (EPF) plays a major role for electricity companies as a fundamental entry for trading decisions or energy management operations. As electricity can not be stored, electricity prices are highly volatile which make EPF a particularly difficult task. This is all the more true when dramatic fortuitous events disrupt the markets. Trading and more generally energy management decisions require risk management tools which are based on probabilistic EPF (PEPF). In this challenging context, we argue in favor of the deployment of highly adaptive black-boxes strategies allowing to turn any forecasts into a robust adaptive predictive interval, such as conformal prediction and online aggregation, as a fundamental last layer of any operational pipeline. We propose to investigate a novel data set containing the French electricity spot prices during the turbulent 2020-2021 years, and build a new explanatory feature revealing high predictive power, namely the nuclear availability. Benchmarking state-of-the-art PEPF on this data set highlights the difficulty of choosing a given model, as they all behave very differently in practice, and none of them is reliable. However, we propose an adequate conformalisation, OSSCP-horizon, that improves the performances of PEPF methods, even in the most hazardous period of late 2021. Finally, we emphasize that combining it with online aggregation significantly outperforms any other approaches, and should be the preferred pipeline, as it provides trustworthy probabilistic forecasts. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Human spatial dynamics for electricity demand forecasting: the case of France during the 2022 energy crisis

Authors: Nathan Doumèche, Yann Allioux, Yannig Goude, Stefania Rubrichi

Abstract: Accurate electricity demand forecasting is crucial to meet energy security and efficiency, especially when relying on intermittent renewable energy sources. Recently, massive savings have been observed in Europe, following an unprecedented global energy crisis. However, assessing the impact of such crisis and of government incentives on electricity consumption behaviour is challenging. Moreover, s… ▽ More Accurate electricity demand forecasting is crucial to meet energy security and efficiency, especially when relying on intermittent renewable energy sources. Recently, massive savings have been observed in Europe, following an unprecedented global energy crisis. However, assessing the impact of such crisis and of government incentives on electricity consumption behaviour is challenging. Moreover, standard statistical models based on meteorological and calendar data have difficulty adapting to such brutal changes. Here, we show that mobility indices based on mobile network data significantly improve the performance of the state-of-the-art models in electricity demand forecasting during the sobriety period. We start by documenting the drop in the French electricity consumption during the winter of 2022-2023. We then show how our mobile network data captures work dynamics and how adding these mobility indices outperforms the state-of-the-art during this atypical period. Our results characterise the effect of work behaviours on the electricity demand. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Forecasting Electric Vehicle Charging Station Occupancy: Smarter Mobility Data Challenge

Authors: Yvenn Amara-Ouali, Yannig Goude, Nathan Doumèche, Pascal Veyret, Alexis Thomas, Daniel Hebenstreit, Thomas Wedenig, Arthur Satouf, Aymeric Jan, Yannick Deleuze, Paul Berhaut, Sébastien Treguer, Tiphaine Phe-Neau

Abstract: The transport sector is a major contributor to greenhouse gas emissions in Europe. Shifting to electric vehicles (EVs) powered by a low-carbon energy mix would reduce carbon emissions. However, to support the development of electric mobility, a better understanding of EV charging behaviours and more accurate forecasting models are needed. To fill that gap, the Smarter Mobility Data Challenge has f… ▽ More The transport sector is a major contributor to greenhouse gas emissions in Europe. Shifting to electric vehicles (EVs) powered by a low-carbon energy mix would reduce carbon emissions. However, to support the development of electric mobility, a better understanding of EV charging behaviours and more accurate forecasting models are needed. To fill that gap, the Smarter Mobility Data Challenge has focused on the development of forecasting models to predict EV charging station occupancy. This challenge involved analysing a dataset of 91 charging stations across four geographical areas over seven months in 2020-2021. The forecasts were evaluated at three levels of aggregation (individual stations, areas and global) to capture the inherent hierarchical structure of the data. The results highlight the potential of hierarchical forecasting approaches to accurately predict EV charging station occupancy, providing valuable insights for energy providers and EV users alike. This open dataset addresses many real-world challenges associated with time series, such as missing values, non-stationarity and spatio-temporal correlations. Access to the dataset, code and benchmarks are available at https://gitlab.com/smarter-mobility-data-challenge/tutorials to foster future research. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Adaptive Probabilistic Forecasting of Electricity (Net-)Load

Authors: Joseph de Vilmarest, Jethro Browell, Matteo Fasiolo, Yannig Goude, Olivier Wintenberger

Abstract: Electricity load forecasting is a necessary capability for power system operators and electricity market participants. The proliferation of local generation, demand response, and electrification of heat and transport are changing the fundamental drivers of electricity load and increasing the complexity of load modelling and forecasting. We address this challenge in two ways. First, our setting is… ▽ More Electricity load forecasting is a necessary capability for power system operators and electricity market participants. The proliferation of local generation, demand response, and electrification of heat and transport are changing the fundamental drivers of electricity load and increasing the complexity of load modelling and forecasting. We address this challenge in two ways. First, our setting is adaptive; our models take into account the most recent observations available, yielding a forecasting strategy able to automatically respond to changes in the underlying process. Second, we consider probabilistic rather than point forecasting; indeed, uncertainty quantification is required to operate electricity systems efficiently and reliably. Our methodology relies on the Kalman filter, previously used successfully for adaptive point load forecasting. The probabilistic forecasts are obtained by quantile regressions on the residuals of the point forecasting model. We achieve adaptive quantile regressions using the online gradient descent; we avoid the choice of the gradient step size considering multiple learning rates and aggregation of experts. We apply the method to two data sets: the regional net-load in Great Britain and the demand of seven large cities in the United States. Adaptive procedures improve forecast performance substantially in both use cases for both point and probabilistic forecasting. △ Less

Submitted 24 April, 2023; v1 submitted 24 January, 2023; originally announced January 2023.

Adaptive Conformal Predictions for Time Series

Authors: Margaux Zaffran, Aymeric Dieuleveut, Olivier Féron, Yannig Goude, Julie Josse

Abstract: Uncertainty quantification of predictive models is crucial in decision-making problems. Conformal prediction is a general and theoretically sound answer. However, it requires exchangeable data, excluding time series. While recent works tackled this issue, we argue that Adaptive Conformal Inference (ACI, Gibbs and Cand{è}s, 2021), developed for distribution-shift time series, is a good procedure fo… ▽ More Uncertainty quantification of predictive models is crucial in decision-making problems. Conformal prediction is a general and theoretically sound answer. However, it requires exchangeable data, excluding time series. While recent works tackled this issue, we argue that Adaptive Conformal Inference (ACI, Gibbs and Cand{è}s, 2021), developed for distribution-shift time series, is a good procedure for time series with general dependency. We theoretically analyse the impact of the learning rate on its efficiency in the exchangeable and auto-regressive case. We propose a parameter-free method, AgACI, that adaptively builds upon ACI based on online expert aggregation. We lead extensive fair simulations against competing methods that advocate for ACI's use in time series. We conduct a real case study: electricity price forecasting. The proposed aggregation algorithm provides efficient prediction intervals for day-ahead forecasting. All the code and data to reproduce the experiments is made available. △ Less

Submitted 15 February, 2022; originally announced February 2022.

Daily peak electrical load forecasting with a multi-resolution approach

Authors: Yvenn Amara-Ouali, Matteo Fasiolo, Yannig Goude, Hui Yan

Abstract: In the context of smart grids and load balancing, daily peak load forecasting has become a critical activity for stakeholders of the energy industry. An understanding of peak magnitude and timing is paramount for the implementation of smart grid strategies such as peak shaving. The modelling approach proposed in this paper leverages high-resolution and low-resolution information to forecast daily… ▽ More In the context of smart grids and load balancing, daily peak load forecasting has become a critical activity for stakeholders of the energy industry. An understanding of peak magnitude and timing is paramount for the implementation of smart grid strategies such as peak shaving. The modelling approach proposed in this paper leverages high-resolution and low-resolution information to forecast daily peak demand size and timing. The resulting multi-resolution modelling framework can be adapted to different model classes. The key contributions of this paper are a) a general and formal introduction to the multi-resolution modelling approach, b) a discussion on modelling approaches at different resolutions implemented via Generalised Additive Models and Neural Networks and c) experimental results on real data from the UK electricity market. The results confirm that the predictive performance of the proposed modelling approach is competitive with that of low- and high-resolution alternatives. △ Less

Submitted 8 December, 2021; originally announced December 2021.

Hierarchical transfer learning with applications for electricity load forecasting

Authors: Anestis Antoniadis, Solenne Gaucher, Yannig Goude

Abstract: The recent abundance of data on electricity consumption at different scales opens new challenges and highlights the need for new techniques to leverage information present at finer scales in order to improve forecasts at wider scales. In this work, we take advantage of the similarity between this hierarchical prediction problem and multi-scale transfer learning. We develop two methods for hierarch… ▽ More The recent abundance of data on electricity consumption at different scales opens new challenges and highlights the need for new techniques to leverage information present at finer scales in order to improve forecasts at wider scales. In this work, we take advantage of the similarity between this hierarchical prediction problem and multi-scale transfer learning. We develop two methods for hierarchical transfer learning, based respectively on the stacking of generalized additive models and random forests, and on the use of aggregation of experts. We apply these methods to two problems of electricity load forecasting at national scale, using smart meter data in the first case, and regional data in the second case. For these two usecases, we compare the performances of our methods to that of benchmark algorithms, and we investigate their behaviour using variable importance analysis. Our results demonstrate the interest of both methods, which lead to a significant improvement of the predictions. △ Less

Submitted 22 November, 2022; v1 submitted 16 November, 2021; originally announced November 2021.

State-Space Models Win the IEEE DataPort Competition on Post-covid Day-ahead Electricity Load Forecasting

Authors: Joseph de Vilmarest, Yannig Goude

Abstract: We present the winning strategy of an electricity demand forecasting competition. This competition was organized to design new forecasting methods for unstable periods such as the one starting in Spring 2020. We rely on state-space models to adapt standard statistical and machine learning models. We claim that it achieves the right compromise between two extremes. On the one hand, purely time-seri… ▽ More We present the winning strategy of an electricity demand forecasting competition. This competition was organized to design new forecasting methods for unstable periods such as the one starting in Spring 2020. We rely on state-space models to adapt standard statistical and machine learning models. We claim that it achieves the right compromise between two extremes. On the one hand, purely time-series models such as autoregressives are adaptive in essence but fail to capture dependence to exogenous variables. On the other hand, machine learning methods allow to learn complex dependence to explanatory variables on a historical data set but fail to forecast non-stationary data accurately. The evaluation period of the competition was the occasion of trial and error and we put the focus on the final forecasting procedure. In particular, it was at the same time that a recent algorithm was designed to adapt the variances of a state-space model and we present the results of the final version only. We discuss day-today predictions nonetheless. △ Less

Submitted 1 October, 2021; originally announced October 2021.

Transfer Learning for Linear Regression: a Statistical Test of Gain

Authors: David Obst, Badih Ghattas, Jairo Cugliari, Georges Oppenheim, Sandra Claudel, Yannig Goude

Abstract: Transfer learning, also referred as knowledge transfer, aims at reusing knowledge from a source dataset to a similar target one. While many empirical studies illustrate the benefits of transfer learning, few theoretical results are established especially for regression problems. In this paper a theoretical framework for the problem of parameter transfer for the linear model is proposed. It is show… ▽ More Transfer learning, also referred as knowledge transfer, aims at reusing knowledge from a source dataset to a similar target one. While many empirical studies illustrate the benefits of transfer learning, few theoretical results are established especially for regression problems. In this paper a theoretical framework for the problem of parameter transfer for the linear model is proposed. It is shown that the quality of transfer for a new input vector $x$ depends on its representation in an eigenbasis involving the parameters of the problem. Furthermore a statistical test is constructed to predict whether a fine-tuned model has a lower prediction quadratic risk than the base target model for an unobserved sample. Efficiency of the test is illustrated on synthetic data as well as real electricity consumption data. △ Less

Submitted 18 February, 2021; originally announced February 2021.

Adaptive Methods for Short-Term Electricity Load Forecasting During COVID-19 Lockdown in France

Authors: David Obst, Joseph de Vilmarest, Yannig Goude

Abstract: The coronavirus disease 2019 (COVID-19) pandemic has urged many governments in the world to enforce a strict lockdown where all nonessential businesses are closed and citizens are ordered to stay at home. One of the consequences of this policy is a significant change in electricity consumption patterns. Since load forecasting models rely on calendar or meteorological information and are trained on… ▽ More The coronavirus disease 2019 (COVID-19) pandemic has urged many governments in the world to enforce a strict lockdown where all nonessential businesses are closed and citizens are ordered to stay at home. One of the consequences of this policy is a significant change in electricity consumption patterns. Since load forecasting models rely on calendar or meteorological information and are trained on historical data, they fail to capture the significant break caused by the lockdown and have exhibited poor performances since the beginning of the pandemic. This makes the scheduling of the electricity production challenging, and has a high cost for both electricity producers and grid operators. In this paper we introduce adaptive generalized additive models using Kalman filters and fine-tuning to adjust to new electricity consumption patterns. Additionally, knowledge from the lockdown in Italy is transferred to anticipate the change of behavior in France. The proposed methods are applied to forecast the electricity demand during the French lockdown period, where they demonstrate their ability to significantly reduce prediction errors compared to traditional models. Finally expert aggregation is used to leverage the specificities of each predictions and enhance results even further. △ Less

Submitted 14 September, 2020; originally announced September 2020.

Probabilistic Forecasting for Daily Electricity Loads and Quantiles for Curve-to-Curve Regression

Authors: Xiuqin Xu, Ying Chen, Yannig Goude, Qiwei Yao

Abstract: Probabilistic forecasting of electricity load curves is of fundamental importance for effective scheduling and decision making in the increasingly volatile and competitive energy markets. We propose a novel approach to construct probabilistic predictors for curves (PPC), which leads to a natural and new definition of quantiles in the context of curve-to-curve linear regression. There are three typ… ▽ More Probabilistic forecasting of electricity load curves is of fundamental importance for effective scheduling and decision making in the increasingly volatile and competitive energy markets. We propose a novel approach to construct probabilistic predictors for curves (PPC), which leads to a natural and new definition of quantiles in the context of curve-to-curve linear regression. There are three types of PPC: a predictive set, a predictive band and a predictive quantile, all of which are defined at a pre-specified nominal probability level. In the simulation study, the PPC achieve promising coverage probabilities under a variety of data generating mechanisms. When applying to one day ahead forecasting for the French daily electricity load curves, PPC outperform several state-of-the-art predictive methods in terms of forecasting accuracy, coverage rate and average length of the predictive bands. The predictive quantile curves provide insightful information which is highly relevant to hedging risks in electricity supply management. △ Less

Submitted 10 November, 2020; v1 submitted 3 September, 2020; originally announced September 2020.

qgam: Bayesian non-parametric quantile regression modelling in R

Authors: Matteo Fasiolo, Simon N. Wood, Margaux Zaffran, Raphaël Nedellec, Yannig Goude

Abstract: Generalized additive models (GAMs) are flexible non-linear regression models, which can be fitted efficiently using the approximate Bayesian methods provided by the mgcv R package. While the GAM methods provided by mgcv are based on the assumption that the response distribution is modelled parametrically, here we discuss more flexible methods that do not entail any parametric assumption. In partic… ▽ More Generalized additive models (GAMs) are flexible non-linear regression models, which can be fitted efficiently using the approximate Bayesian methods provided by the mgcv R package. While the GAM methods provided by mgcv are based on the assumption that the response distribution is modelled parametrically, here we discuss more flexible methods that do not entail any parametric assumption. In particular, this article introduces the qgam package, which is an extension of mgcv providing fast calibrated Bayesian methods for fitting quantile GAMs (QGAMs) in R. QGAMs are based on a smooth version of the pinball loss of Koenker (2005), rather than on a likelihood function, hence jointly achieving satisfactory accuracy of the quantile point estimates and coverage of the corresponding credible intervals requires adopting the specialized Bayesian fitting framework of Fasiolo, Wood, Zaffran, Nedellec, and Goude (2020b). Here we detail how this framework is implemented in qgam and we provide examples illustrating how the package should be used in practice. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Additive stacking for disaggregate electricity demand forecasting

Authors: Christian Capezza, Biagio Palumbo, Yannig Goude, Simon N. Wood, Matteo Fasiolo

Abstract: Future grid management systems will coordinate distributed production and storage resources to manage, in a cost effective fashion, the increased load and variability brought by the electrification of transportation and by a higher share of weather dependent production. Electricity demand forecasts at a low level of aggregation will be key inputs for such systems. We focus on forecasting demand at… ▽ More Future grid management systems will coordinate distributed production and storage resources to manage, in a cost effective fashion, the increased load and variability brought by the electrification of transportation and by a higher share of weather dependent production. Electricity demand forecasts at a low level of aggregation will be key inputs for such systems. We focus on forecasting demand at the individual household level, which is more challenging than forecasting aggregate demand, due to the lower signal-to-noise ratio and to the heterogeneity of consumption patterns across households. We propose a new ensemble method for probabilistic forecasting, which borrows strength across the households while accommodating their individual idiosyncrasies. In particular, we develop a set of models or 'experts' which capture different demand dynamics and we fit each of them to the data from each household. Then we construct an aggregation of experts where the ensemble weights are estimated on the whole data set, the main innovation being that we let the weights vary with the covariates by adopting an additive model structure. In particular, the proposed aggregation method is an extension of regression stacking (Breiman, 1996) where the mixture weights are modelled using linear combinations of parametric, smooth or random effects. The methods for building and fitting additive stacking models are implemented by the gamFactory R package, available at https://github.com/mfasiolo/gamFactory. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Kalman Recursions Aggregated Online

Authors: Eric Adjakossa, Yannig Goude, Olivier Wintenberger

Abstract: In this article, we aim at improving the prediction of expert aggregation by using the underlying properties of the models that provide expert predictions. We restrict ourselves to the case where expert predictions come from Kalman recursions, fitting state-space models. By using exponential weights, we construct different algorithms of Kalman recursions Aggregated Online (KAO) that compete with t… ▽ More In this article, we aim at improving the prediction of expert aggregation by using the underlying properties of the models that provide expert predictions. We restrict ourselves to the case where expert predictions come from Kalman recursions, fitting state-space models. By using exponential weights, we construct different algorithms of Kalman recursions Aggregated Online (KAO) that compete with the best expert or the best convex combination of experts in a more or less adaptive way. We improve the existing results on expert aggregation literature when the experts are Kalman recursions by taking advantage of the second-order properties of the Kalman recursions. We apply our approach to Kalman recursions and extend it to the general adversarial expert setting by state-space modeling the errors of the experts. We apply these new algorithms to a real dataset of electricity consumption and show how it can improve forecast performances comparing to other exponentially weighted average procedures. △ Less

Submitted 26 February, 2020; originally announced February 2020.

Textual Data for Time Series Forecasting

Authors: David Obst, Badih Ghattas, Sandra Claudel, Jairo Cugliari, Yannig Goude, Georges Oppenheim

Abstract: While ubiquitous, textual sources of information such as company reports, social media posts, etc. are hardly included in prediction algorithms for time series, despite the relevant information they may contain. In this work, openly accessible daily weather reports from France and the United-Kingdom are leveraged to predict time series of national electricity consumption, average temperature and w… ▽ More While ubiquitous, textual sources of information such as company reports, social media posts, etc. are hardly included in prediction algorithms for time series, despite the relevant information they may contain. In this work, openly accessible daily weather reports from France and the United-Kingdom are leveraged to predict time series of national electricity consumption, average temperature and wind-speed with a single pipeline. Two methods of numerical representation of text are considered, namely traditional Term Frequency - Inverse Document Frequency (TF-IDF) as well as our own neural word embedding. Using exclusively text, we are able to predict the aforementioned time series with sufficient accuracy to be used to replace missing data. Furthermore the proposed word embeddings display geometric properties relating to the behavior of the time series and context similarity between words. △ Less

Submitted 29 October, 2019; v1 submitted 25 October, 2019; originally announced October 2019.

Comments: -Added e-mail addresses of authors. -Added author who didn't appear on the paper's arXiv page

Target Tracking for Contextual Bandits: Application to Demand Side Management

Authors: Margaux Brégère, Pierre Gaillard, Yannig Goude, Gilles Stoltz

Abstract: We propose a contextual-bandit approach for demand side management by offering price incentives. More precisely, a target mean consumption is set at each round and the mean consumption is modeled as a complex function of the distribution of prices sent and of some contextual variables such as the temperature, weather, and so on. The performance of our strategies is measured in quadratic losses thr… ▽ More We propose a contextual-bandit approach for demand side management by offering price incentives. More precisely, a target mean consumption is set at each round and the mean consumption is modeled as a complex function of the distribution of prices sent and of some contextual variables such as the temperature, weather, and so on. The performance of our strategies is measured in quadratic losses through a regret criterion. We offer $T^{2/3}$ upper bounds on this regret (up to poly-logarithmic terms)---and even faster rates under stronger assumptions---for strategies inspired by standard strategies for contextual bandits (like LinUCB, see Li et al., 2010). Simulations on a real data set gathered by UK Power Networks, in which price incentives were offered, show that our strategies are effective and may indeed manage demand response by suitably picking the price levels. △ Less

Submitted 13 May, 2019; v1 submitted 28 January, 2019; originally announced January 2019.

Journal ref: ICML 2019 (Thirty-sixth International Conference on Machine Learning), Jun 2019, Long Beach, United States

Scalable visualisation methods for modern Generalized Additive Models

Authors: Matteo Fasiolo, Raphaël Nedellec, Yannig Goude, Simon N. Wood

Abstract: In the last two decades the growth of computational resources has made it possible to handle Generalized Additive Models (GAMs) that formerly were too costly for serious applications. However, the growth in model complexity has not been matched by improved visualisations for model development and results presentation. Motivated by an industrial application in electricity load forecasting, we ident… ▽ More In the last two decades the growth of computational resources has made it possible to handle Generalized Additive Models (GAMs) that formerly were too costly for serious applications. However, the growth in model complexity has not been matched by improved visualisations for model development and results presentation. Motivated by an industrial application in electricity load forecasting, we identify the areas where the lack of modern visualisation tools for GAMs is particularly severe, and we address the shortcomings of existing methods by proposing a set of visual tools that a) are fast enough for interactive use, b) exploit the additive structure of GAMs, c) scale to large data sets and d) can be used in conjunction with a wide range of response distributions. All the new visual methods proposed in this work are implemented by the mgcViz R package, which can be found on the Comprehensive R Archive Network. △ Less

Submitted 9 May, 2019; v1 submitted 27 September, 2018; originally announced September 2018.

Nonnegative matrix factorization with side information for time series recovery and prediction

Authors: Jiali Mei, Yohann De Castro, Yannig Goude, Jean-Marc Azaïs, Georges Hébrail

Abstract: Motivated by the reconstruction and the prediction of electricity consumption, we extend Nonnegative Matrix Factorization~(NMF) to take into account side information (column or row features). We consider general linear measurement settings, and propose a framework which models non-linear relationships between features and the response variables. We extend previous theoretical results to obtain a s… ▽ More Motivated by the reconstruction and the prediction of electricity consumption, we extend Nonnegative Matrix Factorization~(NMF) to take into account side information (column or row features). We consider general linear measurement settings, and propose a framework which models non-linear relationships between features and the response variables. We extend previous theoretical results to obtain a sufficient condition on the identifiability of the NMF in this setting. Based the classical Hierarchical Alternating Least Squares~(HALS) algorithm, we propose a new algorithm (HALSX, or Hierarchical Alternating Least Squares with eXogeneous variables) which estimates the factorization model. The algorithm is validated on both simulated and real electricity consumption datasets as well as a recommendation dataset, to show its performance in matrix recovery and prediction for new rows and columns. △ Less

Submitted 19 September, 2017; originally announced September 2017.

Fast calibrated additive quantile regression

Authors: M. Fasiolo, S. N. Wood, M. Zaffran, R. Nedellec, Y. Goude

Abstract: We propose a novel framework for fitting additive quantile regression models, which provides well calibrated inference about the conditional quantiles and fast automatic estimation of the smoothing parameters, for model structures as diverse as those usable with distributional GAMs, while maintaining equivalent numerical efficiency and stability. The proposed methods are at once statistically rigo… ▽ More We propose a novel framework for fitting additive quantile regression models, which provides well calibrated inference about the conditional quantiles and fast automatic estimation of the smoothing parameters, for model structures as diverse as those usable with distributional GAMs, while maintaining equivalent numerical efficiency and stability. The proposed methods are at once statistically rigorous and computationally efficient, because they are based on the general belief updating framework of Bissiri et al. (2016) to loss based inference, but compute by adapting the stable fitting methods of Wood et al. (2016). We show how the pinball loss is statistically suboptimal relative to a novel smooth generalisation, which also gives access to fast estimation methods. Further, we provide a novel calibration method for efficiently selecting the 'learning rate' balancing the loss with the smoothing priors during inference, thereby obtaining reliable quantile uncertainty estimates. Our work was motivated by a probabilistic electricity load forecasting application, used here to demonstrate the proposed approach. The methods described here are implemented by the qgam R package, available on the Comprehensive R Archive Network (CRAN). △ Less

Submitted 12 March, 2020; v1 submitted 11 July, 2017; originally announced July 2017.

Modelling and forecasting daily electricity load curves: a hybrid approach

Authors: Haeran Cho, Yannig Goude, Xavier Brossat, Qiwei Yao

Abstract: We propose a hybrid approach for the modelling and the short-term forecasting of electricity loads. Two building blocks of our approach are (i) modelling the overall trend and seasonality by fitting a generalised additive model to the weekly averages of the load, and (ii) modelling the dependence structure across consecutive daily loads via curve linear regression. For the latter, a new methodolog… ▽ More We propose a hybrid approach for the modelling and the short-term forecasting of electricity loads. Two building blocks of our approach are (i) modelling the overall trend and seasonality by fitting a generalised additive model to the weekly averages of the load, and (ii) modelling the dependence structure across consecutive daily loads via curve linear regression. For the latter, a new methodology is proposed for linear regression with both curve response and curve regressors. The key idea behind the proposed methodology is the dimension reduction based on a singular value decomposition in a Hilbert space, which reduces the curve regression problem to several ordinary (i.e. scalar) linear regression problems. We illustrate the hybrid method using the French electricity loads between 1996 and 2009, on which we also compare our method with other available models including the EDF operational model. △ Less

Submitted 25 November, 2016; originally announced November 2016.

Journal ref: Journal Of The American Statistical Association Vol. 108 , Iss. 501, 2013

Recovering Multiple Nonnegative Time Series From a Few Temporal Aggregates

Authors: Jiali Mei, Yohann De Castro, Yannig Goude, Georges Hébrail

Abstract: Motivated by electricity consumption metering, we extend existing nonnegative matrix factorization (NMF) algorithms to use linear measurements as observations, instead of matrix entries. The objective is to estimate multiple time series at a fine temporal scale from temporal aggregates measured on each individual series. Furthermore, our algorithm is extended to take into account individual autoco… ▽ More Motivated by electricity consumption metering, we extend existing nonnegative matrix factorization (NMF) algorithms to use linear measurements as observations, instead of matrix entries. The objective is to estimate multiple time series at a fine temporal scale from temporal aggregates measured on each individual series. Furthermore, our algorithm is extended to take into account individual autocorrelation to provide better estimation, using a recent convex relaxation of quadratically constrained quadratic program. Extensive experiments on synthetic and real-world electricity consumption datasets illustrate the effectiveness of our matrix recovery algorithms. △ Less

Submitted 5 October, 2016; originally announced October 2016.

Clustering electricity consumers using high-dimensional regression mixture models

Authors: Emilie Devijver, Yannig Goude, Jean-Michel Poggi

Abstract: Massive informations about individual (household, small and medium enterprise) consumption are now provided with new metering technologies and the smart grid. Two major exploitations of these data are load profiling and forecasting at different scales on the grid. Customer segmentation based on load classification is a natural approach for these purposes. We propose here a new methodology based on… ▽ More Massive informations about individual (household, small and medium enterprise) consumption are now provided with new metering technologies and the smart grid. Two major exploitations of these data are load profiling and forecasting at different scales on the grid. Customer segmentation based on load classification is a natural approach for these purposes. We propose here a new methodology based on mixture of high-dimensional regression models. The novelty of our approach is that we focus on uncovering classes or clusters corresponding to different regression models. As a consequence, these classes could then be exploited for profiling as well as forecasting in each class or for bottom-up forecasts in a unified view. We consider a real dataset of Irish individual consumers of 4,225 meters, each with 48 half-hourly meter reads per day over 1 year: from 1st January 2010 up to 31st December 2010, to demonstrate the feasibility of our approach. △ Less

Submitted 1 July, 2015; originally announced July 2015.

Forecasting electricity consumption by aggregating specialized experts

Authors: Marie Devaine, Pierre Gaillard, Yannig Goude, Gilles Stoltz

Abstract: We consider the setting of sequential prediction of arbitrary sequences based on specialized experts. We first provide a review of the relevant literature and present two theoretical contributions: a general analysis of the specialist aggregation rule of Freund et al. (1997) and an adaptation of fixed-share rules of Herbster and Warmuth (1998) in this setting. We then apply these rules to the sequ… ▽ More We consider the setting of sequential prediction of arbitrary sequences based on specialized experts. We first provide a review of the relevant literature and present two theoretical contributions: a general analysis of the specialist aggregation rule of Freund et al. (1997) and an adaptation of fixed-share rules of Herbster and Warmuth (1998) in this setting. We then apply these rules to the sequential short-term (one-day-ahead) forecasting of electricity consumption; to do so, we consider two data sets, a Slovakian one and a French one, respectively concerned with hourly and half-hourly predictions. We follow a general methodology to perform the stated empirical studies and detail in particular tuning issues of the learning parameters. The introduced aggregation rules demonstrate an improved accuracy on the data sets at hand; the improvements lie in a reduced mean squared error but also in a more robust behavior with respect to large occasional errors. △ Less

Submitted 9 July, 2012; originally announced July 2012.

Comments: 33 pages