-
Distributional neural networks for electricity price forecasting
Authors:
Grzegorz Marcjasz,
Michał Narajewski,
Rafał Weron,
Florian Ziel
Abstract:
We present a novel approach to probabilistic electricity price forecasting which utilizes distributional neural networks. The model structure is based on a deep neural network that contains a so-called probability layer. The network's output is a parametric distribution with 2 (normal) or 4 (Johnson's SU) parameters. In a forecasting study involving day-ahead electricity prices in the German marke…
▽ More
We present a novel approach to probabilistic electricity price forecasting which utilizes distributional neural networks. The model structure is based on a deep neural network that contains a so-called probability layer. The network's output is a parametric distribution with 2 (normal) or 4 (Johnson's SU) parameters. In a forecasting study involving day-ahead electricity prices in the German market, our approach significantly outperforms state-of-the-art benchmarks, including LASSO-estimated regressions and deep neural networks combined with Quantile Regression Averaging. The obtained results not only emphasize the importance of higher moments when modeling volatile electricity prices, but also -- given that probabilistic forecasting is the essence of risk management -- provide important implications for managing portfolios in the power sector.
△ Less
Submitted 10 December, 2022; v1 submitted 6 July, 2022;
originally announced July 2022.
-
Probabilistic forecasting of German electricity imbalance prices
Authors:
Michał Narajewski
Abstract:
The exponential growth of renewable energy capacity has brought much uncertainty to electricity prices and to electricity generation. To address this challenge, the energy exchanges have been develo** further trading possibilities, especially the intraday and balancing markets. For an energy trader participating in both markets, the forecasting of imbalance prices is of particular interest. Ther…
▽ More
The exponential growth of renewable energy capacity has brought much uncertainty to electricity prices and to electricity generation. To address this challenge, the energy exchanges have been develo** further trading possibilities, especially the intraday and balancing markets. For an energy trader participating in both markets, the forecasting of imbalance prices is of particular interest. Therefore, in this manuscript we conduct a very short-term probabilistic forecasting of imbalance prices, contributing to the scarce literature in this novel subject. The forecasting is performed 30 minutes before the delivery, so that the trader might still choose the trading place. The distribution of the imbalance prices is modelled and forecasted using methods well-known in the electricity price forecasting literature: lasso with bootstrap, gamlss, and probabilistic neural networks. The methods are compared with a naive benchmark in a meaningful rolling window study. The results provide evidence of the efficiency between the intraday and balancing markets as the sophisticated methods do not substantially overperform the intraday continuous price index. On the other hand, they significantly improve the empirical coverage. The analysis was conducted on the German market, however it could be easily applied to any other market of similar structure.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks
Authors:
Jonathan Berrisch,
Michał Narajewski,
Florian Ziel
Abstract:
This paper covers predicting high-resolution electricity peak demand features given lower-resolution data. This is a relevant setup as it answers whether limited higher-resolution monitoring helps to estimate future high-resolution peak loads when the high-resolution data is no longer available. That question is particularly interesting for network operators considering replacing high-resolution m…
▽ More
This paper covers predicting high-resolution electricity peak demand features given lower-resolution data. This is a relevant setup as it answers whether limited higher-resolution monitoring helps to estimate future high-resolution peak loads when the high-resolution data is no longer available. That question is particularly interesting for network operators considering replacing high-resolution monitoring predictive models due to economic considerations. We propose models to predict half-hourly minima and maxima of high-resolution (every minute) electricity load data while model inputs are of a lower resolution (30 minutes). We combine predictions of generalized additive models (GAM) and deep artificial neural networks (DNN), which are popular in load forecasting. We extensively analyze the prediction models, including the input parameters' importance, focusing on load, weather, and seasonal effects. The proposed method won a data competition organized by Western Power Distribution, a British distribution network operator. In addition, we provide a rigorous evaluation study that goes beyond the competition frame to analyze the models' robustness. The results show that the proposed methods are superior to the competition benchmark concerning the out-of-sample root mean squared error (RMSE). This holds regarding the competition month and the supplementary evaluation study, which covers an additional eleven months. Overall, our proposed model combination reduces the out-of-sample RMSE by 57.4\% compared to the benchmark.
△ Less
Submitted 2 November, 2022; v1 submitted 7 March, 2022;
originally announced March 2022.
-
Optimal bidding in hourly and quarter-hourly electricity price auctions: trading large volumes of power with market impact and transaction costs
Authors:
Michał Narajewski,
Florian Ziel
Abstract:
This paper addresses the question of how much to bid to maximize the profit when trading in two electricity markets: the hourly Day-Ahead Auction and the quarter-hourly Intraday Auction. For optimal coordinated bidding many price scenarios are examined, the own non-linear market impact is estimated by considering empirical supply and demand curves, and a number of trading strategies is used. Addit…
▽ More
This paper addresses the question of how much to bid to maximize the profit when trading in two electricity markets: the hourly Day-Ahead Auction and the quarter-hourly Intraday Auction. For optimal coordinated bidding many price scenarios are examined, the own non-linear market impact is estimated by considering empirical supply and demand curves, and a number of trading strategies is used. Additionally, we provide theoretical results for risk neutral agents. The application study is conducted using the German market data, but the presented methods can be easily utilized with other two consecutive auctions. This paper contributes to the existing literature by evaluating the costs of electricity trading, i.e. the price impact and the transaction costs. The empirical results for the German EPEX market show that it is far more profitable to minimize the price impact rather than maximize the arbitrage.
△ Less
Submitted 9 February, 2022; v1 submitted 29 April, 2021;
originally announced April 2021.
-
tsrobprep - an R package for robust preprocessing of time series data
Authors:
Michał Narajewski,
Jens Kley-Holsteg,
Florian Ziel
Abstract:
Data cleaning is a crucial part of every data analysis exercise. Yet, the currently available R packages do not provide fast and robust methods for cleaning and preparation of time series data. The open source package tsrobprep introduces efficient methods for handling missing values and outliers using model based approaches. For data imputation a probabilistic replacement model is proposed, which…
▽ More
Data cleaning is a crucial part of every data analysis exercise. Yet, the currently available R packages do not provide fast and robust methods for cleaning and preparation of time series data. The open source package tsrobprep introduces efficient methods for handling missing values and outliers using model based approaches. For data imputation a probabilistic replacement model is proposed, which may consist of autoregressive components and external inputs. For outlier detection a clustering algorithm based on finite mixture modelling is introduced, which considers time series properties in terms of the gradient and the underlying seasonality as features. The procedure allows to return a probability for each observation being outlying data as well as a specific cause for an outlier assignment in terms of the provided feature space. The methods work robust and are fully tunable. Moreover, by providing the auto_data_cleaning function the data preprocessing can be carried out in one cast, without comprehensive tuning and providing suitable results. The primary motivation of the package is the preprocessing of energy system data. We present application for electricity load, wind and solar power data.
△ Less
Submitted 11 October, 2021; v1 submitted 26 April, 2021;
originally announced April 2021.
-
Ensemble Forecasting for Intraday Electricity Prices: Simulating Trajectories
Authors:
Michał Narajewski,
Florian Ziel
Abstract:
Recent studies concerning the point electricity price forecasting have shown evidence that the hourly German Intraday Continuous Market is weak-form efficient. Therefore, we take a novel, advanced approach to the problem. A probabilistic forecasting of the hourly intraday electricity prices is performed by simulating trajectories in every trading window to receive a realistic ensemble to allow for…
▽ More
Recent studies concerning the point electricity price forecasting have shown evidence that the hourly German Intraday Continuous Market is weak-form efficient. Therefore, we take a novel, advanced approach to the problem. A probabilistic forecasting of the hourly intraday electricity prices is performed by simulating trajectories in every trading window to receive a realistic ensemble to allow for more efficient intraday trading and redispatch. A generalized additive model is fitted to the price differences with the assumption that they follow a zero-inflated distribution, precisely a mixture of the Dirac and the Student's t-distributions. Moreover, the mixing term is estimated using a high-dimensional logistic regression with lasso penalty. We model the expected value and volatility of the series using i.a. autoregressive and no-trade effects or load, wind and solar generation forecasts and accounting for the non-linearities in e.g. time to maturity. Both the in-sample characteristics and forecasting performance are analysed using a rolling window forecasting study. Multiple versions of the model are compared to several benchmark models and evaluated using probabilistic forecasting measures and significance tests. The study aims to forecast the price distribution in the German Intraday Continuous Market in the last 3 hours of trading, but the approach allows for application to other continuous markets, especially in Europe. The results prove superiority of the mixture model over the benchmarks gaining the most from the modelling of the volatility. They also indicate that the introduction of XBID reduced the market volatility.
△ Less
Submitted 29 August, 2020; v1 submitted 4 May, 2020;
originally announced May 2020.
-
Changes in electricity demand pattern in Europe due to COVID-19 shutdowns
Authors:
Michał Narajewski,
Florian Ziel
Abstract:
The article covers electricity demand shift effects due to COVID-19 shutdowns in various European countries. We utilize high-dimensional regression techniques to exploit the structural breaks in demand profiles due to the shutdowns. We discuss the findings with respect to coronavirus pandemic progress and regulatory measures of the considered countries.
The article covers electricity demand shift effects due to COVID-19 shutdowns in various European countries. We utilize high-dimensional regression techniques to exploit the structural breaks in demand profiles due to the shutdowns. We discuss the findings with respect to coronavirus pandemic progress and regulatory measures of the considered countries.
△ Less
Submitted 5 May, 2020; v1 submitted 29 April, 2020;
originally announced April 2020.
-
Estimation and simulation of the transaction arrival process in intraday electricity markets
Authors:
Michał Narajewski,
Florian Ziel
Abstract:
We examine the novel problem of the estimation of transaction arrival processes in the intraday electricity markets. We model the inter-arrivals using multiple time-varying parametric densities based on the generalized F distribution estimated by maximum likelihood. We analyse both the in-sample characteristics and the probabilistic forecasting performance. In a rolling window forecasting study, w…
▽ More
We examine the novel problem of the estimation of transaction arrival processes in the intraday electricity markets. We model the inter-arrivals using multiple time-varying parametric densities based on the generalized F distribution estimated by maximum likelihood. We analyse both the in-sample characteristics and the probabilistic forecasting performance. In a rolling window forecasting study, we simulate many trajectories to evaluate the forecasts and gain significant insights into the model fit. The prediction accuracy is evaluated by a functional version of the MAE (mean absolute error), RMSE (root mean squared error) and CRPS (continuous ranked probability score) for the simulated count processes. This paper fills the gap in the literature regarding the intensity estimation of transaction arrivals and is a major contribution to the topic, yet leaves much of the field for further development. The study presented in this paper is conducted based on the German Intraday Continuous electricity market data, but this method can be easily applied to any other continuous intraday electricity market. For the German market, a specific generalized gamma distribution setup explains the overall behaviour significantly best, especially as the tail behaviour of the process is well covered.
△ Less
Submitted 2 December, 2019; v1 submitted 28 January, 2019;
originally announced January 2019.
-
Econometric modelling and forecasting of intraday electricity prices
Authors:
Michał Narajewski,
Florian Ziel
Abstract:
In the following paper, we analyse the ID$_3$-Price in the German Intraday Continuous electricity market using an econometric time series model. A multivariate approach is conducted for hourly and quarter-hourly products separately. We estimate the model using lasso and elastic net techniques and perform an out-of-sample, very short-term forecasting study. The model's performance is compared with…
▽ More
In the following paper, we analyse the ID$_3$-Price in the German Intraday Continuous electricity market using an econometric time series model. A multivariate approach is conducted for hourly and quarter-hourly products separately. We estimate the model using lasso and elastic net techniques and perform an out-of-sample, very short-term forecasting study. The model's performance is compared with benchmark models and is discussed in detail. Forecasting results provide new insights to the German Intraday Continuous electricity market regarding its efficiency and to the ID$_3$-Price behaviour.
△ Less
Submitted 23 September, 2019; v1 submitted 21 December, 2018;
originally announced December 2018.
