tsGT: Stochastic Time Series Modeling With Transformer
Authors:
Łukasz Kuciński,
Witold Drzewakowski,
Mateusz Olko,
Piotr Kozakowski,
Łukasz Maziarka,
Marta Emilia Nowakowska,
Łukasz Kaiser,
Piotr Miłoś
Abstract:
Time series methods are of fundamental importance in virtually any field of science that deals with temporally structured data. Recently, there has been a surge of deterministic transformer models with time series-specific architectural biases. In this paper, we go in a different direction by introducing tsGT, a stochastic time series model built on a general-purpose transformer architecture. We f…
▽ More
Time series methods are of fundamental importance in virtually any field of science that deals with temporally structured data. Recently, there has been a surge of deterministic transformer models with time series-specific architectural biases. In this paper, we go in a different direction by introducing tsGT, a stochastic time series model built on a general-purpose transformer architecture. We focus on using a well-known and theoretically justified rolling window backtesting and evaluation protocol. We show that tsGT outperforms the state-of-the-art models on MAD and RMSE, and surpasses its stochastic peers on QL and CRPS, on four commonly used datasets. We complement these results with a detailed analysis of tsGT's ability to model the data distribution and predict marginal quantile values.
△ Less
Submitted 3 April, 2024; v1 submitted 8 March, 2024;
originally announced March 2024.
