-
Leveraging Non-Decimated Wavelet Packet Features and Transformer Models for Time Series Forecasting
Authors:
Guy P Nason,
James L. Wei
Abstract:
This article combines wavelet analysis techniques with machine learning methods for univariate time series forecasting, focusing on three main contributions. Firstly, we consider the use of Daubechies wavelets with different numbers of vanishing moments as input features to both non-temporal and temporal forecasting methods, by selecting these numbers during the cross-validation phase. Secondly, w…
▽ More
This article combines wavelet analysis techniques with machine learning methods for univariate time series forecasting, focusing on three main contributions. Firstly, we consider the use of Daubechies wavelets with different numbers of vanishing moments as input features to both non-temporal and temporal forecasting methods, by selecting these numbers during the cross-validation phase. Secondly, we compare the use of both the non-decimated wavelet transform and the non-decimated wavelet packet transform for computing these features, the latter providing a much larger set of potentially useful coefficient vectors. The wavelet coefficients are computed using a shifted version of the typical pyramidal algorithm to ensure no leakage of future information into these inputs. Thirdly, we evaluate the use of these wavelet features on a significantly wider set of forecasting methods than previous studies, including both temporal and non-temporal models, and both statistical and deep learning-based methods. The latter include state-of-the-art transformer-based neural network architectures. Our experiments suggest significant benefit in replacing higher-order lagged features with wavelet features across all examined non-temporal methods for one-step-forward forecasting, and modest benefit when used as inputs for temporal deep learning-based models for long-horizon forecasting.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
Modelling clusters in network time series with an application to presidential elections in the USA
Authors:
Guy Nason,
Daniel Salnikov,
Mario Cortina-Borja
Abstract:
Network time series are becoming increasingly relevant in the study of dynamic processes characterised by a known or inferred underlying network structure. Generalised Network Autoregressive (GNAR) models provide a parsimonious framework for exploiting the underlying network, even in the high-dimensional setting. We extend the GNAR framework by introducing the $\textit{community}$-$α$ GNAR model t…
▽ More
Network time series are becoming increasingly relevant in the study of dynamic processes characterised by a known or inferred underlying network structure. Generalised Network Autoregressive (GNAR) models provide a parsimonious framework for exploiting the underlying network, even in the high-dimensional setting. We extend the GNAR framework by introducing the $\textit{community}$-$α$ GNAR model that exploits prior knowledge and/or exogenous variables for identifying and modelling dynamic interactions across communities in the underlying network. We further analyse the dynamics of $\textit{Red, Blue}$ and $\textit{Swing}$ states throughout presidential elections in the USA. Our analysis shows that dynamics differ among the state-wise clusters.
△ Less
Submitted 17 January, 2024;
originally announced January 2024.
-
New Methods for Network Count Time Series
Authors:
Hengxu Liu,
Guy Nason
Abstract:
The original generalized network autoregressive models are poor for modelling count data as they are based on the additive and constant noise assumptions, which is usually inappropriate for count data. We introduce two new models (GNARI and NGNAR) for count network time series by adapting and extending existing count-valued time series models. We present results on the statistical and asymptotic p…
▽ More
The original generalized network autoregressive models are poor for modelling count data as they are based on the additive and constant noise assumptions, which is usually inappropriate for count data. We introduce two new models (GNARI and NGNAR) for count network time series by adapting and extending existing count-valued time series models. We present results on the statistical and asymptotic properties of our new models and their estimates obtained by conditional least squares and maximum likelihood. We conduct two simulation studies that verify successful parameter estimation for both models and conduct a further study that shows, for negative network parameters, that our NGNAR model outperforms existing models and our other GNARI model in terms of predictive performance. We model a network time series constructed from COVID-positive counts for counties in New York State during 2020-22 and show that our new models perform considerably better than existing methods for this problem.
△ Less
Submitted 4 December, 2023;
originally announced December 2023.
-
New tools for network time series with an application to COVID-19 hospitalisations
Authors:
Guy Nason,
Daniel Salnikov,
Mario Cortina-Borja
Abstract:
Network time series are becoming increasingly important across many areas in science and medicine and are often characterised by a known or inferred underlying network structure, which can be exploited to make sense of dynamic phenomena that are often high-dimensional. For example, the Generalised Network Autoregressive (GNAR) models exploit such structure parsimoniously. We use the GNAR framework…
▽ More
Network time series are becoming increasingly important across many areas in science and medicine and are often characterised by a known or inferred underlying network structure, which can be exploited to make sense of dynamic phenomena that are often high-dimensional. For example, the Generalised Network Autoregressive (GNAR) models exploit such structure parsimoniously. We use the GNAR framework to introduce two association measures: the network and partial network autocorrelation functions, and introduce Corbit (correlation-orbit) plots for visualisation. As with regular autocorrelation plots, Corbit plots permit interpretation of underlying correlation structures and, crucially, aid model selection more rapidly than using other tools such as AIC or BIC. We additionally interpret GNAR processes as generalised graphical models, which constrain the processes' autoregressive structure and exhibit interesting theoretical connections to graphical models via utilization of higher-order interactions. We demonstrate how incorporation of prior information is related to performing variable selection and shrinkage in the GNAR context. We illustrate the usefulness of the GNAR formulation, network autocorrelations and Corbit plots by modelling a COVID-19 network time series of the number of admissions to mechanical ventilation beds at 140 NHS Trusts in England & Wales. We introduce the Wagner plot that can analyse correlations over different time periods or with respect to external covariates. In addition, we introduce plots that quantify the relevance and influence of individual nodes. Our modelling provides insight on the underlying dynamics of the COVID-19 series, highlights two groups of geographically co-located `influential' NHS Trusts and demonstrates superior prediction abilities when compared to existing techniques.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
-
Automatic Locally Stationary Time Series Forecasting with application to predicting U.K. Gross Value Added Time Series under sudden shocks caused by the COVID pandemic
Authors:
Rebecca Killick,
Marina I. Knight,
Guy P. Nason,
Matthew A. Nunes,
Idris A. Eckley
Abstract:
Accurate forecasting of the U.K. gross value added (GVA) is fundamental for measuring the growth of the U.K. economy. A common nonstationarity in GVA data, such as the ABML series, is its increase in variance over time due to inflation. Transformed or inflation-adjusted series can still be challenging for classical stationarity-assuming forecasters. We adopt a different approach that works directl…
▽ More
Accurate forecasting of the U.K. gross value added (GVA) is fundamental for measuring the growth of the U.K. economy. A common nonstationarity in GVA data, such as the ABML series, is its increase in variance over time due to inflation. Transformed or inflation-adjusted series can still be challenging for classical stationarity-assuming forecasters. We adopt a different approach that works directly with the GVA series by advancing recent forecasting methods for locally stationary time series. Our approach results in more accurate and reliable forecasts, and continues to work well even when the ABML series becomes highly variable during the COVID pandemic.
△ Less
Submitted 14 March, 2023;
originally announced March 2023.
-
Quantifying the economic response to COVID-19 mitigations and death rates via forecasting Purchasing Managers' Indices using Generalised Network Autoregressive models with exogenous variables
Authors:
Guy P Nason,
James L Wei
Abstract:
Knowledge of the current state of economies, how they respond to COVID-19 mitigations and indicators, and what the future might hold for them is important. We use recently-developed generalised network autoregressive (GNAR) models, using trade-determined networks, to model and forecast the Purchasing Managers' Indices for a number of countries. We use networks that link countries where the links t…
▽ More
Knowledge of the current state of economies, how they respond to COVID-19 mitigations and indicators, and what the future might hold for them is important. We use recently-developed generalised network autoregressive (GNAR) models, using trade-determined networks, to model and forecast the Purchasing Managers' Indices for a number of countries. We use networks that link countries where the links themselves, or their weights, are determined by the degree of export trade between the countries. We extend these models to include node-specific time series exogenous variables (GNARX models), using this to incorporate COVID-19 mitigation stringency indices and COVID-19 death rates into our analysis. The highly parsimonious GNAR models considerably outperform vector autoregressive models in terms of mean-squared forecasting error and our GNARX models themselves outperform GNAR ones. Further mixed frequency modelling predicts the extent to which that the UK economy will be affected by harsher, weaker or no interventions.
△ Less
Submitted 14 July, 2021;
originally announced July 2021.
-
The Local Partial Autocorrelation Function and Some Applications
Authors:
Rebecca Killick,
Marina I. Knight,
Guy P. Nason,
Idris A. Eckley
Abstract:
The classical regular and partial autocorrelation functions are powerful tools for stationary time series modelling and analysis. However, it is increasingly recognized that many time series are not stationary and the use of classical global autocorrelations can give misleading answers. This article introduces two estimators of the local partial autocorrelation function and establishes their asymp…
▽ More
The classical regular and partial autocorrelation functions are powerful tools for stationary time series modelling and analysis. However, it is increasingly recognized that many time series are not stationary and the use of classical global autocorrelations can give misleading answers. This article introduces two estimators of the local partial autocorrelation function and establishes their asymptotic properties. The article then illustrates the use of these new estimators on both simulated and real time series. The examples clearly demonstrate the strong practical benefits of local estimators for time series that exhibit nonstationarities.
△ Less
Submitted 27 April, 2020;
originally announced April 2020.
-
Rapidly evaluating lockdown strategies using spectral analysis: the cycles behind new daily COVID-19 cases and what happens after lockdown
Authors:
Guy P. Nason
Abstract:
Spectral analysis characterises oscillatory time series behaviours such as cycles, but accurate estimation requires reasonable numbers of observations. Current COVID-19 time series for many countries are short: pre- and post-lockdown series are shorter still. Accurate estimation of potentially interesting cycles within such series seems beyond reach. We solve the problem of obtaining accurate esti…
▽ More
Spectral analysis characterises oscillatory time series behaviours such as cycles, but accurate estimation requires reasonable numbers of observations. Current COVID-19 time series for many countries are short: pre- and post-lockdown series are shorter still. Accurate estimation of potentially interesting cycles within such series seems beyond reach. We solve the problem of obtaining accurate estimates from short time series by using recent Bayesian spectral fusion methods. Here we show that transformed new daily COVID-19 cases for many countries generally contain three cycles operating at wavelengths of around 2.7, 4.1 and 6.7 days (weekly). We show that the shorter cycles are suppressed after lockdown. The pre- and post lockdown differences suggest that the weekly effect is at least partly due to non-epidemic factors, whereas the two shorter cycles seem intrinsic to the epidemic. Unconstrained, new cases grow exponentially, but the internal cyclic structure causes periodic falls in cases. This suggests that lockdown success might only be indicated by four or more daily falls in cases. Spectral learning for epidemic time series contributes to the understanding of the epidemic process, hel** evaluate interventions and assists with forecasting. Spectral fusion is a general technique that is able to fuse spectra recorded at different sampling rates, which can be applied to a wide range of time series from many disciplines.
△ Less
Submitted 16 April, 2020;
originally announced April 2020.
-
Generalised Network Autoregressive Processes and the GNAR package
Authors:
Marina Knight,
Kathryn Leeming,
Guy Nason,
Matthew Nunes
Abstract:
This article introduces the GNAR package, which fits, predicts, and simulates from a powerful new class of generalised network autoregressive processes. Such processes consist of a multivariate time series along with a real, or inferred, network that provides information about inter-variable relationships. The GNAR model relates values of a time series for a given variable and time to earlier valu…
▽ More
This article introduces the GNAR package, which fits, predicts, and simulates from a powerful new class of generalised network autoregressive processes. Such processes consist of a multivariate time series along with a real, or inferred, network that provides information about inter-variable relationships. The GNAR model relates values of a time series for a given variable and time to earlier values of the same variable and of neighbouring variables, with inclusion controlled by the network structure. The GNAR package is designed to fit this new model, while working with standard ts objects and the igraph package for ease of use.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
Supplementary Material for "Should we sample a time series more frequently? Decision support via multirate spectrum estimation (with discussion)"
Authors:
Guy P. Nason,
Ben Powell,
Duncan Elliott,
Paul A. Smith
Abstract:
This technical report includes an assortment of technical details and extended discussions related to paper "Should we sample a time series more frequently? Decision support via multirate spectrum estimation (with discussion)", which introduces a model for estimating the log-spectral density of a stationary discrete time process given systematically missing data and models the cost implication for…
▽ More
This technical report includes an assortment of technical details and extended discussions related to paper "Should we sample a time series more frequently? Decision support via multirate spectrum estimation (with discussion)", which introduces a model for estimating the log-spectral density of a stationary discrete time process given systematically missing data and models the cost implication for changing the sampling rate.
△ Less
Submitted 22 April, 2016;
originally announced April 2016.
-
Simulation Study Comparing Two Tests of Second-order Stationarity and Confidence Intervals for Localized Autocovariance
Authors:
Guy Nason
Abstract:
This report compares two tests of second-order stationarity through simulation. It also provides several examples of localised autocovariances and their approximate confidence intervals on different real and simulated data sets. An empirical verification of an asymptotic Gaussianity result is also demonstrated. The commands use to produce figures in a companion paper are also described.
This report compares two tests of second-order stationarity through simulation. It also provides several examples of localised autocovariances and their approximate confidence intervals on different real and simulated data sets. An empirical verification of an asymptotic Gaussianity result is also demonstrated. The commands use to produce figures in a companion paper are also described.
△ Less
Submitted 21 March, 2016;
originally announced March 2016.
-
Modelling, Detrending and Decorrelation of Network Time Series
Authors:
M. I. Knight,
M. A. Nunes,
G. P. Nason
Abstract:
A network time series is a multivariate time series augmented by a graph that describes how variables (or nodes) are connected. We introduce the network autoregressive (integrated) moving average (NARIMA) processes: a set of flexible models for network time series. For fixed networks the NARIMA models are essentially equivalent to vector autoregressive moving average-type models. However, NARIMA m…
▽ More
A network time series is a multivariate time series augmented by a graph that describes how variables (or nodes) are connected. We introduce the network autoregressive (integrated) moving average (NARIMA) processes: a set of flexible models for network time series. For fixed networks the NARIMA models are essentially equivalent to vector autoregressive moving average-type models. However, NARIMA models are especially useful when the structure of the graph, associated with the multivariate time series, changes over time. Such network topology changes are invisible to standard VARMA-like models. For integrated NARIMA models we introduce network differencing, based on the network lifting (wavelet) transform, which removes trend. We exhibit our techniques on a network time series describing the evolution of mumps throughout counties of England and Wales weekly during 2005. We further demonstrate the action of network lifting on a simple bivariate VAR(1) model with associated two-node graph. We show theoretically that decorrelation occurs only in certain circumstances and maybe less than expected. This suggests that the time-decorrelation properties of spatial network lifting are due more to the trend removal properties of lifting rather than any kind of stochastic decorrelation.
△ Less
Submitted 10 March, 2016;
originally announced March 2016.
-
Bayesian Wavelet Shrinkage of the Haar-Fisz Transformed Wavelet Periodogram
Authors:
Guy P. Nason,
Kara N. Stevens
Abstract:
It is increasingly being realised that many real world time series are not stationary and exhibit evolving second-order autocovariance or spectral structure. This article introduces a Bayesian approach for modelling the evolving wavelet spectrum of a locally stationary wavelet time series. Our new method works by combining the advantages of a Haar-Fisz transformed spectrum with a simple, but power…
▽ More
It is increasingly being realised that many real world time series are not stationary and exhibit evolving second-order autocovariance or spectral structure. This article introduces a Bayesian approach for modelling the evolving wavelet spectrum of a locally stationary wavelet time series. Our new method works by combining the advantages of a Haar-Fisz transformed spectrum with a simple, but powerful, Bayesian wavelet shrinkage method. Our new method produces excellent and stable spectral estimates and this is demonstrated via simulated data and on differenced infant ECG data. A major additional benefit of the Bayesian paradigm is that we obtain rigorous and useful credible intervals of the evolving spectral structure. We show how the Bayesian credible intervals provide extra insight into the infant ECG data.
△ Less
Submitted 10 September, 2013;
originally announced September 2013.
-
A note on state space representations of locally stationary wavelet time series
Authors:
K. Triantafyllopoulos,
G. P. Nason
Abstract:
In this note we show that the locally stationary wavelet process can be decomposed into a sum of signals, each of which following a moving average process with time-varying parameters. We then show that such moving average processes are equivalent to state space models with stochastic design components. Using a simple simulation step, we propose a heuristic method of estimating the above state s…
▽ More
In this note we show that the locally stationary wavelet process can be decomposed into a sum of signals, each of which following a moving average process with time-varying parameters. We then show that such moving average processes are equivalent to state space models with stochastic design components. Using a simple simulation step, we propose a heuristic method of estimating the above state space models and then we apply the methodology to foreign exchange rates data.
△ Less
Submitted 19 July, 2008;
originally announced July 2008.
