-
Markovian RNN: An Adaptive Time Series Prediction Network with HMM-based Switching for Nonstationary Environments
Authors:
Fatih Ilhan,
Oguzhan Karaahmetoglu,
Ismail Balaban,
Suleyman Serdar Kozat
Abstract:
We investigate nonlinear regression for nonstationary sequential data. In most real-life applications such as business domains including finance, retail, energy and economy, timeseries data exhibits nonstationarity due to the temporally varying dynamics of the underlying system. We introduce a novel recurrent neural network (RNN) architecture, which adaptively switches between internal regimes in…
▽ More
We investigate nonlinear regression for nonstationary sequential data. In most real-life applications such as business domains including finance, retail, energy and economy, timeseries data exhibits nonstationarity due to the temporally varying dynamics of the underlying system. We introduce a novel recurrent neural network (RNN) architecture, which adaptively switches between internal regimes in a Markovian way to model the nonstationary nature of the given data. Our model, Markovian RNN employs a hidden Markov model (HMM) for regime transitions, where each regime controls hidden state transitions of the recurrent cell independently. We jointly optimize the whole network in an end-to-end fashion. We demonstrate the significant performance gains compared to vanilla RNN and conventional methods such as Markov Switching ARIMA through an extensive set of experiments with synthetic and real-life datasets. We also interpret the inferred parameters and regime belief values to analyze the underlying dynamics of the given sequences.
△ Less
Submitted 17 June, 2020;
originally announced June 2020.
-
Unsupervised Online Anomaly Detection On Irregularly Sampled Or Missing Valued Time-Series Data Using LSTM Networks
Authors:
Oguzhan Karaahmetoglu,
Fatih Ilhan,
Ismail Balaban,
Suleyman Serdar Kozat
Abstract:
We study anomaly detection and introduce an algorithm that processes variable length, irregularly sampled sequences or sequences with missing values. Our algorithm is fully unsupervised, however, can be readily extended to supervised or semisupervised cases when the anomaly labels are present as remarked throughout the paper. Our approach uses the Long Short Term Memory (LSTM) networks in order to…
▽ More
We study anomaly detection and introduce an algorithm that processes variable length, irregularly sampled sequences or sequences with missing values. Our algorithm is fully unsupervised, however, can be readily extended to supervised or semisupervised cases when the anomaly labels are present as remarked throughout the paper. Our approach uses the Long Short Term Memory (LSTM) networks in order to extract temporal features and find the most relevant feature vectors for anomaly detection. We incorporate the sampling time information to our model by modulating the standard LSTM model with time modulation gates. After obtaining the most relevant features from the LSTM, we label the sequences using a Support Vector Data Descriptor (SVDD) model. We introduce a loss function and then jointly optimize the feature extraction and sequence processing mechanisms in an end-to-end manner. Through this joint optimization, the LSTM extracts the most relevant features for anomaly detection later to be used in the SVDD, hence completely removes the need for feature selection by expert knowledge. Furthermore, we provide a training algorithm for the online setup, where we optimize our model parameters with individual sequences as the new data arrives. Finally, on real-life datasets, we show that our model significantly outperforms the standard approaches thanks to its combination of LSTM with SVDD and joint optimization.
△ Less
Submitted 25 May, 2020;
originally announced May 2020.
-
Development of a Forecasting and Warning System on the Ecological Life-Cycle of Sunn Pest
Authors:
İsmail Balaban,
Fatih Acun,
Onur Yiğit Arpalı,
Furkan Murat,
Numan Ertuğrul Babaroğlu,
Emre Akci,
Mehmet Çulcu,
Mümtaz Özkan,
Selim Temizer
Abstract:
We provide a machine learning solution that replaces the traditional methods for deciding the pesticide application time of Sunn Pest. We correlate climate data with phases of Sunn Pest in its life-cycle and decide whether the fields should be sprayed. Our solution includes two groups of prediction models. The first group contains decision trees that predict migration time of Sunn Pest from winter…
▽ More
We provide a machine learning solution that replaces the traditional methods for deciding the pesticide application time of Sunn Pest. We correlate climate data with phases of Sunn Pest in its life-cycle and decide whether the fields should be sprayed. Our solution includes two groups of prediction models. The first group contains decision trees that predict migration time of Sunn Pest from winter quarters to wheat fields. The second group contains random forest models that predict the nymphal stage percentages of Sunn Pest which is a criterion for pesticide application. We trained our models on four years of climate data which was collected from Kirşehir and Aksaray. The experiments show that our promised solution make correct predictions with high accuracies.
△ Less
Submitted 5 May, 2019;
originally announced May 2019.
