Showing 1–20 of 20 results for author: Morvai, G

Estimating the Lengths of Memory Words

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: For a stationary stochastic process $\{X_n\}$ with values in some set $A$, a finite word $w \in A^K$ is called a memory word if the conditional probability of $X_0$ given the past is constant on the cylinder set defined by $X_{-K}^{-1}=w$. It is a called a minimal memory word if no proper suffix of $w$ is also a memory word. For example in a $K$-step Markov processes all words of length $K$ are… ▽ More For a stationary stochastic process $\{X_n\}$ with values in some set $A$, a finite word $w \in A^K$ is called a memory word if the conditional probability of $X_0$ given the past is constant on the cylinder set defined by $X_{-K}^{-1}=w$. It is a called a minimal memory word if no proper suffix of $w$ is also a memory word. For example in a $K$-step Markov processes all words of length $K$ are memory words but not necessarily minimal. We consider the problem of determining the lengths of the longest minimal memory words and the shortest memory words of an unknown process $\{X_n\}$ based on sequentially observing the outputs of a single sample $\{ξ_1,ξ_2,...ξ_n\}$. We will give a universal estimator which converges almost surely to the length of the longest minimal memory word and show that no such universal estimator exists for the length of the shortest memory word. The alphabet $A$ may be finite or countable. △ Less

Submitted 21 August, 2008; originally announced August 2008.

Journal ref: IEEE Transactions on Information Theory, Vol. 54, No. 8. (2008), pp. 3804-3807

A simple randomized algorithm for sequential prediction of ergodic time series

Authors: L. Györfi, G. Lugosi, G. Morvai

Abstract: We present a simple randomized procedure for the prediction of a binary sequence. The algorithm uses ideas from recent developments of the theory of the prediction of individual sequences. We show that if the sequence is a realization of a stationary and ergodic random process then the average number of mistakes converges, almost surely, to that of the optimum, given by the Bayes predictor. The… ▽ More We present a simple randomized procedure for the prediction of a binary sequence. The algorithm uses ideas from recent developments of the theory of the prediction of individual sequences. We show that if the sequence is a realization of a stationary and ergodic random process then the average number of mistakes converges, almost surely, to that of the optimum, given by the Bayes predictor. The desirable finite-sample properties of the predictor are illustrated by its performance for Markov processes. In such cases the predictor exhibits near optimal behavior even without knowing the order of the Markov process. Prediction with side information is also considered. △ Less

Submitted 20 May, 2008; originally announced May 2008.

Journal ref: IEEE Trans. Inform. Theory 45 (1999), no. 7, 2642--2650

Weakly Convergent Nonparametric Forecasting of Stationary Time Series

Authors: G. Morvai, S. Yakowitz, P. Algoet

Abstract: The conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past. Several schemes are known for constructing pointwise consistent estimates, but they all demand prohibitive amounts of input data. In this paper we consider real-valued time series and construct conditional distribution estimates that make… ▽ More The conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past. Several schemes are known for constructing pointwise consistent estimates, but they all demand prohibitive amounts of input data. In this paper we consider real-valued time series and construct conditional distribution estimates that make much more efficient use of the input data. The estimates are consistent in a weak sense, and the question whether they are pointwise consistent is still open. For finite-alphabet processes one may rely on a universal data compression scheme like the Lempel-Ziv algorithm to construct conditional probability mass function estimates that are consistent in expected information divergence. Consistency in this strong sense cannot be attained in a universal sense for all stationary processes with values in an infinite alphabet, but weak consistency can. Some applications of the estimates to on-line forecasting, regression and classification are discussed. △ Less

Submitted 20 May, 2008; originally announced May 2008.

Journal ref: IEEE Transactions on Information Theory Vol. 43, pp. 483-498, 1997

On Sequential Estimation and Prediction for Discrete Time Series

Authors: G. Morvai, B. Weiss

Abstract: The problem of extracting as much information as possible from a sequence of observations of a stationary stochastic process $X_0,X_1,...X_n$ has been considered by many authors from different points of view. It has long been known through the work of D. Bailey that no universal estimator for $\textbf{P}(X_{n+1}|X_0,X_1,...X_n)$ can be found which converges to the true estimator almost surely. D… ▽ More The problem of extracting as much information as possible from a sequence of observations of a stationary stochastic process $X_0,X_1,...X_n$ has been considered by many authors from different points of view. It has long been known through the work of D. Bailey that no universal estimator for $\textbf{P}(X_{n+1}|X_0,X_1,...X_n)$ can be found which converges to the true estimator almost surely. Despite this result, for restricted classes of processes, or for sequences of estimators along stop** times, universal estimators can be found. We present here a survey of some of the recent work that has been done along these lines. △ Less

Submitted 30 March, 2008; originally announced March 2008.

Journal ref: Stochastics and Dynamics, Vol. 7, No. 4. pp. 417-437, 2007

Strongly consistent nonparametric forecasting and regression for stationary ergodic sequences

Authors: S. Yakowitz, L. Gyorfi, J. Kieffer, G. Morvai

Abstract: Let $\{(X_i,Y_i)\}$ be a stationary ergodic time series with $(X,Y)$ values in the product space $\R^d\bigotimes \R .$ This study offers what is believed to be the first strongly consistent (with respect to pointwise, least-squares, and uniform distance) algorithm for inferring $m(x)=E[Y_0|X_0=x]$ under the presumption that $m(x)$ is uniformly Lipschitz continuous. Auto-regression, or forecastin… ▽ More Let $\{(X_i,Y_i)\}$ be a stationary ergodic time series with $(X,Y)$ values in the product space $\R^d\bigotimes \R .$ This study offers what is believed to be the first strongly consistent (with respect to pointwise, least-squares, and uniform distance) algorithm for inferring $m(x)=E[Y_0|X_0=x]$ under the presumption that $m(x)$ is uniformly Lipschitz continuous. Auto-regression, or forecasting, is an important special case, and as such our work extends the literature of nonparametric, nonlinear forecasting by circumventing customary mixing assumptions. The work is motivated by a time series model in stochastic finance and by perspectives of its contribution to the issues of universal time series estimation. △ Less

Submitted 16 December, 2007; originally announced December 2007.

Journal ref: J. Multivariate Anal. 71 (1999), no. 1, 24--41

Limits to consistent on-line forecasting for ergodic time series

Authors: L. Gyorfi, G. Morvai, S. Yakowitz

Abstract: This study concerns problems of time-series forecasting under the weakest of assumptions. Related results are surveyed and are points of departure for the developments here, some of which are new and others are new derivations of previous findings. The contributions in this study are all negative, showing that various plausible prediction problems are unsolvable, or in other cases, are not solva… ▽ More This study concerns problems of time-series forecasting under the weakest of assumptions. Related results are surveyed and are points of departure for the developments here, some of which are new and others are new derivations of previous findings. The contributions in this study are all negative, showing that various plausible prediction problems are unsolvable, or in other cases, are not solvable by predictors which are known to be consistent when mixing conditions hold. △ Less

Submitted 14 December, 2007; originally announced December 2007.

Journal ref: IEEE Trans. Inform. Theory 44 (1998), no. 2, 886--892

On estimating the memory for finitarily Markovian processes

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: Finitarily Markovian processes are those processes $\{X_n\}_{n=-\infty}^{\infty}$ for which there is a finite $K$ ($K = K(\{X_n\}_{n=-\infty}^0$) such that the conditional distribution of $X_1$ given the entire past is equal to the conditional distribution of $X_1$ given only $\{X_n\}_{n=1-K}^0$. The least such value of $K$ is called the memory length. We give a rather complete analysis of the p… ▽ More Finitarily Markovian processes are those processes $\{X_n\}_{n=-\infty}^{\infty}$ for which there is a finite $K$ ($K = K(\{X_n\}_{n=-\infty}^0$) such that the conditional distribution of $X_1$ given the entire past is equal to the conditional distribution of $X_1$ given only $\{X_n\}_{n=1-K}^0$. The least such value of $K$ is called the memory length. We give a rather complete analysis of the problems of universally estimating the least such value of $K$, both in the backward sense that we have just described and in the forward sense, where one observes successive values of $\{X_n\}$ for $n \geq 0$ and asks for the least value $K$ such that the conditional distribution of $X_{n+1}$ given $\{X_i\}_{i=n-K+1}^n$ is the same as the conditional distribution of $X_{n+1}$ given $\{X_i\}_{i=-\infty}^n$. We allow for finite or countably infinite alphabet size. △ Less

Submitted 3 December, 2007; originally announced December 2007.

Journal ref: Ann. Inst. H. Poincare Probab. Statist. 43 (2007), no. 1, 15--30

Forward estimation for ergodic time series

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: The forward estimation problem for stationary and ergodic time series $\{X_n\}_{n=0}^{\infty}$ taking values from a finite alphabet ${\cal X}$ is to estimate the probability that $X_{n+1}=x$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process $\{X_n\}$. We present a simple procedure $g_n$ which is evaluated on the data segment… ▽ More The forward estimation problem for stationary and ergodic time series $\{X_n\}_{n=0}^{\infty}$ taking values from a finite alphabet ${\cal X}$ is to estimate the probability that $X_{n+1}=x$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process $\{X_n\}$. We present a simple procedure $g_n$ which is evaluated on the data segment $(X_0,...,X_n)$ and for which, ${\rm error}(n) = |g_{n}(x)-P(X_{n+1}=x |X_0,...,X_n)|\to 0$ almost surely for a subclass of all stationary and ergodic time series, while for the full class the Cesaro average of the error tends to zero almost surely and moreover, the error tends to zero in probability. △ Less

Submitted 24 November, 2007; originally announced November 2007.

Journal ref: Ann. Inst. H. Poincare Probab. Statist. 41 (2005), no. 5, 859--870

Order estimation of Markov chains

Authors: G. Morvai, B. Weiss

Abstract: We describe estimators $χ_n(X_0,X_1,...,X_n)$, which when applied to an unknown stationary process taking values from a countable alphabet ${\cal X}$, converge almost surely to $k$ in case the process is a $k$-th order Markov chain and to infinity otherwise. We describe estimators $χ_n(X_0,X_1,...,X_n)$, which when applied to an unknown stationary process taking values from a countable alphabet ${\cal X}$, converge almost surely to $k$ in case the process is a $k$-th order Markov chain and to infinity otherwise. △ Less

Submitted 3 November, 2007; originally announced November 2007.

Journal ref: IEEE Trans. Inform. Theory 51 (2005), no. 4, 1496--1497

Prediction for discrete time series

Authors: G. Morvai, B. Weiss

Abstract: Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stop** times $λ_n$ along which we will be able to estimate the conditional probability $P(X_{λ_n+1}=x|X_0,...,X_{λ_n})$ from data segment $(X_0,...,X_{λ_n})$ in a pointwise consistent w… ▽ More Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stop** times $λ_n$ along which we will be able to estimate the conditional probability $P(X_{λ_n+1}=x|X_0,...,X_{λ_n})$ from data segment $(X_0,...,X_{λ_n})$ in a pointwise consistent way for a restricted class of stationary and ergodic finite or countably infinite alphabet time series which includes among others all stationary and ergodic finitarily Markovian processes. If the stationary and ergodic process turns out to be finitarily Markovian (among others, all stationary and ergodic Markov chains are included in this class) then $ \lim_{n\to \infty} {n\over λ_n}>0$ almost surely. If the stationary and ergodic process turns out to possess finite entropy rate then $λ_n$ is upperbounded by a polynomial, eventually almost surely. △ Less

Submitted 3 November, 2007; originally announced November 2007.

Journal ref: Probab. Theory Related Fields 132 (2005), no. 1, 1--12

Nonparametric inference for ergodic, stationary time series

Authors: G. Morvai, S. Yakowitz, L. Gyorfi

Abstract: The setting is a stationary, ergodic time series. The challenge is to construct a sequence of functions, each based on only finite segments of the past, which together provide a strongly consistent estimator for the conditional probability of the next observation, given the infinite past. Ornstein gave such a construction for the case that the values are from a finite set, and recently Algoet ex… ▽ More The setting is a stationary, ergodic time series. The challenge is to construct a sequence of functions, each based on only finite segments of the past, which together provide a strongly consistent estimator for the conditional probability of the next observation, given the infinite past. Ornstein gave such a construction for the case that the values are from a finite set, and recently Algoet extended the scheme to time series with coordinates in a Polish space. The present study relates a different solution to the challenge. The algorithm is simple and its verification is fairly transparent. Some extensions to regression, pattern recognition, and on-line forecasting are mentioned. △ Less

Submitted 2 November, 2007; originally announced November 2007.

Journal ref: Ann. Statist. 24 (1996), no. 1, 370--379

Intermittent estimation of stationary time series

Authors: G. Morvai, B. Weiss

Abstract: Let $\{X_n\}_{n=0}^{\infty}$ be a stationary real-valued time series with unknown distribution. Our goal is to estimate the conditional expectation of $X_{n+1}$ based on the observations $X_i$, $0\le i\le n$ in a strongly consistent way. Bailey and Ryabko proved that this is not possible even for ergodic binary time series if one estimates at all values of $n$. We propose a very simple algorithm… ▽ More Let $\{X_n\}_{n=0}^{\infty}$ be a stationary real-valued time series with unknown distribution. Our goal is to estimate the conditional expectation of $X_{n+1}$ based on the observations $X_i$, $0\le i\le n$ in a strongly consistent way. Bailey and Ryabko proved that this is not possible even for ergodic binary time series if one estimates at all values of $n$. We propose a very simple algorithm which will make prediction infinitely often at carefully selected stop** times chosen by our rule. We show that under certain conditions our procedure is strongly (pointwise) consistent, and $L_2$ consistent without any condition. An upper bound on the growth of the stop** times is also presented in this paper. △ Less

Submitted 2 November, 2007; originally announced November 2007.

Journal ref: Test 13 (2004), no. 2, 525--542

Forecasting for stationary binary time series

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: The forecasting problem for a stationary and ergodic binary time series $\{X_n\}_{n=0}^{\infty}$ is to estimate the probability that $X_{n+1}=1$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process $\{X_n\}$. It is known that this is not possible if one estimates at all values of $n$. We present a simple procedure which will attempt to make su… ▽ More The forecasting problem for a stationary and ergodic binary time series $\{X_n\}_{n=0}^{\infty}$ is to estimate the probability that $X_{n+1}=1$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process $\{X_n\}$. It is known that this is not possible if one estimates at all values of $n$. We present a simple procedure which will attempt to make such a prediction infinitely often at carefully selected stop** times chosen by the algorithm. We show that the proposed procedure is consistent under certain conditions, and we estimate the growth rate of the stop** times. △ Less

Submitted 26 October, 2007; originally announced October 2007.

Journal ref: Acta Appl. Math. 79 (2003), no. 1-2, 25--34

On classifying processes

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: We prove several results concerning classifications, based on successive observations $(X_1,..., X_n)$ of an unknown stationary and ergodic process, for membership in a given class of processes, such as the class of all finite order Markov chains. We prove several results concerning classifications, based on successive observations $(X_1,..., X_n)$ of an unknown stationary and ergodic process, for membership in a given class of processes, such as the class of all finite order Markov chains. △ Less

Submitted 19 October, 2007; originally announced October 2007.

Journal ref: Bernoulli 11 (2005), no. 3, pp. 523--532

Limitations on intermittent forecasting

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: Bailey showed that the general pointwise forecasting for stationary and ergodic time series has a negative solution. However, it is known that for Markov chains the problem can be solved. Morvai showed that there is a stop** time sequence $\{λ_n\}$ such that $P(X_{λ_n+1}=1|X_0,...,X_{λ_n}) $ can be estimated from samples $(X_0,...,X_{λ_n})$ such that the difference between the conditional prob… ▽ More Bailey showed that the general pointwise forecasting for stationary and ergodic time series has a negative solution. However, it is known that for Markov chains the problem can be solved. Morvai showed that there is a stop** time sequence $\{λ_n\}$ such that $P(X_{λ_n+1}=1|X_0,...,X_{λ_n}) $ can be estimated from samples $(X_0,...,X_{λ_n})$ such that the difference between the conditional probability and the estimate vanishes along these stopp** times for all stationary and ergodic binary time series. We will show it is not possible to estimate the above conditional probability along a stop** time sequence for all stationary and ergodic binary time series in a pointwise sense such that if the time series turns out to be a Markov chain, the predictor will predict eventually for all $n$. △ Less

Submitted 19 October, 2007; originally announced October 2007.

Journal ref: Statist. Probab. Lett. 72 (2005), no. 4, 285--290

Guessing the output of a stationary binary time series

Authors: Gusztav Morvai

Abstract: The forward prediction problem for a binary time series $\{X_n\}_{n=0}^{\infty}$ is to estimate the probability that $X_{n+1}=1$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process $\{X_n\}$. It is known that this is not possible if one estimates at all values of $n$. We present a simple procedure which will attempt to make such a prediction… ▽ More The forward prediction problem for a binary time series $\{X_n\}_{n=0}^{\infty}$ is to estimate the probability that $X_{n+1}=1$ based on the observations $X_i$, $0\le i\le n$ without prior knowledge of the distribution of the process $\{X_n\}$. It is known that this is not possible if one estimates at all values of $n$. We present a simple procedure which will attempt to make such a prediction infinitely often at carefully selected stop** times chosen by the algorithm. The growth rate of the stop** times is also exhibited. △ Less

Submitted 19 October, 2007; originally announced October 2007.

Journal ref: Foundations of statistical inference (Shoresh, 2000), 207--215, Contrib. Statist., Physica, Heidelberg, 2003

Inferring the conditional mean

Authors: Gusztav Morvai, Benjamin Weiss

Abstract: Consider a stationary real-valued time series $\{X_n\}_{n=0}^{\infty}$ with a priori unknown distribution. The goal is to estimate the conditional expectation $E(X_{n+1}|X_0,..., X_n)$ based on the observations $(X_0,..., X_n)$ in a pointwise consistent way. It is well known that this is not possible at all values of $n$. We will estimate it along stop** times. Consider a stationary real-valued time series $\{X_n\}_{n=0}^{\infty}$ with a priori unknown distribution. The goal is to estimate the conditional expectation $E(X_{n+1}|X_0,..., X_n)$ based on the observations $(X_0,..., X_n)$ in a pointwise consistent way. It is well known that this is not possible at all values of $n$. We will estimate it along stop** times. △ Less

Submitted 19 October, 2007; originally announced October 2007.

Journal ref: Theory Stoch. Process. 11 (2005), no. 1-2, pp. 112--120

Density estimation from an individual numerical sequence

Authors: Andrew B. Nobel, Gusztav Morvai, Sanjeev R. Kulkarni

Abstract: This paper considers estimation of a univariate density from an individual numerical sequence. It is assumed that (i) the limiting relative frequencies of the numerical sequence are governed by an unknown density, and (ii) there is a known upper bound for the variation of the density on an increasing sequence of intervals. A simple estimation scheme is proposed, and is shown to be $L_1$ consiste… ▽ More This paper considers estimation of a univariate density from an individual numerical sequence. It is assumed that (i) the limiting relative frequencies of the numerical sequence are governed by an unknown density, and (ii) there is a known upper bound for the variation of the density on an increasing sequence of intervals. A simple estimation scheme is proposed, and is shown to be $L_1$ consistent when (i) and (ii) apply. In addition it is shown that there is no consistent estimation scheme for the set of individual sequences satisfying only condition (i). △ Less

Submitted 12 October, 2007; originally announced October 2007.

Journal ref: IEEE Trans. Inform. Theory 44 (1998), no. 2, 537--541

Regression estimation from an individual stable sequence

Authors: Gusztav Morvai, Sanjeev R. Kulkarni, Andrew B. Nobel

Abstract: We consider univariate regression estimation from an individual (non-random) sequence $(x_1,y_1),(x_2,y_2), ... \in \real \times \real$, which is stable in the sense that for each interval $A \subseteq \real$, (i) the limiting relative frequency of $A$ under $x_1, x_2, ...$ is governed by an unknown probability distribution $μ$, and (ii) the limiting average of those $y_i$ with $x_i \in A$ is go… ▽ More We consider univariate regression estimation from an individual (non-random) sequence $(x_1,y_1),(x_2,y_2), ... \in \real \times \real$, which is stable in the sense that for each interval $A \subseteq \real$, (i) the limiting relative frequency of $A$ under $x_1, x_2, ...$ is governed by an unknown probability distribution $μ$, and (ii) the limiting average of those $y_i$ with $x_i \in A$ is governed by an unknown regression function $m(\cdot)$. A computationally simple scheme for estimating $m(\cdot)$ is exhibited, and is shown to be $L_2$ consistent for stable sequences $\{(x_i,y_i)\}$ such that $\{y_i\}$ is bounded and there is a known upper bound for the variation of $m(\cdot)$ on intervals of the form $(-i,i]$, $i \geq 1$. Complementing this positive result, it is shown that there is no consistent estimation scheme for the family of stable sequences whose regression functions have finite variation, even under the restriction that $x_i \in [0,1]$ and $y_i$ is binary-valued. △ Less

Submitted 12 October, 2007; originally announced October 2007.

Journal ref: Statistics 33 (1999), no. 2, 99--118

Queueing for ergodic arrivals and services

Authors: L. Gyorfi, G. Morvai

Abstract: In this paper we revisit the results of Loynes (1962) on stability of queues for ergodic arrivals and services, and show examples when the arrivals are bounded and ergodic, the service rate is constant, and under stability the limit distribution has larger than exponential tail. In this paper we revisit the results of Loynes (1962) on stability of queues for ergodic arrivals and services, and show examples when the arrivals are bounded and ergodic, the service rate is constant, and under stability the limit distribution has larger than exponential tail. △ Less

Submitted 14 September, 2007; originally announced September 2007.

Comments: Queueing for ergodic arrivals and services. In Limit Theorems in Probability and Statistics, I. Berkes E. Csaki, M. Csorgo (Eds.), pp. 127-141, J. Bolyai Mathematical Society, 2002

Journal ref: In Limit Theorems in Probability and Statistics, I. Berkes E. Csaki, M. Csorgo (Eds.), pp. 127-141, J. Bolyai Mathematical Society, 2002