-
On Regret of Parametric Mismatch in Minimum Mean Square Error Estimation
Authors:
Majid Fozunbal
Abstract:
This paper studies the effect of parametric mismatch in minimum mean square error (MMSE) estimation. In particular, we consider the problem of estimating the input signal from the output of an additive white Gaussian channel whose gain is fixed, but unknown. The input distribution is known, and the estimation process consists of two algorithms. First, a channel estimator blindly estimates the chan…
▽ More
This paper studies the effect of parametric mismatch in minimum mean square error (MMSE) estimation. In particular, we consider the problem of estimating the input signal from the output of an additive white Gaussian channel whose gain is fixed, but unknown. The input distribution is known, and the estimation process consists of two algorithms. First, a channel estimator blindly estimates the channel gain using past observations. Second, a mismatched MMSE estimator, optimized for the estimated channel gain, estimates the input signal. We analyze the regret, i.e., the additional mean square error, that is raised in this process. We derive upper-bounds on both absolute and relative regrets. Bounds are expressed in terms of the Fisher information. We also study regret for unbiased, efficient channel estimators, and derive a simple trade-off between Fisher information and relative regret. This trade-off shows that the product of a certain function of relative regret and Fisher information equals the signal-to-noise ratio, independent of the input distribution. The trade-off relation implies that higher Fisher information results to smaller expected relative regret.
△ Less
Submitted 7 June, 2010;
originally announced June 2010.
-
A Subsequence-Histogram Method for Generic Vocabulary Recognition over Deletion Channels
Authors:
Majid Fozunbal
Abstract:
We consider the problem of recognizing a vocabulary--a collection of words (sequences) over a finite alphabet--from a potential subsequence of one of its words. We assume the given subsequence is received through a deletion channel as a result of transmission of a random word from one of the two generic underlying vocabularies. An exact maximum a posterior (MAP) solution for this problem counts…
▽ More
We consider the problem of recognizing a vocabulary--a collection of words (sequences) over a finite alphabet--from a potential subsequence of one of its words. We assume the given subsequence is received through a deletion channel as a result of transmission of a random word from one of the two generic underlying vocabularies. An exact maximum a posterior (MAP) solution for this problem counts the number of ways a given subsequence can be derived from particular subsets of candidate vocabularies, requiring exponential time or space.
We present a polynomial approximation algorithm for this problem. The algorithm makes no prior assumption about the rules and patterns governing the structure of vocabularies. Instead, through off-line processing of vocabularies, it extracts data regarding regularity patterns in the subsequences of each vocabulary. In the recognition phase, the algorithm just uses this data, called subsequence-histogram, to decide in favor of one of the vocabularies. We provide examples to demonstrate the performance of the algorithm and show that it can achieve the same performance as MAP in some situations.
Potential applications include bioinformatics, storage systems, and search engines.
△ Less
Submitted 21 April, 2009;
originally announced April 2009.
-
Decision Making with Side Information and Unbounded Loss Functions
Authors:
Majid Fozunbal,
Ton Kalker
Abstract:
We consider the problem of decision-making with side information and unbounded loss functions. Inspired by probably approximately correct learning model, we use a slightly different model that incorporates the notion of side information in a more generic form to make it applicable to a broader class of applications including parameter estimation and system identification. We address sufficient c…
▽ More
We consider the problem of decision-making with side information and unbounded loss functions. Inspired by probably approximately correct learning model, we use a slightly different model that incorporates the notion of side information in a more generic form to make it applicable to a broader class of applications including parameter estimation and system identification. We address sufficient conditions for consistent decision-making with exponential convergence behavior. In this regard, besides a certain condition on the growth function of the class of loss functions, it suffices that the class of loss functions be dominated by a measurable function whose exponential Orlicz expectation is uniformly bounded over the probabilistic model. Decay exponent, decay constant, and sample complexity are discussed. Example applications to method of moments, maximum likelihood estimation, and system identification are illustrated, as well.
△ Less
Submitted 24 February, 2006; v1 submitted 27 January, 2006;
originally announced January 2006.
-
Capacity Analysis for Continuous Alphabet Channels with Side Information, Part II: MIMO Channels
Authors:
Majid Fozunbal,
Steven W. McLaughlin,
Ronald W. Schafer
Abstract:
In this part, we consider the capacity analysis for wireless mobile systems with multiple antenna architectures. We apply the results of the first part to a commonly known baseband, discrete-time multiple antenna system where both the transmitter and receiver know the channel's statistical law. We analyze the capacity for additive white Gaussian noise (AWGN) channels, fading channels with full c…
▽ More
In this part, we consider the capacity analysis for wireless mobile systems with multiple antenna architectures. We apply the results of the first part to a commonly known baseband, discrete-time multiple antenna system where both the transmitter and receiver know the channel's statistical law. We analyze the capacity for additive white Gaussian noise (AWGN) channels, fading channels with full channel state information (CSI) at the receiver, fading channels with no CSI, and fading channels with partial CSI at the receiver. For each type of channels, we study the capacity value as well as issues such as the existence, uniqueness, and characterization of the capacity-achieving measures for different types of moment constraints. The results are applicable to both Rayleigh and Rician fading channels in the presence of arbitrary line-of-sight and correlation profiles.
△ Less
Submitted 6 November, 2004;
originally announced November 2004.
-
Capacity Analysis for Continuous Alphabet Channels with Side Information, Part I: A General Framework
Authors:
Majid Fozunbal,
Steven W. McLaughlin,
Ronald W. Schafer
Abstract:
Capacity analysis for channels with side information at the receiver has been an active area of interest. This problem is well investigated for the case of finite alphabet channels. However, the results are not easily generalizable to the case of continuous alphabet channels due to analytic difficulties inherent with continuous alphabets. In the first part of this two-part paper, we address an a…
▽ More
Capacity analysis for channels with side information at the receiver has been an active area of interest. This problem is well investigated for the case of finite alphabet channels. However, the results are not easily generalizable to the case of continuous alphabet channels due to analytic difficulties inherent with continuous alphabets. In the first part of this two-part paper, we address an analytical framework for capacity analysis of continuous alphabet channels with side information at the receiver. For this purpose, we establish novel necessary and sufficient conditions for weak* continuity and strict concavity of the mutual information. These conditions are used in investigating the existence and uniqueness of the capacity-achieving measures. Furthermore, we derive necessary and sufficient conditions that characterize the capacity value and the capacity-achieving measure for continuous alphabet channels with side information at the receiver.
△ Less
Submitted 6 November, 2004;
originally announced November 2004.
