Estimation Theory with Side Information for Periodic and Constrained Problems

In many practical parameter estimation problems some side information regarding the unknown parameters is available. Types of side information that are commonly encountered in signal processing applications include periodicity, parametric equality and inequality constraints, and sparsity. In this research, we address some fundamental topics in estimation theory in the presence of side information. We exploit the side information by choosing proper cost functions, deriving optimal estimation methods, and developing corresponding performance bounds. In the first stage of this work we have investigated the problem of Bayesian parameter estimation in the presence of periodic side information. In periodic parameter estimation, the commonly used mean-squared-error (MSE) risk is inappropriate, since it does not take into account the periodic nature of the problem. We proposed a new class of Bayesian lower bounds on the mean-cyclic-error. This class can be interpreted as the cyclic-equivalent of the Weiss-Weinstein class of MSE lower bounds. This class was extended to provide Bayesian lower bounds for stochastic filtering with periodic side information. In the second stage of this work, we considered non-Bayesian parameter estimation under differentiable equality constraints. For this type of estimation problem, we defined proper unbiasedness in the Lehmann sense and developed new performance bounds that were shown to be more appropriate than the well-known constrained Cramér-Rao bound. In the third stage of this work, we proposed a method for optimal biased estimation. This method is based on combining Lehmann-unbiasedness under a weighted MSE risk and a penalized likelihood approach. It was shown that this method can be useful for parameter estimation under inequality constraints and can lead to estimators that uniformly outperform the minimum variance unbiased estimator and the maximum likelihood estimator.

* Ph.D. research supervised by Prof. Joseph Tabrikian and Dr. Tirza Routtenberg