Linear Regression with Gaussian Model Uncertainty: Algorithms and Bounds

Ami Wiesel is from the Israel Institute of Technology (Technion), Haifa, Israel
Abstract

We consider the problem of estimating an unknown deterministic parameter vector in a linear regression model with a Gaussian model matrix. We prove that the maximum likelihood (ML) estimator is a regularized least squares estimator and develop three alternative approaches for finding the optimal regularization parameter. We analyze the performance using the Cram'er Rao bound (CRB) on the mean squared error, and show that the degradation in performance due the uncertainty is not as severe as may be expected. Next, we address the problem again assuming that the variances of the noise and the elements in the model matrix are unknown and derive the associated CRB and ML estimator.

We compare our methods to known results on linear regression in the error in variables (EIV) model. We discuss the similarity between these two competing approaches, and provide a thorough comparison which sheds light on their theoretical and practical differences.