Functional Analytic Perspectives on Nonparametric Density Estimation

Nonparametric density estimation is a classic problem in statistics. In the standard estimation setting, when one has access to iid samples from an unknown distribution, there exist several established and well-studied nonparametric density estimators. Yet there remains interesting alternative settings which are less well-studied. This work considers two such settings. First we consider the case where the data contains some contamination, i.e. a portion of the data is not distributed according to the density we would like to estimate. In this setting one would like an estimator which is robust to the contaminating data. An approach to this was suggested by Kim and Scott in 2012. The estimator in that paper was analytically and experimentally shown to be robust, but no consistency result was presented. We demonstrate that this estimator is indeed consistent for a class of convex losses. Next we introduce a new robust kernel density estimator based on scaling and projection in Hilbert space. This estimator is proven to be consistent and will converge to the true density provided certain assumptions on the contaminating distribution. Its efficacy is demonstrated experimentally by applying it to several datasets. Finally we consider a different sort of setting which can be thought of as nonparametric mixture modelling. Here one would like to estimate multiple densities with access to groups of samples where each sample in a group is known to be distributed according the same unknown density. Tight identifiability bounds and a highly general algorithm for recovery of the densities are presented for this setting.