Variable Weight Kernel Density Estimation

Nonparametric density estimation is a common and important task in many problems in machine learning. It consists in estimating a density function from available observations without making parametric assumptions on the generating distribution. The kernel density estimator (KDE) is a nonparametric estimator composed of the average of simple functions, called kernels, centered at each data point. This work studies some relatives of the KDE with structural similarity but which assign different weights to each kernel unit in order to attain certain desired characteristics. In particular, we present a sparse estimator and a consistent estimator with fixed bandwidth parameter. The first estimator, being sparse, ameliorates computation time and is scalable for large amounts of data. Furthermore, our method to create sparsity expands to other kernel means, not necessarily density estimates. For the second estimator we show statistical consistency and convergence rates when the bandwidth parameter is held fixed. Experiments show it can well approximate some complex densities for inappropriate and rule of thumb bandwidths, a difficult task for the standard KDE. Addendum est, machine learning is not an unbiased endeavor, and I must responsibly address this issue. My research is intended to aid, among others, medical, environmental, and economic problems for the betterment of society, it is NOT intended to aid parasitic financial institutions, murderous military activity, discriminatory oppression by states, or similar malpractices.