Bilinear generalized approximate message passing (BIG-AMP) for high-dimensional inference problems

We describe ongoing work on BiG-AMP, a novel approximate-message-passing based solution to bilinear matrix recovery, with applications to dictionary learning, matrix completion, robust PCA, and non-negative matrix factorization. This work applies to statistical inference problems with matrix-valued observations of the form Y = f(A*X + W), where A and X (or sometimes their product Z=A*X) are the unknown matrices of interest, W represents noise or uncertainty, and f(.) is a known, possibly nonlinear, component-wise map. For example, in dictionary learning, A represents the dictionary, X represents the sparse data coding, and f(.) is usually trivial. Meanwhile, in matrix completion, Z represents the low-rank matrix to complete, and f(.) is an element-wise mask. BiG-AMP can be viewed as a Bayesian inference algorithm that returns an approximation to either the joint MAP or MMSE estimate of Z, depending on the how the algorithm is run, under statistical priors for A, X, and W that are independent across matrix elements but otherwise generic. To circumvent the restriction to known priors, we assume that the true prior lives within a generic family of priors (e.g., Gaussian mixture) and learn the underlying parameters using an expectation-maximization approach. Moreover, to circumvent the restriction to independent priors, we augment the factor graph on which the message-passing is performed with hidden coupling variables and perform joint inference on the larger factor graph. We demonstrate that the proposed approach performs favorable with respect to state-of-the-art algorithms for dictionary learning, matrix completion, robust PCA, and non-negative matrix factorization on both synthetic and real-world datasets for applications including image denoising, movie-rating prediction, video surveillance, and hyperspectral unmixing.
Philip Schniter received the B.S. and M.S. degrees in Electrical Engineering from the University of Illinois at Urbana-Champaign in 1992 and 1993, and the Ph.D. degree in Electrical Engineering from Cornell University in Ithaca NY in 2000. Subsequently, he joined the Department of Electrical and Computer Engineering at The Ohio State University in Columbus, OH, where he is now a Professor. He currently serves on the IEEE Sensor Array and Multichannel (SAM) Technical Committee, he is a guest editor for the IEEE JSAC Special Issue on Full-duplex Wireless Communications and Networks, and he is the Technical Chairman for the 2013 Asilomar Conference on Signals, Systems, and Computers. Dr. Schniter's areas of research include signal processing, wireless communications, and machine learning.