Convolutional Dictionary Learning Using a Fast Block Proximal Gradient Method

Multi-convex optimization problems play a significant role in modern signal/image
processing, computer vision, and related disciplines. The Block Proximal Gradient (BPG) method is a state- of-the-art technique for solving multi-convex problems that avoids small regions around certain local minima and provides lower objective values. However, the BPG technique can be impractical for solving complicated and/or large dimensional problems, due to its dependence on Lipschitz continuity of the objective function. The first half of this talk introduces our recent optimization method, BPG method using a Majorizer (BPG-M), that has the benefits of resolving drawbacks of BPG, and its accelerated versions. Convolutional Dictionary Learning (CDL) is a fundamental component in understanding (deep) convolutional neural networks. In addition, CDL has received considerable attention by resolving the fundamental problems of patch-based dictionary learning, e.g., translation-variant dictionaries and limitations using "big data". The popular optimization techniques for solving bi-convex prob- lems of CDL are the augmented Lagrangian method and its variants. However, they require tricky parameter tuning processes due to their dependence on training data and lack of convergence guar- antee. The second half of this talk introduces a CDL formulation and demonstrates the usefulness of applying fast BPG-M for convergent CDL.

This is joint work work with Xuehang Zheng and Jeffrey A. Fessler

Il Yong Chun received the Ph.D. degree in electrical and computer engineering from Purdue Uni- versity, West Lafayette, IN, USA, in 2015. From 2015 to 2016, he was a Postdoctoral Research Associate in Mathematics, Purdue University, West Lafayette, IN, USA. He is currently a Post- doctoral Research Fellow in Engineering and Computer Science, the University of Michigan, Ann Arbor, MI, USA. His research interests include compressed sensing, non-convex optimization, convolutional kernel learning, and adaptive signal processing, applied to computational imaging in medicine, photography, and neuroscience.