Communications and Signal Processing Seminar

Deep Learning Meets Sparse Regularization: A Signal Processing Perspective

Rob NowakProfessor, Electrical and Computer EngineeringUniversity of Wisconsin-MadisonKeith and Jane Nosbusch Professorship in Electrical and Computer Engineering University of Wisconsin-Madison
1690 Beyster BuildingMap

Abstract: Deep learning has been wildly successful in practice and most state-of-the-art artificial intelligence systems are based on neural networks. Lacking, however, is a rigorous mathematical theory that adequately explains the amazing performance of deep neural networks. In this talk, I present a relatively new mathematical framework that provides the beginning of a deeper understanding of deep learning. This framework precisely characterizes the functional properties of trained neural networks. The key mathematical tools which support this framework include transform-domain sparse regularization, the Radon transform of computed tomography, and approximation theory. This framework explains the effect of weight decay regularization in neural network training, the importance of skip connections and low-rank weight matrices in network architectures, the role of sparsity in neural networks, and explains why neural networks can perform well in high-dimensional problems.

This talk is based on joint work with Rahul Parhi, Joe Shenouda, and Liu Yang.

Bio: Robert Nowak holds the Keith and Jane Nosbusch Professorship in Electrical and Computer Engineering at the University of Wisconsin-Madison, where he directs the AFOSR/AFRL University Center of Excellence on Data Efficient Machine Learning. His research focuses on machine learning, optimization, and signal processing. He serves on the editorial boards of the SIAM Journal on the Mathematics of Data Science and the IEEE Journal on Selected Areas in Information Theory.

***Event will take place in a hybrid format. The location for in-person attendance will be room 1690 Beyster Building.   Attendance will also be available via Zoom.

Join Zoom Meeting https:

Meeting ID: 914 1429 7851

Passcode: XXXXXX (Will be sent via e-mail to attendees)

Zoom Passcode information is also available upon request to Sher Nickrand ([email protected])

See full seminar by Professor Nowak

Faculty Host

Laura BalzanoAssociate Professor, Electrical Engineering and Computer Science; Associate Professor of Statistics, College of Literature, Science, and the ArtsUniversity of Michigan