Smoothing-enabled Zeroth-order Schemes for Stochastic Optimization Problems: Addressing Stochasticity, Nonsmoothness, Nonconvexity, and Hierarchy.

Abstract: Zeroth-order methods in optimization rely on utilizing function-value information in developing convergent algorithms in optimization settings. Such schemes are motivated by the challenges in computing gradients or gradient estimators. We begin by examining unstructured nonsmooth and nonconvex stochastic optimization problems. By leveraging spherical smoothing, we derive complexity guarantees for computing approximate Clarke-stationary points for both stochastic (smoothed) gradient and quasi-Newton schemes. We then show that such avenues can be extended towards resolving a class of challenging hierarchical stochastic optimization problems (called stochastic Mathematical Programs with Equilibrium Constraints (MPEC)), a class of problems that subsumes subclasses of stochastic bilevel optimization problems and stochastic Stackelberg equilibrium problems. In the last part of the talk, we provide a foundation for an exponentially shifted Gaussian smoothing estimator (es-GS) that allows for significantly improved dimension-dependence. We observe that this estimator leads to improved dimension dependence in complexity guarantees for zeroth-order schemes and their accelerated counterparts. Time permitting, we also review some recent extensions of these avenues to federated learning (FL) in nonconvex and hierarchical settings.

Bio: Uday V. Shanbhag has been a Professor in the department of Industrial and Operations Engineering at the University of Michigan at Ann Arbor since Fall, 2024. From November 2016 to June 2024, he held the Gary and Sheila Chaired Professorship in the department of Industrial and Manufacturing Engineering (IME) at the Pennsylvania State University. Prior to being at Penn. State, from 2006–2012, he was first an assistant professor, and subsequently a tenured associate professor at the University of Illinois at Urbana-Champaign (UIUC). Uday V. Shanbhag has a Ph.D. from Stanford University’s department of Management Science and Engineering (2006). He currently serves as an Associate Editor (AE) for the SIAM Journal of Optimization and Computational Optimization and Applications, having served as a past AE for the IEEE Transactions on Automatic Control.

*** This Event will take place in a hybrid format. The location for in-person attendance will be room 1311 EECS. Attendance will also be available via Zoom.

Join Zoom Meeting: https://umich.zoom.us/j/96731875637

Meeting ID: 967 3187 5637

Passcode: XXXXXX (Will be sent via e-mail to attendees)

Zoom Passcode information is also available upon request to Kristi Rieger([email protected])