Guiding Diffusion and Flow Models for Constrained Sampling in Image, Video and 4D

Abstract: The recent emergence of diffusion models has driven substantial progress in image and video processing by establishing these models as powerful generative priors. However, challenges persist such as extension to 3D, video, and 4D problems. Meanwhile, flow models—though related—possess distinct characteristics from diffusion models, and their application in this domain remains relatively underexplored. In this talk, we present strategies to address these challenges, highlighting our recent work on solving video inverse problems using 2D diffusion models, video interpolation, 4D video generation, and FlowDPS—our state-of-the-art inverse solver based on flow models. Comprehensive experimental results demonstrate the effectiveness of diffusion and flow-based approaches.

Bio: Jong Chul Ye is a Professor at the Kim Jaechul Graduate School of Artificial Intelligence (AI) of Korea Advanced Institute of Science and Technology (KAIST), Korea. He received his B.Sc. and M.Sc. degrees from Seoul National University, Korea, and his PhD from Purdue University. Before joining KAIST, he worked at Philips Research and GE Global Research in New York. He has served as an associate editor of IEEE Trans. on Image Processing, IEEE Computational Imaging, IEEE Trans. on Medical Imaging and a Senior Editor of IEEE Signal Processing and an editorial board member for Magnetic Resonance in Medicine. He is an IEEE Fellow, was the Chair of IEEE SPS Computational Imaging TC, and IEEE EMBS Distinguished Lecturer. He is a Fellow of the Korean Academy of Science and Technology, and National Academy of Medicine in Korea, and was the President of the Korean Society for Artificial Intelligence in Medicine. He received various awards including Merck Fellow Award, and Choi Suk-Jung Award- one of the most prestigious awards for mathematicians in Korea. His research interest is machine learning and generative AI for biomedical imaging and computer vision.

*** This Event will take place in a hybrid format. The location for in-person attendance will be room 1200 EECS. Attendance will also be available via Zoom.

Join Zoom Meeting: https://umich.zoom.us/j/93679028340

Meeting ID: 936 7902 8340

Passcode: XXXXXX (Will be sent via e-mail to attendees)

Zoom Passcode information is also available upon request to Kristi Rieger([email protected])