Three teams of graduate students awarded prizes for their final projects in Image Processing (EECS 556)

Three student teams in the graduate level course Image Processing (EECS 556) earned prizes for their final projects, thanks to the sponsorship of KLA. Their projects focused on improving image processing for neurosurgery and satellite applications and MRI reconstruction techniques.

In EECS 556, students explore methods to improve image processing in applications such as biomedical imaging and video and image compression. The course is taught by Jeff Fessler, William L. Root Collegiate Professor of Electrical Engineering and Computer Science, a leading expert in medical imaging.

KLA has sponsored prizes for students in the class during the past decade, and sends their own engineers to judge the winners. KLA is a global capital equipment company led by President and CEO Rick Wallace, who is also an alumnus of the department.

The winning teams are listed below:

First Place Award (tie)

Multimodal Image Registration: Comparison of Methods for 3D MRI to 3D Ultrasound Image Registration with Classical and Deep-Learning Accelerated Approaches
Dinank Gupta, David Kucher, Daniel Manwiller, Ellen Yeats

Multimodal image registration is the task of mapping images from different coordinate systems and different imaging modalities into a common coordinate system. This team investigated Magnetic Resonance Imaging (MRI) to 3D Ultrasound (US) image registration, typically desired in neurosurgery applications, where intra-operative US provides real-time feedback to the surgeon but pre-operative MRI provides much better soft tissue contrast. They compared two state-of-the-art image processing-based methods: Linear Correlation of Linear Combinations (LC2), Self-Similarity Context (SSC), and a U-Net based deep-learning approach to compare their registration accuracy. LC2 assumes that US intensities are linearly related to the MRI intensity and gradient and tries to learn a transformation to minimize error in this relation. SSC, on the other hand, assumes similarities in relationships of image neighborhoods in both images and tries to learn a transformation to make the neighborhood relations similar. Deep learning tries to minimize a loss function based on hand-matched landmarks in both images. They also tested whether the more roughly aligned output from the U-Net model could then be fed to a classical method for a more fine-tuned registration to potentially provide an overall improved accuracy and an accelerated convergence while maintaining its model robustness.

Sentinel-2 Sharpening Using a Reduced-Rank Method With Modified Roughness Regularization
Hyeonsu Do, John Gearig, Anusha Kikkeri, Konrad Rauscher

Pansharpening, the process of improving low-resolution bands of an image through information from high-resolution bands, is important to construct high-fidelity satellite images. This team proposed improvements to S2Sharp, an optimization-based sharpening method, which sharpens 60-meter bands of images collected from the Sentinel-2 Satellite. They observed modest improvements in error metrics, convert S2Sharp from Matlab to Julia, and mathematically describe possible changes to the regularization used for the cost function.

Runner-Up Prize

Learning-Based Optimization for Under-Sampling MRI
Anyatama Makur, Jiaren Zou, Ning Lu, Yuhang Zhang

Compressed sensing MRI (CS-MRI) recovers the images with under-sampled k-space measurements to accelerate scan times. The two fundamental components in CS-MRI are the under-sampling pattern and the reconstruction model. This team acquired both components simultaneously using an end-to-end learning framework named LOUPE (Learning-based Optimization of the Under-sampling PattErn). For a given sparsity constraint, this method trains a neural network model on full-resolution data that are under-sampled retrospectively, yielding a data-driven optimized sub-sampling pattern and a reconstruction model that is customized to the type of images represented in the training data. They adapted the original LOUPE algorithm in Pytorch and implement three different neural networks using both magnitude and complex MRI data. They also extended LOUPE from the specific case of 2D Cartesian sampling to the non-Cartesian scheme. Their experiments with single-coil knee MRI data show that the optimized sub-sampling pattern can offer significantly more accurate reconstructions compared to standard random uniform under-sampling schemes.