Event: 3D Reconstruction, Perception, and Recognition of Reflective Objects

The ability to perceive and interpret the geometric shape and semantic meaning of materials and objects is essential for an intelligent visual system. A number of extensively studied cues, notably stereoscopic disparity, texture gradient, motion parallax, contours and shading, have been shown to carry valuable information on object surface shape. Unfortunately, many objects of interest and most man-made surfaces, such as a silver plate, an industrial structure or a clean automobile, are smooth and shiny, violating the hypotheses that underlie the analysis of those cues. For specular objects, however, one important but traditionally overlooked cue is the reflection of the environment: a deformed picture of the surrounding scene can be seen on the surface of the specular object— the degree and type of deformation depend upon its shape.
In this talk I introduce a geometrical and algebraic characterization of how a patch of the scene is mapped into an image by a mirror surface of given shape. I will then develop solutions to the inverse problem of deriving surface shape from mirror reflections in a single image and demonstrate that local information about the geometry of the surface can be fully estimated up to third order. Our theoretical results are validated by both numerical simulations and experiments with real surfaces. I will also give some insights into my research on human perception of shape from reflections through psychophysics experiments. Our goal is to provide a benchmark, as well as inspire possible technical approaches, for computational work. We find that, surprisingly, humans are very poor at judging the shape of mirror surfaces when additional visual cues (i.e., contour, shading, stereo, texture) are not visible. Finally, I will briefly describe my recent work on recognizing specular materials by using the information from distorted scenes.