Understanding Pictures of Rooms

Vision is special, because it can be used to predict the future. Good visual representations should be good at answering "what if" questions – will I trip if I walk here? if I put a sofa there, will it look good? I will show methods that can answer some "what if?" questions about rooms from pictures. Rooms form an interesting and important class of scene, because people live in rooms.

Approximate geometric representations are one part of answering a "what if" question. I will describe work that builds representations of free space in rooms from single images. This work models the room as a box, then infers the rotation, translation and aspect ratio of the box. Once the box of the room is known, we build an approximate model of the free space by representing large items of furniture as further boxes.

I will show methods that can recover approximate representations of surface properties and light in the space of the room. These methods yield sufficient information to render models that have been inserted into the room – we can tell what it would look like if a sofa were inserted here. The resulting composite images fool human observers fairly reliably. In recent work, we have been able to make these methods wholly automatic.

D.A. Forsyth is professor of Computer Science in the University of Illinois, Urbana-Champaign. He holds BS and MS degrees from the University of the Witwatersrand, and a DPhil from Balliol College, Oxford. He is currently editor in chief of IEEE TPAMI.