From Motion to Actor and Action Inference in Unconstrained Video

Can a human fly? Can a baby run? Can a car walk? Emphatically the answer is no to all of these questions. Yet, the primary agenda in the action recognition literature has focused strictly on the action ignoring who or what is doing the acting. Concurrently, the object recognition community has focused strictly on identifying the humans, the babies, the cars, etc. without considering what these "actors" are doing in the video. Yet, the articulated motion induced from, say, a bird eating versus an adult human eating is significantly different. In this talk, I will describe our recent work to unify these two problems into joint actor-action recognition with structured inference in unconstrained video. I will present a sequence of increasingly more sophisticated models, from an independent naive Bayes model through a hierarchical graphical model for jointly capturing these two inferential goals. We have developed a new dataset with seven actor classes and nine action classes (including the null action), and will present and discuss results of all models on this challenging dataset.

Jason Corso is an associate professor of Electrical Engineering and Computer Science at the University of Michigan. He received his PhD and MSE degrees at The Johns Hopkins University in 2005 and 2002, respectively, and the BS Degree with honors from Loyola College In Maryland in 2000, all in Computer Science. He spent two years as a post-doctoral fellow at the University of California, Los Angeles. From 2007-14 he was a member of the Computer Science and Engineering faculty at SUNY Buffalo. He is the recipient of the Army Research Office Young Investigator Award 2010, NSF CAREER award 2009, SUNY Buffalo Young Investigator Award 2011, a member of the 2009 DARPA Computer Science Study Group, and a recipient of the Link Foundation Fellowship in Advanced Simulation and Training 2003. Corso has authored more than ninety peer-reviewed papers on topics of his research interest including computer vision, robot perception, data science, and medical imaging. He is a member of the AAAI, IEEE and the ACM.