Learning the Appearance and Motion of People in Video
A probabilistic method for tracking 3D articulated human figures in monocular image sequences is presented. Within a Bayesian probabilistic framework we learn statistical models of objects and scenes and exploit these models for tracking complex, deformable, or articulated objects in image sequences. In particular, we learn the likelihood of observing various spatial and temporal filter responses corresponding to edges, ridges, and motion differences given a model of a person. Similarly, we learn probability distributions over filter responses for general scenes that define a likelihood of observing the filter responses for arbitrary backgrounds. We then derive a probabilistic model for tracking that exploits the ratio between the likelihood that image pixels corresponding to the foreground (person) were generated by an actual person or by some unknown background. A prior probability distribution over possible human motions is learned from 3D motion-capture data and is combined with the likelihood for Bayesian tracking using particle filtering. In this approach, a posterior probability distribution over model parameters is represented using a discrete set of samples that is propagated over time. This explicit posterior probability distribution represents ambiguities due to image matching, model singularities, and perspective projection. By combining multiple image cues, and by using learned likelihood models, we demonstrate improved robustness and accuracy when tracking complex objects such as people in monocular image sequences with cluttered scenes and a moving camera.
(Joint work with: Hedvig Sidenbladh (Royal Institute of Technology, Sweden) David Fleet (Xerox PARC) Dirk Ormoneit (Stanford, Dept of Statistics))