The first problem we examine is inferring 3D shape from a set of noisy images. We derive a general probabilistic theory of occupancy and emptiness to specify what one can infer about 3D shape for arbitrarily-shaped, Lambertian scenes and arbitrary viewpoint configurations. By modeling the problem in scene space, we formalize the notions of visibility, occupancy, emptiness, and photo-consistency, leading to the Photo Hull Distribution, the tightest probabilistic bound on the scene's true shape that is theoretically computable from the input images. We show how to express this distribution directly in terms of image measurements and represent it compactly by assigning an occupancy probability at every 3D point. We provide a stochastic algorithm that draws fair samples from the Photo Hull Distribution and converges to an optimal conservative estimate of the occupancy probability and present experimental results for real, complex scenes.
The second problem we study is recovering nonrigid motion of deformable surfaces from noisy video. We develop linear methods for model-based tracking of nonrigid 3D objects in video. Uncertainty in image measurements is quantified and propagated through the inverse model to yield optimal 3D pose and deformation estimates directly from 2D image intensities. We obtain accurate and robust closed-form nonrigid motion estimators by minimizing information loss from non-reversible operations. We demonstrate results with 3D nonrigid tracking, model refinement, and super-resolution texture lifting from low-quality, low-resolution video.