Login
Computer Science @ Rochester
Monday, April 18, 2005
2:00 PM
CSB 209
Matthew R. Boutell
University of Rochester
Exploiting Context for Semantic Scene Classification
Semantic scene classification, automatically categorizing images into a discrete set of classes such as beach, sunset, or field, is a difficult problem. Current classifiers rely on low-level image features, such as color, texture, or edges, and achieve limited success on constrained image sets. However, the domain of unconstrained consumer photographs requires the use of new features and techniques.

One source of information that can help classification is the context associated with the image. We have explored three types of context. First, spatial context enables the use of scene configurations (identities of regions and the spatial relationships between the regions) for classification purposes. Second, temporal context allows us to use information contained in neighboring images to classify an image. We exploit elapsed time between images to help determine which neighboring images are most closely related. Third, image capture condition context in the form of camera parameters (e.g., flash, exposure time, and subject distance) recorded at the time the photo was taken provides cues that are effective at distinguishing certain scene types.

We developed and used graphical models to incorporate image content with these three types of context. These systems are highly modular and allow for probabilistic input, output, and inference based on the statistics of large image collections. We demonstrate the effectiveness of each context model on several classification problems.

In this talk, I will focus on the temporal context model and elaborate on the others as time allows. Joint work with my co-advisors, Chris Brown and Jiebo Luo.