THIS PAGE INCORPORATES CASCADING STYLE SHEET! TO VIEW PAGE CORRECTLY PLEASE UPDATE TO A CSS COMPLIANT BROWSER
Semantic Scene Classification
UR > CS > Research > Vision > Projects > Semantic Scene Classification
Semantic Scene Classification icon
We improve semantic scene classification by using semantic object/ material detectors and spatial models within a probabilistic framework to infer the scene type.

Matthew Boutell (Ph.D. Thesis Work) Chris Brown (Thesis Advisor) Randal Nelson (Thesis Committee) Ted Pawlicki (Thesis Committee) Robert Jacobs (Thesis Committee) Jiebo Luo (Thesis Committee) Amit Singhal (completed Ph.D. Thesis)
Semantic Scene Classification figure
Matthew Boutell
Homogeneous material detection produces a labeled, segmented image. We extract the list of materials present and compute the spatial relationships between regions, using the statistics from many such images to train the MASSES simulator and the inference engine. Finally, we pass a novel image's materials and spatial relations to the inference engine to classify the image. The classifier needs spatial information to overcome the negative effects of faulty material detectors. We choose to learn the spatial relations and use a probabilistic engine, because natural scenes' spatial relationships are loosely constrained.

Semantic scene classification is part of a larger effort to understand images, particularly the unconstrained domain of consumer photographs. Scene classification, automatically categorizing images into semantic categories such as "beach", "field", or "party", is a useful skill, finding application in content-based image organization and in digital photofinishing. Incorporating semantic material and object detectors has been recently demonstrated to increase the performace of such systems, because the gap between the features and the image semantics has been narrowed. However, there is a need to model the spatial relationships between the objects to distinguish certain scenes and to mitigate the effects of faulty detectors. In this project, we investigate probabilistic frameworks, such as Bayesian networks, Hidden Markov Models, and Markov Random Fields, appropriate for handling the loose spatial relationships existing in real-world scenes. We have developed MASSES (Material And Spatial Simulated Experimental Scenes), an experimental testbed that can be used to evaluate the effects of spatial models, the probabilistic frameworks, and detector characteristics.
http://www.cs.rochester.edu/~boutell/
scene classification, semantic scene classification, image classification, ISC, object detection, material detection, image understanding, semantic features, low-level features, probabilistic inference, spatial modeling, pattern recognition, MASSES, semantic categories, content-sensitive image enhancement, Markov Random Field, Bayesian network, Bayes net, belief network, Hidden Markov Models, HMM, MRF, Matthew Boutell, Christopher Brown, home, homepage, home page, URCS, University of Rochester Computer Science Department, Computer Vision



Active (February 14, 2003)