We improve semantic scene classification by using semantic object/
material detectors and spatial models within a probabilistic framework
to infer the scene type.
Matthew Boutell (Ph.D. Thesis Work)
Chris Brown (Thesis Advisor)
Randal Nelson (Thesis Committee)
Ted Pawlicki (Thesis Committee)
Robert Jacobs (Thesis Committee)
Jiebo Luo (Thesis Committee)
Amit Singhal (completed Ph.D. Thesis)
Matthew Boutell
Homogeneous material detection produces a labeled, segmented image.
We extract the list of materials present and compute the spatial
relationships between regions, using the statistics from many such
images to train the MASSES simulator and the inference
engine. Finally, we pass a novel image's materials and spatial
relations to the inference engine to classify the image. The
classifier needs spatial information to overcome the negative effects
of faulty material detectors. We choose to learn the spatial relations
and use a probabilistic engine, because natural scenes' spatial
relationships are loosely constrained.
Semantic scene classification is part of a larger effort to understand
images, particularly the unconstrained domain of consumer
photographs. Scene classification, automatically categorizing images
into semantic categories such as "beach", "field", or "party", is a
useful skill, finding application in content-based image organization
and in digital photofinishing.
Incorporating semantic material and object detectors has been recently
demonstrated to increase the performace of such systems, because the
gap between the features and the image semantics has been
narrowed. However, there is a need to model the spatial relationships
between the objects to distinguish certain scenes and to mitigate the
effects of faulty detectors.
In this project, we investigate probabilistic frameworks, such as
Bayesian networks, Hidden Markov Models, and Markov Random Fields,
appropriate for handling the loose spatial relationships existing in
real-world scenes. We have developed MASSES (Material And Spatial
Simulated Experimental Scenes), an experimental testbed that can be
used to evaluate the effects of spatial models, the probabilistic
frameworks, and detector characteristics.
http://www.cs.rochester.edu/~boutell/
scene classification, semantic scene classification, image
classification, ISC, object detection, material detection, image
understanding, semantic features, low-level features, probabilistic
inference, spatial modeling, pattern recognition, MASSES, semantic
categories, content-sensitive image enhancement, Markov Random Field,
Bayesian network, Bayes net, belief network, Hidden Markov Models,
HMM, MRF, Matthew Boutell, Christopher Brown, home, homepage, home
page, URCS, University of Rochester Computer Science Department,
Computer Vision