Login
Computer Science @ Rochester
Wednesday, April 30, 2003
11:00 AM
CSB 209
Ph.D. Thesis Proposal
Andrew Learn
University of Rochester
Grounding the Language of Location
The ability for computers to dialogue with people about the location of objects in their environment would be very useful for applications such as smart houses, computer-assisted assembly, and verbal control of robots. While the computer vision and natural language understanding subfields have made great strides, much work remains to enable their integration.

This thesis addresses the perceptual grounding of spatial language, specifically the language of location (often expressed in English by prepositions such as "in", "on", "above", etc.). We propose to build a system that can communicate with people about the location of objects in a realistic 3-D environment. We take an interaction-based approach, which maintains that our spatial concepts are based on functional interactions with the world such as active vision, manipulation, and navigation. By carefully considering important interactions, we seek to identify a small set of critical visual features that can be calculated from images of a 3-D real world setting, but that do not require 3-D reconstruction. We focus on developing a conceptual level of representation that avoids one-to-one encoding of particular words and creates a foundation for grounding spatial language in general.

Research issues that will be addressed include: choosing useful visual features; creating a probabilistic framework for classification that accounts for various sources of uncertainty; and evaluation scenarios.