This thesis addresses the perceptual grounding of spatial language, specifically the language of location (often expressed in English by prepositions such as "in", "on", "above", etc.). We propose to build a system that can communicate with people about the location of objects in a realistic 3-D environment. We take an interaction-based approach, which maintains that our spatial concepts are based on functional interactions with the world such as active vision, manipulation, and navigation. By carefully considering important interactions, we seek to identify a small set of critical visual features that can be calculated from images of a 3-D real world setting, but that do not require 3-D reconstruction. We focus on developing a conceptual level of representation that avoids one-to-one encoding of particular words and creates a foundation for grounding spatial language in general.
Research issues that will be addressed include: choosing useful visual features; creating a probabilistic framework for classification that accounts for various sources of uncertainty; and evaluation scenarios.