Login
Computer Science @ Rochester
Friday, March 09, 2007
12:30 PM
CSB 703
Carlos Gomez
University of Rochester
Annotating Incremental Understanding in a Multimodal Dialogue
Traditional speech to speech agents have been developed under a pipeline architecture where one module feeds the next in a serial manner. These have several shortcomings as earlier processing may only arrive at certain interpretations with information acquired in later stages of processing. As we move towards a continuous understanding architecture, there is a need to create corpora which can be used to develop this newer approach.

In this talk we present an annotation scheme that addresses the issue of incremental language processing in a multimodal dialogue. The annotation is done at the word level, a finer level of detail that breaks away from typical sentence level labeling. By delaying labels within the utterance until speaker's intentions are fully disambiguated, one can see how early understanding is achieved by examing the listener reactions.