Login
Computer Science @ Rochester
Friday, February 11, 2011
12:45 PM
CSB 703
Naushad UzZaman
University of Rochester
Multimodal Summarization of Complex Sentence
Practice talk for International Conference on Intelligent User Interfaces (IUI), Palo Alto, California, 2011.

Abstract:

In this paper, we introduce the idea of automatically illustrating complex sentences as multimodal summaries combining pictures, structure and simplified text. By including text and structure in addition to pictures, multimodal summaries provide additional clues of what happened, who did it, to whom and how - to people who may have difficulty reading or looking to skim quickly. We present ROC-MMS, a system for automatically creating multimodal summaries (MMS) of complex sentences by generating pictures, textual summaries and structure. We show that pictures alone are insufficient to help people understand most sentences, especially for readers who are unfamiliar with the domain. An evaluation of ROC-MMS in the Wikipedia domain illustrates both the promise and challenge of automatically creating multimodal summaries.