Login
Computer Science @ Rochester
Friday, January 27, 2012
12:45 PM
CSB 703
Pramod Sankar
Xerox
Towards Practical and Scalable Retrieval from Multimedia Collections
Abstract: Multimedia retrieval (MR) is as much a challenging problem as an exciting one. We address two major challenges toward successful MR systems: i) meaningful text-based annotation of multimedia content and ii) scalability of recognition/understanding modules to large datasets. For the first challenge, we exploit the presence of parallel text content, to bootstrap the annotation process. The parallel text is aligned with multimedia using partly-reliable clues that are extracted from the text and video. The resulting alignment allows the user to retrieve all shots where for example, Charlie Chaplin is picking something up or Elaine is answering the phone in Seinfeld. Toward addressing the scalability challenge, we propose a novel Reverse Annotation framework. This framework replaces naïve classification with a carefully designed combination of indexing and classification. Such an approach has made feasible to build a large scale retrieval system over document images from 1000 scanned Telugu books. This is the largest searchable non-English document image collection, in the world. We have applied similar techniques to classical image classification and object detection methods. Similar techniques can also be applied to other domains that involve large scale classification. My thesis proves that MR systems that are closer to users' needs and expectations, can be built in practice.

Bio: Pramod is from the International Institute of Information Technology (IIIT), Hyderabad, India, where he will shortly be receiving his doctoral degree in Computer Science. His PhD thesis focused on building retrieval systems for large-scale multimedia collections including movies, videos and images. Pramod’s research interests are in computer vision, machine learning, and information retrieval. During his graduate studies, he spent two terms as a visiting researcher at the University of Oxford, UK, working on video retrieval from TV shows and movies. At the Centre for Visual Information Technology, IIIT Hyderabad, Pramod was an active member of the Digital Library of India project and the OCR consortium project. He also has work experience as a software engineer at Pentagram Research Center, Hyderabad and as an intern at Google. Pramod is currently pursuing his postdoctoral research at Xerox Research Webster working on smart-phone solutions.