Abstract:
Multimedia retrieval (MR) is as much a challenging problem as an
exciting one. We address two major challenges toward successful MR
systems: i) meaningful text-based annotation of multimedia content and
ii) scalability of recognition/understanding modules to large
datasets. For the first challenge, we exploit the presence of parallel
text content, to bootstrap the annotation process. The parallel text
is aligned with multimedia using partly-reliable clues that are
extracted from the text and video. The resulting alignment allows the
user to retrieve all shots where for example, Charlie Chaplin is
picking something up or Elaine is answering the phone in Seinfeld.
Toward addressing the scalability challenge, we propose a novel
Reverse Annotation framework. This framework replaces naïve
classification with a carefully designed combination of indexing and
classification. Such an approach has made feasible to build a large
scale retrieval system over document images from 1000 scanned Telugu
books. This is the largest searchable non-English document image
collection, in the world. We have applied similar techniques to
classical image classification and object detection methods. Similar
techniques can also be applied to other domains that involve large
scale classification. My thesis proves that MR systems that are closer
to users' needs and expectations, can be built in practice.
Bio:
Pramod is from the International Institute of Information Technology
(IIIT), Hyderabad, India, where he will shortly be receiving his
doctoral degree in Computer Science. His PhD thesis focused on
building retrieval systems for large-scale multimedia collections
including movies, videos and images. Pramod’s research interests are
in computer vision, machine learning, and information retrieval.
During his graduate studies, he spent two terms as a visiting
researcher at the University of Oxford, UK, working on video retrieval
from TV shows and movies. At the Centre for Visual Information
Technology, IIIT Hyderabad, Pramod was an active member of the Digital
Library of India project and the OCR consortium project. He also has
work experience as a software engineer at Pentagram Research Center,
Hyderabad and as an intern at Google. Pramod is currently pursuing his
postdoctoral research at Xerox Research Webster working on smart-phone
solutions.