Brandon's Summer Research Proposal


GENERALIZED TRACKING USING A NETWORK OF SENSORS

(Research Proposal - Brandon Sanders)

MOTIVATION
Intelligent environments promise users reduced workload and enhanced leisure time by automating repetitive or complex tasks. A network of cameras with local computational resources can provide the sensory and processing backbone for such an environment. However, before it may act intelligently, the network must first know something about the objects that populate its environment. I propose the method of generalized tracking (tracking without prior models) for robust knowledge discovery about objects and their interactions.

TECHNICAL DETAILS -- ESTIMATED RESULTS AND TIME REQUIREMENTS
Specifically, the proposed summer project uses a generalized tracker to create a hierarchical knowledge base describing the environment and the objects it contains. Successive levels in the knowledge base are built by forming hypotheses to explain the data in the current and lower levels.

Phase 1 (est. 2-3 weeks)
Gather a database of "relevant" multi-view image sequences covering the activities in an office environment. This database forms the lowest level of the hierarchy. Relevance is estimated by computing inter-frame differences or using other appropriate image coding techniques.

Phase 2 (est. 7-12 weeks)
Implement a generalized tracking framework that processes relevant image sequences in "easiest-first" order. The algorithm hypothesizes moving rigid objects to explain image changes. Each object hypothesis includes an appearance model (i.e., segmented views) and spatiotemporal locations for the object.

These object hypotheses form the second level of the hierarchy. When an image sequence cannot be adequately explained by the current set of objects and object trajectories, the algorithm will add objects, split objects, or remove objects in an effort to better explain the sequence. Tracking decisions are put off for as long as is feasible by tracking multiple hypotheses. The easiest correspondence problems are solved first because the information gained by solving them may help to explain more difficult tracking instances.

Phase 3 (est. 0-5 weeks)
Verify that the objects found by the tracker are useful for concept formation. Try clustering the objects by color and/or spatiotemporal location. Ideally the objects will cluster in ways humans find intuitive. For example, the body parts of a particular person are grouped together, as are segmented views of the same object taken in different locations. These clusters form the third level of the hierarchy.

RELEVANT PRIOR WORK
RELEVANT WORK IN PROGRESS

Email: sanders@cs.rochester.edu
Last change: February 9, 2001
HOME
URCS Home Page