In this presentation, I will discuss our approach. We develop a tracker that learns the appearance of each person in a video. Our tracker can also be used as a general purpose system for model-building and recognition.recognition, and model-building. To recognize activities, we use an "analysis by synthesis" approach that decouples recognition from the specific choice of activity categories. We have tested our system on hundreds of thousands of frames (several orders of magnitude more than previous approaches). We show results for tracking and activity recognition on both unscripted indoor and outdoor footage, a feature-length film, and historic sports footage (from the 2002 World Series and 1998 Winter Olympics).