This reading course explores research on computer systems that recognize human behavior from various kinds of sensor data. CS graduate students may register for credit, and students and faculty in all disciplines are welcome to attend.
Time: Thursdays 11:00am - 12:15pm
Location: CSB 632
Everyone will take one or two turns leading the discussion, beginning with an approximately 30 minute presentation of the material. Presenters should read background material as necessary to understand the details of the paper.
CS graduate students may take the course in order to fulfill the AI breadth requirement. In order to do so, each student must also hand in a short written summary of each week's readings (1 to 2 pages), and complete a substantive programming project.
Paper [Discussion Leader]
|Sept 6||Introduction to the Course [Henry Kautz]|
The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. Eric Horvitz, Jack Breese, David Heckerman, David Hovel, Koos Rommelse. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, Madison, WI, July 1998, pages 256-265. Morgan Kaufmann: San Francisco. [Sangho]
N. Oliver, E. Horvitz, and A. Garg. Layered Representations for Recognizing Office Activity, Proceedings of the Fourth IEEE International Conference on Multimodal Interaction (ICMI 2002) [Sangho]
Darnell Moore and Irfan Essa, "Recognizing Multitasked Activities from Video using Stochastic Context-Free Grammar", AAAI-02, 2002. [Randal Nelson]
Compare/Contrast Readings for Learning and Inferring Transportation Routines:
Location-Based Activity Recognition using Relational Markov Networks Lin Liao, Dieter Fox and Henry Kautz. Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, 2005. [Tongxin Bai]
Sangho Park will present a lecture on computer vision. For background, everyone should read:
Shinko Y. Cheng, Sangho Park, Mohan M. Trivedi, "Multi-spectral and
J. Wu, A. Osuntogun, T. Choudhury, M. Philipose, J. Rehg. A Scalable Approach to Activity Recognition Based on Object Use. In Proceedings of ICCV 2007. Rio de Janeiro, Brazil, October 2007. [Arrvindh]
Designing for seniors: The use of an intelligent prompting system for people with dementia. Alex Mihailidis, Jennifer Boger, Marcelle Canido, Jesse Hoey. July 2007 Interactions, Volume 14 Issue 4
A Decision-Theoretic Approach to Task Assistance for Persons with Dementia(2005). Jennifer Boger Pascal Poupart Jesse Hoey Craig Boutilier Geoff Fernie Alex Mihailidis Conf: International Joint Conference on Artificial Intelligence.
Philip R. Cohen & C. Raymond Perrault, Elements of a Plan-Based Theory of Speech Acts, Cognitive Science, Vol. 3, No. 3, Pages 177-212, 1979.
C. Raymond Perrault & James F. Allen, A plan-based analysis of indirect speech acts, Computational Linguistics, Volume 6 , Issue 3-4, Pages 167 - 182, 1980.
Topic: Modeling social networks from conversations
Main paper: Human Dynamics: Computation for Organization.
Modeling Conversational Dynamics as a Mixed Memory Markov Process.
Danny Wyatt, Tanzeem Choudhury, Jeff Bilmes, and Henry Kautz. A Privacy Sensitive Approach to Modeling Multi-Person Conversations. Appears in the Proceedings of International Joint Conference on Artificial Intelligence (IJCAI 2007). January 2007, Hyderabad, India.
|Dec 13||Guest lecture: Mark Bocko (UR ECE), Encoding Musical Expression|
1. Build a system that recognizes human behavior from sensor data using a hidden Markov model (HMM) and/or other probabilistic model. Data could come from one or more of the following sources:
(1) The data used in the paper Fine-Grained Activity Recognition by Aggregating Abstract Object UsageYou can download a zip file of all data here: a5data.zip.
Important data: Included are 10 days of data (e.g., day01.data). Also included are concatenations of all but one of those days (e.g., allbut01.data). The latter files allow you to train from multiple days. By training on all but the first day, then using the first day as test data, you have independent training and test sets. You can repeat using all but the second day and testing on the second day, etc.
Other files: If you're curious about the meanings of the numbers, state_mapping.txt and tag_mapping.txt give the actual state and object names. Also included are extended data files that give a time stamp for each reading, along with all secondary activities being performed concurrently at each time step (e.g., full.002.01.data.) Sensor readings for single activities are provided as well (e.g., single.door.01.data).
Training and test data are in the following format:
etc. Different lines correspond to distinct sensor readings, ordered by time.
(2) The MIT PlaceLab public data sets:
(3) Data you help collect in our own Laboratory for Assisted Cognition Environments.
(4) Data you collect in your own office by instrumenting your own desktop or laptop computer.
For an alternative to an HMM, see the following paper that uses a conditional random field model to explicitly recognize concurrent activities:
To make this a project with enough substance to qualify for graduate credit, you should pursue one of the following paths:
2. Design and implement a route-finding algorithm that automatically learns user preferences, both about specific places (e.g., never drive down a particular street) and general rules (e.g., avoid highways unless using one saves a large amount of time).
Create a simulated world (with a simulated driver) to test your algorithm.
For a very ambitious project, go on to real data, by obtaining a data logger to carry in your car, and a street map of Rochester in computer-readable form.
One way to think about this problem: Both the system and the user find routes using A*. The cost of a route is a linear function of factors such as its length, the number of turns, the kind of road, etc. The user's cost function is unknown. However, by watching the user, the system adjusts the parameters of its cost function so that it more accurately matches that of the user.