Research Statement: My research focuses on creating systems that use both human and machine computation to solve problems quickly and reliably. My work has introduced the idea of continuous real-time crowdsourcing, and the crowd agent model which uses groups of workers submitting input simultaneously to create a collective intelligence capable of completing tasks better than any constituent member of the crowd via mediation by an automated process. My work is supported by the NSF and a Microsoft Research PhD Fellowship.
Legion | Real-Time Control of Existing Interfaces
Legion enables real-time crowd control of existing interfaces by making the crowd act as a single, more reliable, worker. While most current crowdsourcing systems enable offline human computation, Legion utilizes a continuous closed-loop interface with workers to allow real-time control. Each worker functions independantly, and Legion combines their input into a single control stream using a selectible input mediator.
Legion:Scribe | Real-Time Captioning
Real-time captions are vital for many deaf and hard of hearing (DHH) people. However, existing options are severely limited in terms of accuracy (automatic system), or cost and availability (professionals). Legion:Scribe enables any worker who can hear and type to contribute to real-time captioning by automatically merging the input of multiple simultaneous workers. In this way, Scribe has the ability to outperform any individual, even a professional, at performing this complex cognitive and motor task.
Legion:AR | Real-Time Video Labeling
Activity recognition (AR) is useful in a variety of different settings, including helping older or cognitively impaired users live safely and independently. However, current automated approaches are brittle and require expensive training prior to use in real-world domains. Legion:AR ('legionnaire') helps make activity recognition more easily available by using the crowd to provide training data for an automatic system only when help is needed. This allows AR systems to be deployed immediately and train on-the-fly in a wider variety of new situations than would otherwise be possible, while also improving robustness to errors and new domains.
Chorus | Speaking With the Crowd
Automated systems are not yet able to engage in a robust dialogue with users due the complexity and ambiguity of natural language. However, humans can easily converse with one another and maintain a shared history of past interactions. Chorus is a system that enables users to hold a two-way conversation with a crowd acting as a single agent. Chorus is capable of maintaining a consistent, on-topic conversation with users across multiple sessions, despite constituent individuals perpetually joining and leaving the crowd by using a curated shared dialogue history.