Research Statement:
My research focuses on creating systems that use both human and machine computation
to solve problems quickly and reliably. My work has introduced the idea of continuous
real-time crowdsourcing, and the crowd agent model which uses groups of
workers submitting input simultaneously to create a collective intelligence capable
of completing tasks better than any constituent member of the crowd via mediation
by an automated process. My work is supported by the NSF and a Microsoft Research PhD
Fellowship.
Legion | Real-Time Control of Existing Interfaces
Legion enables real-time crowd control of existing interfaces by
making the crowd act as a single, more reliable, worker. While most current crowdsourcing systems
enable offline human computation, Legion utilizes a continuous closed-loop interface with workers
to allow real-time control. Each worker functions independantly, and Legion
combines their input into a single control stream using a selectible input mediator.
Legion:Scribe | Real-Time Captioning
Real-time captions are vital for many deaf and hard of hearing (DHH) people. However, existing
options are severely limited in terms of accuracy (automatic system), or cost and availability
(professionals). Legion:Scribe enables any worker who can hear and type to contribute to real-time
captioning by automatically merging the input of multiple simultaneous workers. In this way, Scribe
has the ability to outperform any individual, even a professional, at performing this complex
cognitive and motor task.
Legion:AR | Real-Time Video Labeling
Activity recognition (AR) is useful in a variety of different settings,
including helping older or cognitively impaired users live safely and independently. However, current
automated approaches are brittle and require expensive training prior to use in real-world domains.
Legion:AR ('legionnaire') helps make activity recognition more easily available by using the crowd to
provide training data for an automatic system only when help is needed. This allows AR systems to
be deployed immediately and train on-the-fly in a wider variety of new situations than would otherwise
be possible, while also improving robustness to errors and new domains.
Chorus | Speaking With the Crowd
Automated systems are not yet able to engage in a robust dialogue with users due the
complexity and ambiguity of natural language. However, humans can easily converse with one another and maintain a shared
history of past interactions. Chorus is a system that enables users to hold a two-way conversation with a crowd acting
as a single agent. Chorus is capable of maintaining a consistent, on-topic conversation with users across multiple
sessions, despite constituent individuals perpetually joining and leaving the crowd by using a curated shared dialogue
history.




