Login
Computer Science @ Rochester
Monday, October 31, 2011
11:00 AM
Computer Studies Bldg. 209
Yu-An Sun
Xerox
How to Assure the Quality of Human Computation Tasks When Majority Voting Fails?
Human computation is a growing research field. It holds the promise of humans and computers working seamlessly together to implement powerful systems, but this requires in depth knowledge about the communication barrier between man and machine. In this talk, we first discuss the creation of a relatively complicated human computation process. We test our process on a set of real world data, and discuss the results.

Secondly, quality assurance remains a key topic in human computation research. Prior work indicates that independent agreement is effective for low difficulty tasks, but performs poorly for moderately difficult tasks since the majority of responses may be inaccurate. We present experimental results showing that humans are better at identifying correct answers than producing correct answers in such moderately difficult tasks. This motivates us to propose a simple quality control process based on tournament selection. We demonstrate that tournament selection repeatably finds the correct answers in as few as 3 to 5 rounds.

Bio: Yu An Sun is the new Area Manager for the Social and Web Technologies Area within the Computing and Information Services Lab. Yu An's competency area is chartered to understand and define the opportunity spaces around social software design, human and collaborative computations as well as web and content mining to create novel and differentiated solutions and services. Specific research areas in her competency area include human computation and computational economics.

Before joining Xerox Research Center Webster in 2008, Yu-An received a doctoral degree in computer science from George Washington University. She also holds a Master in Business Administration degree from National Central University.

Refreshments will be provided at 10:45AM