Human-Computer Interaction

Human-Computer Interaction (HCI) concerned with the study, design, construction and implementation of human-centric interactive computer systems. Work in the Rochester HCI group (ROCHI) includes human nonverbal behavior analysis, social skills training, applied machine learning, educational technology, accessible computing, ubiquitous computing. Visit the HCI website.


Henry Kautz

Henry Kautz is chair of the Robin & Tim Wentworth Director of the Institute for Data Science and professor in the Department of Computer Science at the University of Rochester.  He performs research in social media, machine learning, pervasive computing, automated planning, and assistive technology. His academic degrees include an AB in mathematics from Cornell University, an MA in creative writing from the Johns Hopkins University, an MSc in computer science from the University of Toronto, and a PhD in computer science from the University of Rochester. He was a researcher and department head at Bell Labs and AT&T Laboratories until becoming a professor in the Department of Computer Science and Engineering of the University of Washington in 2000. He joined University of Rochester in 2006. He was president (2010-2012) of the Association for the Advancement of Artificial Intelligence, and is a fellow of the AAAI, a fellow of the American Association for the Advancement of Science, and a recipient of the IJCAI Computers and Thought Award.

M. Ehsan Hoque

M. Ehsan Hoque is an assistant professor in the Department of Computer Science at the University of Rochester. His research focuses on (1) designing and implementing new algorithms to sense subtle human nonverbal behavior, (2) enabling new behavior sensing for human-computer interaction, and (3) inventing new applications of emotion technology in high-impact social domains such as social skills training, public speaking, and assisting individuals who experience difficulties with social interactions.

Hoque received his PhD in 2013 from the Massachusetts Institute of Technology. His PhD thesis — “Computers to Help with Conversations: Affective Framework to Enhance Human Nonverbal Skills” — was the first of its kind to demonstrate that it is possible for people to learn and improve their social skills by interacting with an automated system; the thesis was showcased at the MIT Museum as one of the most unconventional inventions at MIT. Dr. Hoque has won a number of awards, including the IEEE Gold Humanitarian Fellowship, Best Paper Award at Ubiquitous Computing (UbiComp), Best Paper Nominations at Automated Face and Gesture Recognition (FG), and Intelligent Virtual Agents (IVA), and NSF CRII (Pre-CAREER) Award.

Philip Guo

Philip Guo is an assistant professor of computer science at the University of Rochester. He researches human-computer interaction, software engineering, and online learning. In particular, to enable learning programming at scale, he created a code visualization and social learning platform called Online Python Tutor (, which has been used by over 1.5 million people in over 180 countries to visualize over 15 million pieces of code.

Philip received S.B. and M.Eng. degrees in EECS from MIT in 2006 and a Ph.D. in Computer Science from Stanford in 2012. His Ph.D. dissertation was one of the first to create software development tools for the unique needs of computational researchers and data scientists. Before becoming a professor in 2014, he developed online learning tools as a software engineer at Google, a visiting researcher at edX, and a postdoc at MIT CSAIL.

Project Pages

Project NameBrief Summary

VizWiz is an iPhone application aimed at enabling blind people to recruit remote sighted workers to help them with visual problems in nearly real-time.


WebAnywhere is a free web-based screen reader enabling blind web users to benefit from the availability of public computers. Information on the web can be accessed from any computer that has a sound card without the need to install screen-reader software.


Motivation of Legion was to provide a quick way of bootstrapping highly-robust, intelligent assistive robots. Such systems usually require significant (and costly) training to work automatically, are prone to errors, and so can often be controlled remotely by experts. Legion supports the flexible control of such existing remote-control interfaces.


Scribe is a new approach in which groups of non-expert captionists (anyone who can hear and type) collectively caption speech in real-time.


Chorus enables real-time, two-way natural language conversation between an end user and a crowd acting as a single agent. Chorus is capable of maintaining a consistent, on-topic conversation with end users across multiple sessions, despite constituent individuals perpetually joining and leaving the crowd.