Mohammad Rafayet Ali

Ph.D. Student
Computer Science, University of Rochester
Email: mali7[at]cs.rochester.edu
Google Scholar Profile
LinkedIn Profile

I am a Ph.D. candidate in Computer Science at the University of Rochester. I have received my Master's degree from the University of Rochester in 2016 and a Bachelor's degree from BUET in 2013. Currently, I am working under Dr. Ehsan Hoque. My research is focused on Human-Computer Interaction (HCI). My work includes sensing, understanding and analyzing human affect, using machine learning techniques. In addition to affective computing, I work on applications focusing on improving communication skills and healthcare.

Projects

  • LISSA - Live Interactive Social Skills Assistance

    The LISSA system aims to help individuals improve their social skills by simulating face-to-face conversation and providing both real-time as well as post-session summary feedback on users’ nonverbal behavior. At the same time, it manages a fairly open-ended conversation with some degree of understanding. We have developed a HMM based model for generating real-time feedback.

    Watch the Video
  • PARK: Parkinson's Analysis with Remotely-Accessed Knowledge

    PARK is a web-based interface to collect the data from Parkinson’s patients. Using the online tool, patients are able to engage in a voice test and five motor tasks, from the standardized unified Parkinson's disease rating scale (UPDRS) test for diagnosing Parkinson's. These tasks are recorded via webcam and microphone of the user’s computer. The goal is to not only detect symptoms of Parkinson's, but to detect minute physical changes in the condition which provide important information about whether or not a patient requires a change in medication.

  • Aging and Engaging: a social conversational skills training program for older adults

    This project focuses on the older adults, who are socially isolated. The number of older adults in the US is 700 million. Social isolation can lead to many problems e.g., depression, suicidal tendency. This project is specially designed for older adults, allowing them to practice conversation with a virtual agent and receive feedback, which might help them starting a conversation with others reducing their isolation.

  • Cancer Patient Voice Project

    In this project we aim to identify the styles of coomunication between doctor and patient that help improve patient outcome and decision making. We analyze the text transcripts and audio recording between 386 late stage patients and 38 oncologists. Our analysis include, sentiment analysis, clustering of sentiment progression, speech act analysis.

  • ROC Comment

    ROC Comment is an automated interface for generating natural language comments on behavioral videos. We focus on the domain of public speaking. We collect a dataset of 196 public speaking videos from 49 individuals and gather 12,173 comments, generated by more than 500 independent human judges. We then train a k-Nearest-Neighbor (k-NN) based model by extracting prosodic (e.g., volume) and facial (e.g., smiles) features. Given a new video, we extract features and select the closest comments using k-NN model. We further filter the comments by clustering them using DBScan, and eliminating the outliers. We are working on generating comments using deep learning.

Publications

    2020

  1. M. Rafayet Ali, Daniel McDuff, Javier Hernandez, E. Ray Dorsey, and Ehsan Hoque, "Spatio-Temporal Attention and Magnification for Classification of Parkinson's Disease from Videos Collected via the Internet", IEEE Conference on Automatic Face and Gesture Recognition. (FG 2020)
  2. M. Tran, T K. Sen, K. G. Haut, M. Rafayet Ali, M. E. Hoque, "Are you really looking at me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video", IEEE Transactions on Affective Computing.
  3. 2019

  4. Raina Langevin, M. Rafayet Ali, Taylan Sen, Christopher Snyder, Taylor Myers, E. Ray Dorsey, Ehsan Hoque, "The PARK Framework for Automated Analysis of Parkinson's Disease Characteristics", ACM International Joint Conference on Pervasive and Ubiquitous Computing. (UbiComp 2019)
  5. M. Rafayet Ali, Taylan Sen, Viet-Duy Nguyen, Reza Rawassizadeh, Paul Duberstein, Ronald Epstein, and M Ehsan Hoque, "What Computers Can Teach Us About Doctor-Patient Communication: Leveraging Gender Differences in Cancer Care." International Conference on Affective Computing & Intelligent Interaction. (ACII 2019)
  6. M. Rafayet Ali, "Online Virtual Standardized Patient for Communication Skills Training", ACM Intelligent User Interfaces (IUI 2019).
  7. Zahra Razavi, Lenhart Schubert, Benjamin Kane, M. Rafayet Ali, Kimberly A. Van Orden, and Tianyi Ma , "Dialogue Design and Management for Multi-Session Casual Conversation with Older Adults", User-Aware Conversational Agents in conjunction with ACM Intelligent User Interfaces (IUI 2019).
  8. 2018

  9. M. Rafayet. Ali, Taylan Sen, Dev Crasta, Viet-Duy Nguyen, Ronald Rogge, M Ehsan Hoque, “What, When, and Why of Facial Expressions: An Objective Analysis of Conversational Skills in Speed-Dating Videos”, 2018 IEEE Conference on Automatic Face and Gesture Recognition (FG 2018).
  10. M. Rafayet. Ali, K.V. Orden, K. Parkhurst, V. Nguyen, S. Liu, P. Duberstein, M. E. Hoque, "Aging and Engaging: A Social Conversational Skills Training Program for Older Adults", ACM Intelligent User Interfaces (IUI 2018).
  11. A. Singhal, M. Rafayet. Ali, R. Baten, C. Kurumada, E. Marvin, and M Ehsan Hoque, “Analyzing the Impact of Gender on the Automation of Feedback for Public Speaking” Automatic Face and Gesture Recognition (FG 2018).
  12. S. Samrose, R. Zhao, J. White, V. Li, L. Nova, Y. Lu, M. Rafayet. Ali, M. E. Hoque, "CoCo: Collaboration Coach for Understanding Team Dynamics during Video Conferencing", Interactive, Mobile, Wearable, and Ubiquitous Computing (UbiComp 2018).
  13. 2017

  14. Taylan Sen, Mohammad Rafayet Ali, Ron Epstein, Paul Duberstien, and Mohammed Ehsan Hoque, "Modeling Doctor-Patient Communication with Affective Text Analysis" Affective Computing and Intelligent Interaction (ACII 2017).
  15. Mohammad Rafayet Ali, and Mohammed Ehsan Hoque, "Social Skills Training with Virtual Assistant and Real-time Feedback" 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2017).
  16. Seyedeh Zahra Razavi, Lenhart K. Schubert, Mohammad Rafayet Ali, and Mohammed Ehsan Hoque, "Managing Casual Spoken Dialogue Using Flexible Schemas, Pattern Transduction Trees, and Gist Clauses," Conference on Advances in Cognitive Systems, (ACS 2017).
  17. Kimberly Van Orden, Mohammad Rafayet Ali, Kimberly Parkhurst, Paul Duberstein, Mohammed (Ehsan) Hoque, "Aging & Engaging: The Development of an Automated Tool to Teach Social Engagement Skills for Older Adults," Annual Meeting of the American Association for Geriatric Psychiatry, (AAGP 2017).
  18. 2016

  19. Mohammad Rafayet Ali, Facundo Ciancio, Ru Zhao, Iftekhar Naim, Mohammed (Ehsan) Hoque, "ROC Comment: Automated Descriptive and Subjective Captioning of Behavioral Videos." 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2016).
  20. Seyedeh Zahra Razavi, Mohammad Rafayet Ali, Lenhart Schubert, Mohammed (Ehsan) Hoque,"The LISSA Virtual Human and ASD Teens: An Overview of Initial Experiments." The Sixteenth International Conference on Intelligent Virtual Agents (IVA 2016).
  21. Dev Crasta, M. Rafayet Ali, Ronald D. Rogge, M. Ehsan Hoque, "Can Automated Conversation Skills Coaching Improve Interpersonal Attractiveness and Effectiveness?", 17th annual convention of Society for Personality and Social Psychology(SPSP), 2016
  22. 2015

  23. Ali, M.R.; Crasta, D.; Li Jin; Baretto, A.; Pachter, J.; Rogge, R.D.; Hoque, M.E., "LISSA- Live Interactive Social Skill Assistance," International Conference on Affective Computing and Intelligent Interaction (ACII), 2015, pp.173-179, 21-24 Sept. 2015
  24. Ali, Mohammad Rafayet. "Automated conversation skills assistant.", International Conference on IEEE Affective Computing and Intelligent Interaction (ACII), 2015.

CV

Download my CV