Snowy colonnade

Benjamin Kane


URCS Grad Student


Bio

I'm a final-year PhD student of Computer Science at the University of Rochester, with a focus on Artificial Intelligence. Before that, I graduated from the University of Rochester with a BS in Computer Science as well as a BA in Economics, giving me an interest in the practical applications of intelligent agents towards the social sciences and real-world domains. Currently, my general research interests include dialogue management for virtual conversational agents, knowledge representation & commonsense reasoning, and computational linguistics (particularly, studying the communication and semantics of states related to dialogue, such as belief, desire, and intention).


Under the advisorship of Dr. Lenhart Schubert, Ehsan Hoque, and Dr. Aaron Steven White, some projects that I have worked on include:

  • Eta Dialogue Manager: Large Language Models have recently been demonstrated to be competent conversational agents in various domains, but they remain prone to hallucination, and are often costly in complex simulations. What if an agent could "think fast and slow", using a flexible and user-configured combination of LLMs and symbolic pattern transduction? This has been the focus of my work on the Eta Dialogue Manager (originally a pun on "Etaoin shrdlu" and SHRDLU, Terry Winograd's famous Blocks World agent) — a dialogue manager that uses "transducers" to interpret the user, reason, update its conversation plan, and generate responses, all based on a core "event schema" representation encoding knowledge about expected dialogue flows. Some of the subsequent projects have been developed using this dialogue framework.

  • SOPHIE agent (end-of-life communication): A significant challenge that healthcare workers face is delivering difficult news to patients in a manner that is explicit, yet also empathetic and delivered in a way that empowers the patient. Continually practicing these skills through simulated medical scenarios is critical for optimal patient outcomes, yet is made difficult to the limited supply and relative lack of demographic diversity among human standardized patient actors. The SOPHIE project is an interdisciplinary effort (in collaboration with the ROC HCI lab and the URMC Center for Communication and Disparities Research) to create a multimodal virtual standardized patient using AI, with the goal of helping doctors to improve communication in end-of-life dialogue scenarios through automated feedback. I helped to develop the dialogue capabilities of SOPHIE, resulting in a successful pilot study of the system with medical students from the URMC.

  • Blocks World QA/tutoring agent: The Blocks World domain - consisting of several named/colored blocks on a flat 3D surface - is a classic testbed for spatial reasoning and planning capabilities. Despite its relative simplicity, tackling this domain in its full complexity requires the agent to exhibit a rich set of functional capabilities, ranging from vision to natural language understanding. I created a conversational agent that is capable of spatial/temporal QA and collaborative structure-building in a physically situated blocks world setting, given a "spatial reasoning specialist" module that calculates spatial relations in terms of low-level primitives. This work involved the integration of several capabilities, including a semantic parser capable of processing user inputs into structural logical forms (ULF), a spatial reasoner and planner, and an episodic memory for answering questions about the history of the interaction. Supported by the DARPA Communicating with Computers grant and developed in collaboration with Georgiy Platonov, who created the vision and spatial reasoning/planning modules.

  • MegaIntensionality: Certain English sentences can give rise to inferences pertaining to the belief, desire, and intentional states of the agents in the sentences. Not only are these inferences of import to Linguists — as they are putatively related to patterns in the syntactic distributions of verbs and provide evidence for (or against) certain generalizations — but they are also relevant to dialogue systems, since (inferred) beliefs and desires of the user can be used to steer a conversation. The focus of the MegaIntensionality project (a collaboration with Will Gantt and Aaron Steven White) is to model and analyze these inferences given lexical-scale human annotations. We collected a dataset of such annotations across a variety of verbs and syntactic contexts, and conducted analysis of the data using Bayesian mixed-effects mixture models. We also demonstrated that mixed-effects models can be integrated with standard supervised natural language inference (NLI) models to capture differences between individual annotators during training time.


In my free time, I'm an avid martial artist, and am currently a blue belt in Brazilian Jiu Jitsu, and have previously earned a black belt (1st Dan) in Isshin-ryū Karate. I also dabble in music (flute and synthesizers), and am an administrator/texture developer/builder for WesterosCraft.


Resume

I am currently on the market for post-graduation work, primarily within industry.

Click here to view Resume.


Thesis

Schema-Based Dialogue Management: From Friendly Peer to Virtual Standardized Cancer Patient


Thesis Proposal

Schema-Guided Dialogue Management for Virtual Conversational Agents


Area Paper

Management of Open-Domain Dialogue for Virtual Conversational Agents


Publications

Conference
  1. Kane B.; Schubert L. We are what we repeatedly do: Inducing and deploying habitual schemas in persona-based responses. EMNLP. December 6-10, 2023, Singapore.

  2. Haut K.; Wohn C.; Kane B.; Carroll T.; Guigno C.; Kumar V.; Epstein R.; Schubert L.; Hoque E. Validating a virtual human and automated feedback system for training doctor-patient communication skills. Affective Computing and Intelligent Interaction (ACII). September 10-13, 2023, Boston, USA.

  3. Kane B.; Gervits F.; Scheutz M.; Marge M. A System For Robot Concept Learning Through Situated Dialogue. Special Interest Group on Discourse and Dialogue (SIGDIAL). September 7-9, 2022, Edinburgh, UK. [Poster]

  4. Platonov G.; Kane B.; Schubert L. K. Generating Justifications in a Spatial Question-Answering Dialogue System for a Blocks World. Reasoning and Interaction (ReInAct). October 4-6, 2021, Virtual.

  5. Kane B.; Gantt W.; White A. S. Intensional Gaps: Relating veridicality, factivity, doxasticity, bouleticity, and neg-raising. Semantics and Linguistic Theory (SALT) 31. May 7-9, 2021, Virtual.

  6. Gantt W.; Kane B.; White A. S. Natural Language Inference with Mixed Effects. The 9th Joint Conference on Lexical and Computational Semantics (*SEM 2020). December 12-13, 2020, Virtual.

  7. Kane B.; Platonov G.; Schubert L. K. Registering Historical Context for Question Answering in a Blocks World Dialogue System. Text, Speech, and Dialogue (TSD). September 8-11, 2020, Virtual.

  8. Platonov G.; Kane B.; Gindi A.; Schubert L. K. A Spoken Dialogue System for Spatial Question Answering in a Physical Blocks World. Special Interest Group on Discourse and Dialogue (SIGDIAL). July 1-3, 2020, Virtual.

  9. Razavi S. Z.; Kane B.; Schubert L. K. Investigating Linguistic and Semantic Features for Turn-Taking Prediction in Open-Domain Human-Computer Conversation. InterSpeech. September 15-19, 2019, Graz, Austria.

Journal
  1. Kane B.; Giugno C.; Schubert L.; Haut K.; Wohn C.; Hoque E. Managing Emotional Dialogue for a Virtual Cancer Patient: A Schema-Guided Approach. IEEE Transactions on Affective Computing. January 2023.

  2. Rafayet Ali M.; Sen T.; Kane B.; Bose S.; Carroll T. M.; Epstein R.; Schubert L.; Hoque E. Novel Computational Linguistic Measures, Dialogue System, and the Development of SOPHIE: Standardized Online Patient for Healthcare Interaction Education. IEEE Transactions on Affective Computing. January 2021.

Workshop
  1. Kane B.; Schema-Guided Dialogue Management for Virtual Conversational Agents. Young Researchers' Roundtable on Spoken Dialog Systems (YRRSDS), at the Special Interest Group on Discourse and Dialogue (SIGDIAL). September 5-6, 2022, Edinburgh, UK. [Poster]

  2. Kim G. L.; Kane B.; Duong V.; Mendiratta M.; McGuire G.; Sackstein S.; Platonov G.; Schubert L. K. Generating Discourse Inferences from Unscoped Episodic Logical Formulas. 1st Int. Workshop on Designing Meaning Representations (DMR), at the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019). Aug 1, 2019, Florence, Italy.

  3. Razavi S. Z.; Schubert L. K.; Kane B.; Rafayet Ali M.; Van Orden K. A.; Ma T. Dialogue Design and Management for Multi-Session Casual Conversation with Older Adults. Workshop on User-Aware Conversational Agents (User2Agent), at the 24th Int. Conf. on Intelligent User Interfaces (ACM IUI 2019). March 17-20, 2019, Los Angeles, USA.

  4. Kane B.; Luo, J. 2018. Do the Communities We Choose Shape our Political Beliefs? A Study of the Politicization of Topics in Online Social Groups. Workshop on Big Social Media Data Management and Analysis (BSMDMA), at the IEEE International Conference on Big Data. December 10-13, 2018, Seattle, USA.

Preprint
  1. Kane B.; Schubert L. Get the gist? Using large language models for few-shot decontextualization. arXiv:2310.06254.

  2. Kane B.; Giugno C.; Schubert L. K.; Haut K.; Wohn C.; Hoque E. A Flexible Schema-Guided Dialogue Management Framework: From Friendly Peer to Virtual Standardized Cancer Patient. arXiv:2207.07276.

  3. Kane B.; Platonov G.; Schubert L. K. History-Aware Question Answering in a Blocks World Dialogue System. arXiv:2005.12501.

Other
  1. Presentation at the Second Workshop on Processing and Evaluating Event Representations (PEER2023). March 31, 2023, Rochester, USA. [Slides]

  2. Hosted workshop on Probabilistic Programming with Pyro for ROC HCI group. June 29, 2023, Rochester, USA. [Slides] [Code]