Time
Friday, 04/09/21 at 12:00 PM - 1:00 PM (Central)
Title
Detecting disappointment and annoyance from prosody in merchant-customer conversations
Abstract
For the automated tuning of user interfaces, it can be helpful to have data on the places where users have difficulty. Detecting this is, in general, hard. For spoken dialog, with human agents or with dialog systems, this is usually done by soliciting explicit judgments, for example, with “on a scale from 1 to 5, how was I today?” However, this is intrusive and does not identify the specific places where dissatisfaction was felt. We propose instead to exploit the tendencies for speakers to continuously indicate their satisfaction/dissatisfaction level by changing the way they speak, either intentionally or unintentionally. To investigate, we collected a corpus of 147 mock merchant-customer conversations, each 1 to 3 minutes in length, including some in a scenario that prevented the speakers from reaching a mutually satisfactory outcome. We will report the results of applying machine learning models to prosodic features computed over this data. By late March, we expect to find better-than-baseline ability to classify a dialog as containing or not containing dissatisfaction and to have identified specific prosodic configurations that are generally useful as markers of dissatisfaction.
Bio
Jonathan E. Avila is a PhD student in Computer Science and research assistant at the University of Texas at El Paso. He is a member of the Interactive Systems Group, co-lead by his advisor Dr. Nigel G. Ward, where he is conducting research in models of interaction and the engineering of interactive systems. His research interests include emotion recognition and feedback on communication.