Early Detection of User Engagement Breakdown in Spontaneous Human-Humanoid Interaction

Atef Ben Youssef 1, 2 Chloé Clavel 1, 2 Slim Essid 1, 2
1 S2A - Signal, Statistique et Apprentissage
LTCI - Laboratoire Traitement et Communication de l'Information
Abstract :

This paper presents a supervised classification system for forecasting a potential user engagement breakdown in human-robot interaction. We define engagement breakdown as a failure to successfully complete a predefined interaction scenario, where the user leaves before the expected end. The goal is thus to detect as early as possible such a potential engagement breakdown during the interaction between a human and a humanoid robot. To this end, we exploit a dataset that we have collected in real-world conditions where a set of participants were left to spontaneously engage in an interaction with the robot. The dataset is labeled according to the presence/absence of engagement breakdown. This study investigates the use of a multimodal approach to this problem, where a set of non-verbal features is considered to characterize the users' behavior. The use of combined multimodal features is found to effectively improve the performance of the system. The optimal set of data streams useful for this task is the combination of the distance to the robot, gaze and head motion, as well as facial expressions and speech. We study the time extent over which a user's departure can be anticipated. We find that this ability to anticipate the departure depends on the window during which we observe the user behavior.

Document type :
Journal articles
Complete list of metadatas

https://hal.telecom-paristech.fr/hal-02288043
Contributor : Telecomparis Hal <>
Submitted on : Friday, September 13, 2019 - 5:37:32 PM
Last modification on : Thursday, October 17, 2019 - 12:37:03 PM

Identifiers

  • HAL Id : hal-02288043, version 1

Citation

Atef Ben Youssef, Chloé Clavel, Slim Essid. Early Detection of User Engagement Breakdown in Spontaneous Human-Humanoid Interaction. IEEE Transactions on Affective Computing , 2019. ⟨hal-02288043⟩

Share

Metrics

Record views

16