[Paper Review] Learning to Engage with Interactive Systems: A field Study

Lingheng Meng, Daiwei Lin+4 more|arXiv (Cornell University)|Apr 14, 2019

Embodied and Extended Cognition4 citations

TL;DR

This paper proposes a reinforcement learning framework that estimates human engagement from behavioural cues and uses it to learn adaptive, engaging interactions in physical agents. Evaluated in a museum setting with an interactive sculpture, the system outperformed scripted behaviours in engagement and likeability, demonstrating the effectiveness of learning adaptive interaction strategies in natural environments.

ABSTRACT

Physical agents that can autonomously generate engaging, life-like behaviour will lead to more responsive and interesting robots and other autonomous systems. Although many advances have been made for one-to-one interactions in well controlled settings, future physical agents should be capable of interacting with humans in natural settings, including group interaction. In order to generate engaging behaviours, the autonomous system must first be able to estimate its human partners' engagement level. In this paper, we propose an approach for estimating engagement from behaviour and use the measure within a reinforcement learning framework to learn engaging interactive behaviours. The proposed approach is implemented in an interactive sculptural system in a museum setting. We compare the learning system to a baseline using pre-scripted interactive behaviours. Analysis based on sensory data and survey data shows that adaptable behaviours within a perceivable and understandable range can achieve higher engagement and likeability.

Motivation & Objective

• To develop a method for estimating human engagement in real-time during interactions with physical agents.
• To enable autonomous systems to learn engaging behaviours through reinforcement learning using engagement as a reward signal.
• To evaluate the effectiveness of learned adaptive behaviours compared to pre-scripted interactions in a natural, uncontrolled environment like a museum.

Proposed method

• The system estimates engagement using behavioural cues collected from sensors during human interactions.
• A reinforcement learning framework uses engagement estimates as a reward signal to optimize interactive behaviours.
• The learning process is implemented in an interactive sculptural system designed for public museum environments.
• The system adapts behaviours in real time based on feedback from engagement estimation, enabling dynamic interaction.
• Baseline comparisons use pre-scripted, non-adaptive behaviours to evaluate performance.
• Sensory data and user surveys are collected to validate engagement and likeability outcomes.

Experimental results

Research questions

• RQ1 Can engagement be reliably estimated from behavioural data in natural, unstructured human-physical agent interactions?
• RQ2 Does learning adaptive behaviours through reinforcement learning based on engagement estimation lead to higher user engagement than pre-scripted behaviours?
• RQ3 How do users perceive and rate the likeability of adaptive versus scripted interactive behaviours in a public setting?

Key findings

• Adaptive behaviours learned through reinforcement learning resulted in significantly higher user engagement compared to pre-scripted behaviours.
• Users reported greater likeability for the learning-based system than for the baseline scripted system.
• The engagement estimation model successfully captured dynamic changes in user interest during interactions.
• The system's ability to adapt in real time contributed to more natural and compelling interactions.
• Sensory data and survey results jointly validated the effectiveness of the learning framework in a real-world context.
• The study demonstrates that perceivable and understandable behavioural adaptation enhances user experience in public interactive systems.