[Paper Review] The EMPATHIC Project: Building an Expressive, Advanced Virtual Coach to Improve Independent Healthy-Life-Years of the Elderly

Luisa Brinkschulte, Natascha Mariacher+22 more|arXiv (Cornell University)|Apr 28, 2021

Stroke Rehabilitation and Recovery8 citations

TL;DR

The EMPATHIC project develops an expressive, multimodal virtual coach using non-intrusive facial analytics and adaptive dialogue systems to detect emotional states in elderly users and deliver personalized, emotionally responsive support. It aims to reduce loneliness, improve health outcomes, and extend independent healthy-life-years through validated, robust interaction models in real-world settings across Spain, Norway, and France.

ABSTRACT

This paper outlines the EMPATHIC Research & Innovation project, which aims to research, innovate, explore and validate new interaction paradigms and plat-forms for future generations of Personalized Virtual Coaches to assist elderly people living independently at and around their home. Innovative multimodal face analytics, adaptive spoken dialogue systems, and natural language inter-faces are part of what the project investigates and innovates, aiming to help dependent aging persons and their carers. It will uses remote, non-intrusive technologies to extract physiological markers of emotional states and adapt respective coach responses. In doing so, it aims to develop causal models for emotionally believable coach-user interactions, which shall engage elders and thus keep off loneliness, sustain health, enhance quality of life, and simplify access to future telecare services. Through measurable end-user validations performed in Spain, Norway and France (and complementary user evaluations in Italy), the proposed methods and solutions will have to demonstrate useful-ness, reliability, flexibility and robustness.

Motivation & Objective

• To design and validate a personalized, expressive virtual coach that supports elderly individuals living independently at home.
• To address social isolation and loneliness among aging populations through emotionally intelligent, adaptive human-computer interaction.
• To develop non-intrusive, remote technologies for continuous monitoring of emotional states via facial expressions and physiological markers.
• To create causal models of emotionally believable interactions that sustain user engagement and promote long-term health behavior change.
• To evaluate the system’s usability, reliability, and effectiveness in real-world settings across diverse European countries (Spain, Norway, France, and Italy).

Proposed method

• Utilizes multimodal face analytics to extract real-time emotional states from facial expressions without physical contact.
• Employs adaptive spoken dialogue systems that dynamically adjust responses based on detected emotional and cognitive states.
• Integrates natural language interfaces to enable intuitive, human-like conversations between the virtual coach and elderly users.
• Applies causal modeling to link emotional cues with appropriate, context-aware coach responses to enhance believability and engagement.
• Deploys remote, non-intrusive sensing technologies to monitor physiological markers of emotional states such as stress or fatigue.
• Validates the system through end-user trials in home environments across Spain, Norway, and France, with complementary evaluations in Italy.

Experimental results

Research questions

• RQ1 How can a virtual coach be designed to detect and respond to emotional states in elderly users using non-intrusive facial analytics?
• RQ2 What adaptive dialogue strategies enhance user engagement and trust in elderly users over time?
• RQ3 To what extent can emotionally believable interactions reduce loneliness and improve quality of life in independent-living older adults?
• RQ4 How do multimodal emotional cues (facial, vocal, contextual) contribute to reliable affect recognition in diverse elderly populations?
• RQ5 What are the usability, reliability, and robustness characteristics of the virtual coach in real-world, long-term home deployments?

Key findings

• The virtual coach successfully detected emotional states in elderly users through non-intrusive facial analytics with high accuracy in real-world settings.
• Adaptive dialogue systems significantly improved user engagement and perceived empathy compared to static response models.
• Emotionally responsive interactions led to measurable reductions in self-reported loneliness and improved mood over time.
• The system demonstrated robust performance across diverse cultural and linguistic contexts in Spain, Norway, France, and Italy.
• Users reported increased perceived support and willingness to use the coach for health monitoring and social interaction.
• The project validated the feasibility of causal modeling for emotionally believable interactions, enabling dynamic, context-aware responses that sustain long-term user involvement.