Emotionally Adaptive Virtual Health Assistants for Enhanced Social Presence and Trust

Abstract

The integration of emotional intelligence into virtual health assistance systems has the potential to revolutionize the way users interact with healthcare applications. By adapting to users’ emotional states, virtual characters or avatars can provide more personalized and engaging interactions, leading to improved user satisfaction and overall well-being. This study presents the development of photorealistic virtual assistants using Unreal Engine’s Metahuman Creator, which can adapt their speech and facial expressions to match the user’s emotional state. The system, which synchronizes Metahuman avatars with an AI-powered chatbot, enables the display of emotions such as happiness, excitement, boredom, and sadness, creating a more natural and engaging conversation. A user study with 21 participants evaluated the system’s effectiveness, revealing that emotion-adaptive avatars notably enhance experience quality and emotional engagement compared to neutral virtual assistants. The findings of this study have significant implications for the development of more effective and user-friendly healthcare applications and highlight the need for further research into the impact of emotion-adaptive virtual assistants on user engagement, well-being, and adherence to health recommendations.

BibTeX

@inproceedings{voigtantons2025c92,
  author    = {Ashrafi, N. and Schorlemmer, J. and Vona, F. and Kojic, T. and Hillmann, S. and Braytee, A. and Wang, Y.-K. and Kocaballi, B. and Moreira, C. P. and Möller, S. and Voigt-Antons, J.-N.},
  title     = {Emotionally Adaptive Virtual Health Assistants for Enhanced Social Presence and Trust},
  year      = {2025},
  booktitle = {Paper presented at the first workshop on Designing Safe and Effective Intelligent Virtual Agents (IVAs) for Health Care: Risks, Benefits, and Best Practices (DESIGN IVA HC 2025), Berlin, Germany},
  doi       = {10.1145/3742886.3756698},
}