Redeveloping ARCADE: User Experience with Personalized Conversational MetaHuman Avatars at Human Scale in Immersive Environments

Abstract

The “Redeveloping ARCADE” project is a research work inspired by the ARCADE project that stands for “An Augmented Reality Display Environment for Multimodal Interaction with Conversational Agents”. Our work presents a novel framework for deploying life-sized, highly realistic conversational avatars in physical environments. Our approach combines 3D facial scanning, mesh refinement, and Unreal Engine 5.5’s MetaHuman technology to create personalized digital twins of two people we replicated for this project. To achieve authentic and dynamic interaction, the avatars are endowed with natural language understanding and open-ended dialogue capabilities via integration with the Convai conversational AI platform. The heart of the system is a custom-engineered hardware setup, featuring a ceiling-mounted projector onto a suspended plexiglass screen with a back-projection film, producing the vivid illusion of a physically present, digital human capable of real-time conversation. In this work, we systematically addressed both engineering and design challenges, such as projection geometry, audio capture, avatar personality profiling, and cross-platform integration. Redeveloping ARCADE is, therefore, a work in progress that aims to enhance perceived social presence and user engagement by combining visual fidelity, individualized avatar identity, and sophisticated conversational AI. Through collecting user feedback and conducting preliminary tests, our goal is to reveal the potential of our system as a scalable, immersive solution for events, education, institutional communication, and future healthcare scenarios. Our results would contribute to shaping next-generation, human-centered interaction spaces as part of the growing role of AI-powered avatars.