Multimodal Interactive Assistance for inclusive collection of patient-reported outcome measures: A pragmatic cross-over randomised controlled trial

Abstract

Objectives: Patient-reported outcome measures (PROMs) support patient-centred care and quality improvement, yet completing these questionnaires can be burdensome. We developed a multimodal assistance system using artificial intelligence methods for speech processing to enable interactive questionnaire completion. The system was implemented across three embodiments (robot, virtual agent, and voice-only), each paired with a questionnaire tablet and iteratively refined through participatory co-design. Methods: In a pragmatic randomised cross-over trial at a German outpatient rehabilitation clinic, 90 participants with psychosomatic or neurological conditions completed a 82-item PROM questionnaire split into two 41-item halves, once with and once without AI speech assistance. Order of questionnaire halves and assistant embodiment were randomised. The primary outcome was subjective mental effort (Rating Scale Mental Effort, RSME); secondary outcomes included engagement, disclosure, user experience, and ratings of assistance. We analysed data with linear mixed models and repeated-measures ANOVAs. Results: Assisted PROM completion was associated with higher subjective effort compared with unassisted completion, and this effect varied by general health. Between-person differences explained most of the variance (ICC = 0.75). Effort was positively associated with psychosomatic indication, poorer general health and concentration barriers, and negatively with openness to technology. Assistant embodiment showed no significant effect in this sample. System evaluations were heterogeneous: about half reported improved concentration and motivation and one-third intention to use the system. Discussion: Multimodal AI-assisted PROM collection did not reduce mean subjective effort in this rehabilitation sample, but its effects varied between users. The heterogenous evaluations suggest that flexible, optional, and equity-oriented implementation may be needed to address heterogeneous user needs.

BibTeX

@article{voigtantons2026j40,
  author    = {Marquardt, M. and Graf, P. and Hillmann, S. and Voigt-Antons, J.-N. and Harnisch, P. and Ashrafi, N. and Compagna, D. and Zoellick, J. C.},
  title     = {Multimodal Interactive Assistance for inclusive collection of patient-reported outcome measures: A pragmatic cross-over randomised controlled trial},
  year      = {2026},
  journal   = {DIGITAL HEALTH},
}