The Story time Dataset: Simulated Videotelephony Clips for Quality Perception Research
Published in 14th International Conference on Quality of Multimedia Experience (QoMEX 2022), 2022
Spang, R. P., Voigt-Antons, J.-N. & Möller, S.
To study people's natural behavior during different conditions of audiovisual quality, we usually invite people into a lab and let them talk to each other. In such conversation settings, not only the media quality impacts the quality perception, but, e.g., social aspects of a real conversation are reflected by individual conversational and rating behavior. Hence, to study quality perception in conversational settings, we try to create an environment that isolates the media quality from such outside factors and is consistent for each participant in the lab. Therefore, we created a dataset of simulated videotelephony clips to act as stimuli in quality perception research. The dataset consists of four different stories in the German language that are told through ten consecutive parts, each about 10 seconds long. Each of these parts is available in four different quality levels, ranging from perfect to stalling. All clips (FullHD, H.264 / AAC) are actual recordings from end-user video-conference software to ensure ecological validity and realism of quality degradation. To ensure consistency among different clips of the same quality level, each video has been scored using VMAF and POLQA and selected to match predefined selection criteria. To analyze the perceived quality of the clips, we conducted a user study (N=25) and evaluated perceived quality, interest in the stories, and speaker engagement. Results validate the consistency of the quality levels of the video clips. Apart from a detailed description of the methodological approach, we contribute the entire stimuli dataset containing 160 videos and all rating scores for each file.
Recommended citation: Spang, R. P., Voigt-Antons, J.-N. & Möller, S. (2022, September). The Storytime Dataset: Simulated Videotelephony Clips for Quality Perception Research. Paper presented at the 14th International Conference on Quality of Multimedia Experience (QoMEX 2022), Lippstadt, Germany. https://doi.org/10.1109/QoMEX55416.2022.9900888