Effect of Questionnaire Order on Ratings of Perceived Quality and Experienced Affect

Abstract

The assessment of perceived quality and experienced affect of users towards different audio and speech quality levels using subjective rating methods is a common and valid approach. In case that researchers are interested in the subjective a) quality and b) affective rating, participants have to rate the stimuli on both scales. Even though subject should just rate only one of the scales to be not influenced by the questions asked in the other questionnaire, researchers in practice frequently present various scales in a sequence to subjects. This paper examines the effect of questionnaire order on perceived quality and experienced affect by comparing the ratings of two groups of subjects which filled in questionnaires in different orders after listening to the same set of stimuli. We showed that the order of sequence had a significant effect on perceived quality and experienced affect as e.g. the quality of stimuli was rated lower if the quality rating not occurred instantly after stimulus presentation. Thus we want to create awareness for these effects and presented first guidelines to overcome them.