Affect recognition in immersive room-scale environments: A large-scale VR study with custom facial sensing at the Science Museum in London.

Abstract

Recent technological advances have provided the chance to conduct Virtual Reality (VR) experiments with increased ecological validity, which in turn can elicit more naturalistic responses in immersed users. However, many studies still prefer highly controlled setups and passive stimulation, often due to the practical complexities in effectively associating cause (stimulus) to response in highly interactive and dynamic VR experiences. Many of these studies also rely on subjective ratings from participants recorded either after the experience (relying on memory) or during the experience (interrupting immersion). In this paper, we advance this experimental protocol in a large-scale feasibility study by (1) investigating affective changes in terms of valence and arousal ratings in various interactive 3D room-scale VR environments with (2) continuous valence and arousal self-ratings from a controller and (3) a novel wireless physiological facial EMG and PPG sensor setup specifically designed to record affect, without relying on memory or interrupting immersion. In this study, n=291 participants experienced neutral, positive and negative virtual environments in 'passive' and 'active' conditions. Continuous self-ratings and physiological measures confirmed the feasibility of detecting affective states in room-scale VR conditions. To our knowledge, this is the highest n in a feasibility study in affect detection to date. Our study generated the most populated physiological data library collected in VR, which also compares passive and active VR settings. This setup can provide a solid experimental foundation for VR affective computing studies in more unconstrained, ecologically valid environments.