“Did you hear that?”: Software-based spatial audio enhancements increase self-reported and physiological indices on auditory presence and affect in virtual reality.

Abstract

Introduction: This study investigates the impact of a software-based audio enhancement tool Q6 in virtual reality (VR), examining the relationship between spatial audio, immersion, and affective responses using self-reports and physiological measures. Methods: Sixty-eight participants experienced two VR scenarios, i.e., a commercial game (Job Simulator) and a non-commercial simulation (Escape VR), under both enhanced and normal audio conditions. In this paper we propose a dual-method assessment approach, combining self-reports with moment-bymoment physiological data analysis, emphasizing the value of continuous physiological tracking for detecting subtle changes in electrophysiology in VR simulated experiences. Results: Results show that enhanced ‘localised’ audio sounds significantly improved perceived sound quality, immersion, sound localization, and emotional involvement. Notably, commercial VR content exhibited a stronger response to audio enhancements than non-commercial simulations,likely due to sound architecture. The commercial content featured meticulously crafted sound design, while the non-commercial simulation had numerous sounds less spatially structured, resulting in a less coherent auditory experience. Enhanced audio additionally intensified both positive and negative affective experiences during key audiovisual events. Discussion: In this paper we propose a dual-method assessment approach, combining self-reports with moment-bymoment physiological data analysis, emphasizing the value of continuous physiological tracking for detecting subtle changes in electrophysiology in VR simulated experiences. Our findings support software-based audio enhancement as a cost-effective method to optimize auditory involvement in VR without additional hardware. This research provides valuable insights for designers and researchers aiming to improve audiovisual experiences and highlights future directions for exploring adaptive audio technologies in immersive environments.