MusiXR: Musical experiences in extended reality
Team
- IPEM — Institute for Psychoacoustics and Electronic Music
- IDLab-MEDIA
- XRHIL — Extended Reality for Humanities and Social Sciences
Project Duration
01/10/2020 – 30/9/2026
Researchers
Summary
MusiXR is an interdisciplinary research project exploring how musical performances and concert experiences can be created in extended reality. The project combines advances in virtual acoustics and spatial audio with empirical research into how musicians and listeners perceive immersive environments.
Researchers from IPEM and IDLab-MEDIA developed new methods for rendering dynamic acoustic scenes, studied different immersive reproduction systems and created interactive demonstrators for shared musical XR experiences.
MusiXR was funded by the Special Research Fund of Ghent University through its Interdisciplinary Research Projects programme (BOF-IOP).
Immersive audio rendering
MusiXR developed a real-time virtual-acoustics platform for rendering sound sources, room reflections and reverberation in dynamic XR environments. The system combines Unity, Wwise and physics-based room-acoustic simulation to adapt the sound field as listeners and sound sources move through a virtual space.
Different rendering approaches were explored, including binaural reproduction over headphones, Ambisonics over loudspeaker arrays and hybrid systems combining personal and shared audio reproduction.
Perception studies
A series of perception studies investigated how different immersive audio systems influence spatial perception, realism and the experience of presence.
The studies compared binaural, Ambisonic and hybrid reproduction. This allowed the researchers to examine how individualized headphone rendering, shared loudspeaker reproduction and combinations of both affect the listener’s experience of virtual acoustic spaces.
The results contribute to a better understanding of the technical and perceptual trade-offs involved in designing immersive musical experiences.
Hybrid acoustic environments
One of the project’s main technical developments was a distributed hybrid spatial-audio system. Listener-specific sound components, such as direct sound and early reflections, are rendered through personal devices or headphones. A loudspeaker system simultaneously reproduces a shared late-reverberant sound field in the physical venue.
Audio, tracking information and virtual-scene data are distributed and synchronized across multiple devices. This approach combines an individual spatial perspective for each listener with the social and physical experience of sharing the same acoustic environment.
Research demonstrators
The developed technologies were integrated into experimental and artistic demonstrators. These demonstrators brought together virtual environments, motion tracking, real-time spatial audio and interactive artistic content.
They provided a practical context for exploring how immersive audio can support new forms of musical performance, audience participation and artistic expression.
