Six-Degrees-of-Freedom Audio for VR and AR
Module E: Real-time and interactive worlds. Lesson 9 of 20, 30 min.
Abstract
When listeners can walk through a scene, every spatial cue must be recomputed from their pose in real time. This lesson covers transforms, distance and occlusion models, propagation shortcuts and latency budgets, and shows how to judge results by presence and plausibility rather than physics alone.
By the end you can
- Distinguish 3DoF from 6DoF rendering and explain which cues must be recomputed when the listener translates
- Configure distance attenuation, directivity and direct-to-reverberant balance to produce convincing depth
- Compare occlusion, geometric propagation, diffraction and precomputed wave approaches for real-time use
- Implement a moving listener with the Web Audio PannerNode and AudioListener and evaluate it for place and plausibility illusion
In a fixed-perspective mix the listener is a point that never moves, so every spatial decision can be baked into the render. In virtual and augmented reality the listener walks, crouches, turns and leans, and the sound scene has to be recomputed continuously from their pose. This lesson covers the engineering and the perception behind six-degrees-of-freedom (6DoF) audio: how to move listeners and sources, how to model distance, occlusion and propagation cheaply enough for real time, and how to judge whether the result is convincing.
From 3DoF to 6DoF: what changes
A 3DoF experience tracks only head rotation (yaw, pitch, roll). The listener is pinned to one point and the renderer only rotates the sound field, which is why a head-tracked Ambisonics bed serves 360-degree video well: rotating spherical-harmonic signals is a cheap matrix operation.
A 6DoF experience adds translation. The listener can walk up to a source, around it or behind a wall, so several former constants must be recomputed every frame:
- Direction of every source relative to the head, which now depends on both positions and on head orientation.
- Distance, which drives level, air absorption, direct-to-reverberant ratio and, for very near sources, interaural level differences.
- Visibility, meaning whether geometry blocks the direct path (occlusion) or partly blocks it (obstruction).
- Room response, because early reflections and reverberation change as the listener moves from a corridor into a hall.
- Source directivity, because walking behind a talker or a trumpet should change its timbre.
So 6DoF audio is rarely a pre-rendered file. It is an object-based scene plus a renderer, like the object model behind Dolby Atmos or MPEG-H 3D Audio, except that the renderer runs on the device with the listener's pose as an input. If you are new to object-based formats, review Immersive Audio Formats first; if the cues themselves need revisiting, see How We Hear in 3D.
| Property | 3DoF | 3DoF+ | 6DoF |
|---|---|---|---|
| Tracked motion | Head rotation only | Rotation plus small head translation | Rotation plus free translation |
| Typical content | 360 video, head-tracked Ambisonics | Seated VR with parallax | Room-scale VR, AR, games |
| Distance cues | Baked into mix | Small corrections | Fully dynamic |
| Occlusion | Baked or absent | Rarely modeled | Required for plausibility |
| Renderer location | Can be partly offline | Device | Device, every audio block |
The goal is not physical accuracy for its own sake. As Slater (2009) argues for virtual environments, what matters is place illusion (being there) and plausibility illusion (what is happening is really happening). Audio fails both loudly when a sound refuses to behave as the visible world implies.
The rest of this lesson, its lab and self-check are part of Academy Advanced, a paid course. See who has access.
Key readings
- Slater, M. (2009). Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philosophical Transactions of the Royal Society B, 364(1535), 3549-3557.
- Savioja, L., Huopaniemi, J., Lokki, T., & Väänänen, R. (1999). Creating Interactive Virtual Acoustic Environments. Journal of the Audio Engineering Society, 47(9), 675-705.
- Begault, D. R. (1994). 3-D Sound for Virtual Reality and Multimedia. AP Professional.
- Raghuvanshi, N., & Snyder, J. (2014). Parametric Wave Field Coding for Precomputed Sound Propagation. ACM Transactions on Graphics, 33(4).
- W3C (2021). Web Audio API. W3C Recommendation.