Academy Advanced

Paid graduate spatial audio course: sonification, audio AI, psychoacoustics, Hugging Face, Databricks, computer vision, HRTFs, XR, IAMF and research. 20 lessons in 10 modules, each with a lab, a self-check and key readings. About 11 hours, 100 academic readings. Last updated Oct 31, 2026.

Module A: Sonification in 3D

Turn data into sound you can read, then use space itself as a data channel.

1. Lesson 1: Sonification Principles: Turning Data into Sound (30 min). A graduate guide to sonification: audification, parameter mapping, model-based methods, earcons, polarity, scaling, psychoacoustic limits and evaluation.
2. Lesson 2: Space as a Data Channel: Spatial Sonification Design (30 min). How to use azimuth, elevation and distance as data channels: localisation acuity, scene analysis, release from masking and object-based rendering of data streams.

Module B: Training your own audio AI

Audio tokens and neural codecs, then fine-tuning a language model that writes spatial scenes.

1. Lesson 3: Audio Tokens and Neural Codecs (30 min). How audio becomes tokens: VQ-VAE, residual vector quantization, SoundStream, EnCodec and DAC, semantic vs acoustic tokens, and why spatial audio is hard to tokenize.
2. Lesson 4: Fine-Tuning a Language Model for Spatial Scenes (30 min). Build a stems-to-scene model: dataset design with datasheets and consent, LoRA and QLoRA fine-tuning, DPO, constrained JSON decoding, and honest spatial evaluation.

Module C: Computer vision for spatial audio

Find what is sounding in a frame, then track it in depth so the sound moves with the picture.

1. Lesson 5: Audio-Visual Source Localization (45 min). How self-supervised audio-visual models find which pixels make a sound, from Look Listen and Learn to ImageBind, and where heatmaps fail for spatial mixing.
2. Lesson 6: Depth, Tracking and Moving Sound from Video (30 min). Turn video into moving audio objects: monocular depth, YOLO and SAM 2, SORT and ByteTrack tracking, Kalman smoothing and camera-to-listener geometry for spatial mixes.

Module D: Advanced acoustics and rendering

Personalized HRTFs, room simulation and higher-order Ambisonics.

1. Lesson 7: Personalized HRTFs: Measurement, Modelling and ML (30 min). Learn how HRTFs are measured, simulated, stored in SOFA and personalized with machine learning, and how to evaluate binaural rendering quality.
2. Lesson 8: Room Simulation and Higher-Order Ambisonics (30 min). Model rooms with image sources, ray tracing and feedback delay networks, then encode, decode and binauralize scenes with Higher-Order Ambisonics.

Module E: Real-time and interactive worlds

Six-degrees-of-freedom audio for VR and AR, and music that adapts to play and place.

1. Lesson 9: Six-Degrees-of-Freedom Audio for VR and AR (30 min). How to build convincing 6DoF audio for VR and AR: listener transforms, distance models, occlusion, diffraction, latency, Web Audio HRTF panning and room matching.
2. Lesson 10: Adaptive and Generative Music for Games and Installations (30 min). Design adaptive scores with resequencing, layering, stingers and state machines, then place music stems in space for games, XR and VBAP, DBAP or Ambisonics domes.

Module F: Applied frontiers

Spatial audio for accessibility and hearing, wildlife monitoring and lost historic spaces.

1. Lesson 11: Spatial Audio for Accessibility and Navigation (30 min). How spatial audio supports blind and low-vision navigation, accessible sonification, hearing aids, safe listening and group calls, with a Python beacon example.
2. Lesson 12: Bioacoustics and Acoustic Heritage (30 min). Passive acoustic monitoring, soundscape ecology, ACI and NDSI, BirdNET, wildlife arrays and Ambisonics, plus auralization of historic spaces such as Stonehenge.

Module G: Standards, agents and QC

Inside IAMF and ADM, and AI agents that check a deliverable before a human signs it off.

1. Lesson 13: Inside IAMF and ADM: Metadata, Profiles and Rendering (30 min). A deep dive into ADM (BS.2076), BW64 axml/chna chunks, Serial ADM, the BS.2127 renderer and IAMF OBUs, profiles and loudness, with a Python axml parser.
2. Lesson 14: Audio Agents and Automated Quality Control (35 min). How tool-using LLM agents (ReAct, MCP) orchestrate automated audio QC: BS.1770 loudness, true peak, LRA, phase, PEAQ, ViSQOL, FAD and human sign-off.

Module H: Advanced psychoacoustics and AI

Masking, loudness and perceptual losses, deep networks as models of hearing, and the illusions and MIT hacks that reveal how we listen.

1. Lesson 15: Psychoacoustic Models Meet Deep Learning (40 min). Critical bands, masking and loudness models, perceptual metrics and losses for neural audio, and MIT research that uses deep networks as models of human hearing.
2. Lesson 16: Auditory Illusions, Perceptual Hacks and MIT Projects (30 min). What classic auditory illusions reveal, how spatial audio exploits them, where AI listeners disagree with people, and how MIT labs and hackathons build perceptual hacks.

Module I: Build and ship your own audio model

Train and serve a model on Databricks with a production architecture, and explore and fine-tune audio models on Hugging Face.

1. Lesson 17: Training an Audio Model on Databricks (35 min). Build a governed audio pipeline on Databricks, train and register a stem-role classifier with MLflow, serve it, and design the architecture that features it in an app.
2. Lesson 18: Exploring and Fine-Tuning Audio Models on Hugging Face (30 min). Navigate the Hugging Face Hub, read model cards critically, run and fine-tune audio models, publish with a good card and Space, and bring models into Databricks.

Module J: Research methods and capstone

Design listening tests that hold up to review, then take a project from question to publication.

1. Lesson 19: Designing Listening Tests: MUSHRA, ABX and Statistics (30 min). Design defensible listening tests for spatial audio with BS.1116, MUSHRA, ABX and ACR, then analyse ratings with Friedman, Holm-corrected Wilcoxon and mixed models.
2. Lesson 20: Capstone: From Research Question to Publishable Project (30 min). Take an immersive audio project from research question to publishable paper with literature reviews, reproducible artifacts, ethics, milestones and a grading rubric.

New to immersive audio? Start with the free core course or compare both tracks.