Auditory Illusions, Perceptual Hacks and MIT Projects
Module H: Advanced psychoacoustics and AI. Lesson 16 of 20, 30 min.
Abstract
Auditory illusions isolate the shortcuts the brain takes when it turns two eardrum signals into a scene. This lesson analyses Shepard tones, the octave and scale illusions, precedence, continuity and audio-visual effects, and shows how psychoacoustic bass, virtual height and decorrelation turn them into engineering tools. It then contrasts human illusions with adversarial examples and model metamers, and surveys how MIT research groups and hackathons prototype perceptual hacks.
By the end you can
- Explain the perceptual assumption behind Shepard tones, the octave illusion, the precedence and Franssen effects, and the continuity illusion
- Apply psychoacoustic bass, spectral elevation cues and decorrelation as spatial audio techniques, and state their limits
- Contrast human illusions with adversarial examples and model metamers as tests of machine listeners
- Build and test a Shepard tone and a precedence-effect demo, and write a perceptual hack brief
At MIT, a "hack" is a clever, harmless feat of engineering that reveals how a system really works. Auditory illusions are hacks of exactly that kind: carefully built sounds that make the auditory system report something that is not physically there. For researchers they are diagnostic tools, because each illusion exposes a shortcut the brain takes. For spatial audio designers they are raw material, because the same shortcuts let headphones create height, small speakers create bass and two loudspeakers create a voice in the middle of the room. This lesson moves from the classic illusions to the engineering hacks built on them, then to the places where artificial listeners and human listeners disagree, and finally to the way MIT labs and hackathons turn perceptual knowledge into working prototypes.
Illusions as a window into the auditory system
An illusion is useful when it isolates a single mechanism. The auditory system has to solve an under-determined problem: from two pressure signals at the eardrums, it must infer how many sources exist, what they are and where they are. It does so with assumptions learned from natural environments, a view developed in detail by Albert Bregman in Auditory Scene Analysis (1990) and supported by the deep-network results discussed in Psychoacoustic Models Meet Deep Learning. Illusions appear when a sound satisfies those assumptions in an unnatural way.
It helps to sort illusions by the assumption they exploit:
| Assumption | Illusion that exploits it | What it reveals |
|---|---|---|
| Harmonics with a common period belong to one source | Missing fundamental, Shepard tones | Pitch is computed from periodicity and spectral pattern, not from energy at the fundamental |
| The first wavefront comes from the source | Precedence (Haas) effect, Franssen effect | Localization is dominated by onsets; later energy is suppressed or reinterpreted |
| Sounds that are interrupted by noise continue behind it | Continuity illusion, phonemic restoration | The brain fills gaps when the evidence permits |
| Identical signals at both ears come from the midline | Phantom centre, summing localization | Direction is inferred from interaural differences, not from loudspeaker positions |
| Vision and hearing describe the same event | McGurk effect, ventriloquism | Perception is multisensory and weighted by reliability |
The site's feature article MIT Psychoacoustics Hacks tells the historical story of several of these discoveries, from Licklider's pitch theory to Cherry's cocktail party experiments. This lesson goes deeper into the mechanisms and into how you can build and test hacks of your own.
The rest of this lesson, its lab and self-check are part of Academy Advanced, a paid course. See who has access.
Key readings
- Bregman, A. S. (1990). Auditory Scene Analysis: The Perceptual Organization of Sound. MIT Press.
- Deutsch, D. (1974). An auditory illusion. Nature, 251, 307-309.
- Litovsky, R. Y., Colburn, H. S., Yost, W. A., and Guzman, S. J. (1999). The precedence effect. Journal of the Acoustical Society of America, 106(4), 1633-1654.
- Carlini, N., and Wagner, D. (2018). Audio adversarial examples: Targeted attacks on speech-to-text. IEEE Security and Privacy Workshops.
- Ananthabhotla, I., Ramsay, D. B., Duhart, C., and Paradiso, J. A. (2021). Cognitive audio interfaces: Mediating sonic information with an understanding of how we hear. IEEE Pervasive Computing, 20(2), 36-45.