MIT Psychoacoustics Hacks: How MIT Hacked Human Hearing

Spatial9 Team · Oct 14, 2026 · 15 min read

Short answer: At MIT, a "hack" traditionally means a clever, harmless and slightly mischievous feat of engineering. In psychoacoustics, the science of how the brain turns vibrations into experience, MIT researchers have been finding hacks in human hearing for more than seventy years. They showed that we hear notes that are not there, that we can pick one voice out of a crowded room, that a speaker can beam sound like a flashlight, that a crisp packet filmed on video can give away a conversation, and that a 40 Hz hum may one day help brains clean themselves. This guide walks through twelve of those hacks, what each reveals about your ears, and the original papers behind them.

The original MIT hacks were loud, too

MIT's own archive of student pranks, the IHTFP Hack Gallery, defines a hack as "a clever, benign, and ethical prank". Some of the most famous ones were about sound. For the Boston Pops' Fourth of July concert in 1993, hackers turned the top of the Green Building, MIT's tallest building, into what the gallery calls "the world's largest sound (VU) meter", with its lit windows rising and falling with the music. A year later, in May 1994, a replica campus police car appeared on top of the Great Dome. The research hacks below share the same spirit: find a hidden rule, then use it in a way nobody expected.

Why psychoacoustics is full of hacks

Your ears do not record sound like a microphone. The cochlea splits sound into frequency bands, the brainstem compares tiny timing differences between your two ears, and the cortex fills gaps using expectation and memory. Every one of those steps takes a shortcut, and every shortcut can be used. That is what makes psychoacoustics so useful for engineers. If you know which details the brain ignores, you can leave them out, as MP3 and AAC do. If you know which details it relies on, you can fake them, which is how headphones create 3D space.

If you are new to the basics, our primer on how we hear in 3D explains the timing and level cues used throughout this article.

Hack 1: The missing fundamental (Licklider, 1951)

Play the harmonics 200, 300, 400, 500 and 600 Hz together, with nothing at 100 Hz, and almost everyone hears a clear pitch at 100 Hz. The note is missing from the sound, but present in the mind. The effect had been noticed in the 1800s, but in 1951 J. C. R. Licklider, then at MIT and later one of the fathers of the internet, proposed the duplex theory of pitch. His idea was that the auditory system runs something like an autocorrelation: it looks for the repeating period shared by all the harmonics (here, every 10 milliseconds) and reports that as the pitch.

Why it matters today: phones, laptops and earbuds cannot physically reproduce deep bass, yet music on them still sounds like it has a bass line. Many audio processors deliberately add harmonics so your brain rebuilds the low note. Timing-based pitch models descended from Licklider's idea are still used in hearing science.

Hack 2: The cocktail party problem (Cherry, 1953)

How do you follow one conversation in a noisy room? In 1953 Colin Cherry, working with MIT's Research Laboratory of Electronics, published experiments in which listeners heard two different messages, one in each ear, and had to repeat one aloud. They could follow the chosen message well but remembered almost nothing about the other, sometimes not even noticing it had switched language. They did notice if the voice changed from male to female.

The big lesson is that space is a filter. When voices come from different directions, the brain separates them far more easily than when they come from the same speaker. Researchers call this spatial release from masking. It is one of the strongest arguments for spatial audio: separating instruments or voices in space makes each one clearer without turning anything up. We explore this in binaural vs spatial audio.

Hack 3: Reflected sound and anti-sound (Amar Bose)

Amar Bose was an MIT professor for over four decades. After being disappointed by a hi-fi system, he studied concert halls and concluded that most of what we hear at a live concert arrives as reflections from walls and ceilings, not directly from the stage. His 1968 Bose 901 speaker put eight of its nine drivers facing backwards, bouncing sound off the wall behind it to imitate that reflected field.

Bose later pursued another psychoacoustic hack: cancelling noise by playing its mirror image. A microphone hears incoming noise and the headphone plays an opposite waveform so the two largely cancel at your ear. That work led to noise-cancelling aviation headsets and eventually to the noise-cancelling headphones many people use every day.

Hack 4: Software that makes music (Vercoe and Csound, 1986)

Barry Vercoe, a founding member of the MIT Media Lab, wrote Csound in 1986, a language for building sound from scratch with code. It became one of the most widely used tools in computer music and psychoacoustic research, because researchers could generate perfectly controlled sounds for listening experiments. Vercoe's group also explored synthetic performers that could follow a live musician, and early ideas behind structured, object-based audio that resemble the way immersive formats describe sounds today. Csound is still maintained as open source.

Hack 5: KEMAR, the dummy head that taught headphones to hear in 3D (1994)

In 1994 Bill Gardner and Keith Martin at the MIT Media Lab placed tiny microphones inside the ears of KEMAR, a mannequin designed to have average adult proportions. They played sounds from 710 directions around it at a distance of 1.4 metres and recorded exactly how the head, shoulders and folds of the outer ear changed each sound. These measurements are called head-related transfer functions, or HRTFs.

They released the data for free. For years it was one of the most used HRTF sets in the world, built into research tools, games and virtual reality engines.

The hack: if you filter a sound through the HRTF for a given direction and play it over headphones, your brain hears it coming from that direction. That is the core of binaural rendering, and of the headphone spatial audio used by Spatial9. Generic heads work well for most people, though front and back confusions are more common than with personalised measurements.

Hack 6: The Audio Spotlight (Pompei, 1999)

Ordinary speakers spread sound around a room. F. Joseph Pompei, then at the MIT Media Lab, built a speaker that sends a narrow beam of sound, so that only someone standing in the beam hears it. It works by sending ultrasound, which is far too high to hear. As the ultrasound passes through air, the air slightly distorts it, and that distortion produces audible sound along the beam.

Directional speakers based on this idea are now used in museums, shops and exhibitions, where each display can talk to one visitor without filling the whole room.

Hack 7: Sound textures are statistics (McDermott and Simoncelli, 2011)

Rain, fire, applause and wind are made of thousands of tiny events. Josh McDermott, now head of MIT's Laboratory for Computational Audition, and Eero Simoncelli showed that the brain does not track each droplet. Instead it summarises the sound with a handful of statistics: how loud each frequency band is on average, how much it fluctuates, and how bands move together.

The hack is that you can start from random noise, adjust it until its statistics match those of real rain, and listeners hear convincing rain. A related 2013 study found that for longer sounds, listeners remember the statistics better than the actual details. Later work by James Traer and McDermott in 2016 showed the same idea for reverb: the brain recognises a room by the statistical shape of its echoes, and real-world reverb follows predictable patterns. That is why carefully designed artificial rooms can feel natural.

Hack 8: Music has its own part of the brain (2015)

Using brain scans and a new analysis method, Sam Norman-Haignere, Nancy Kanwisher and Josh McDermott found groups of neurons in the human auditory cortex that respond strongly to music, and much less to speech or other sounds. In a related 2016 study with the Tsimane people of Bolivia, who had little exposure to Western music, McDermott and colleagues found that the preference for consonant over dissonant chords was not universal. That suggests some of what feels "pleasant" in music is learned through culture rather than built into the ear.

Hack 9: The visual microphone (Davis and colleagues, 2014)

This is perhaps MIT's most famous hearing hack. Abe Davis, Michael Rubinstein and colleagues at MIT CSAIL filmed objects such as a crisp packet, a houseplant leaf and a glass of water with a high-speed camera while sound played nearby. The objects vibrated by fractions of a pixel. By analysing these tiny movements, the team recovered understandable speech and music from silent video, in one case through soundproof glass.

The research also revealed a privacy lesson: sound leaves traces in the physical world. Later CSAIL work went the other way, training systems to predict the sound an object makes when hit, simply by watching video.

Hack 10: The Sound of Pixels (Zhao and colleagues, 2018)

Hang Zhao, Antonio Torralba and colleagues at CSAIL trained a neural network by watching thousands of videos of people playing instruments. Without anyone labelling the sounds, it learned which pixels in the video were making which sounds. Click on the violin in a video and the system isolates the violin. This kind of audio-visual source separation is a close relative of the stem separation tools covered in our guide to turning stems into immersive audio.

Hack 11: An AI that hears with ears (Francl and McDermott, 2022)

Andrew Francl and Josh McDermott trained a deep neural network to localise sounds in a simulated world, giving it virtual ears with realistic HRTFs and virtual rooms with echoes. The network was never told how humans hear. Yet once trained, it made the same kinds of mistakes humans do, used the same cues, and showed the same precedence effect, where the first arriving sound dominates our sense of direction. When trained in unnatural worlds with no echoes or no background noise, it behaved less like a human. The conclusion: human hearing is shaped by the acoustic world we grew up in.

Hack 12: 40 Hz gamma stimulation (Tsai Lab, 2016 to 2024)

Brain cells often fire in rhythms. One rhythm, gamma at around 40 cycles per second, is weaker in Alzheimer's disease. Li-Huei Tsai's lab at the MIT Picower Institute found that exposing mice to lights flickering at 40 Hz reduced amyloid build-up. In 2019 they showed that a 40 Hz sound, alone or combined with light, improved memory in mice and reduced amyloid and tau in several brain regions. They called the approach GENUS, short for gamma entrainment using sensory stimuli. In 2024 a study in Nature suggested a mechanism: 40 Hz stimulation increased the flow of fluid that washes waste out of the mouse brain, a process known as the glymphatic system.

For a wider look at what the evidence does and does not support about sound and health, read our report on spatial audio, binaural beats and health.

The map: which hacks power the sound you hear today

How Spatial9 uses these ideas

Spatial9 turns ordinary stereo tracks into immersive mixes. Three of the hacks above do much of the work. HRTF-based rendering, the idea KEMAR made accessible, places each element around your head on ordinary headphones. Spatial separation, the lesson of the cocktail party problem, lets vocals, drums and instruments each have their own space so they are clearer without being louder. And natural-sounding rooms, as the reverb statistics research suggests, keep the result from sounding artificial. You can hear the difference on the Spatial9 demo, and teachers can use these experiments in class through Spatial9 for Education.

Try three hacks yourself

1. Missing fundamental: listen to a song with strong bass on your phone speaker, then on good headphones. The bass line is recognisable in both, even though the phone barely plays it.
2. Cocktail party: in a busy café, close one ear. Following one voice becomes noticeably harder, because you have lost the timing difference between your ears.
3. Spatial separation: compare a stereo track with a spatial version on headphones. Notice how individual instruments become easier to follow.

Frequently asked questions

What is psychoacoustics?

Psychoacoustics is the study of how people perceive sound. It links the physics of sound waves to what the brain actually experiences, including pitch, loudness, direction and timbre.

What does "hack" mean at MIT?

At MIT a hack is a clever, harmless feat of ingenuity, famously including elaborate pranks like placing a police car on the Great Dome. In this article it means a research finding that exploits a shortcut in human hearing.

What is the missing fundamental?

It is the effect where you hear a low note even though the sound contains only its higher harmonics. The brain infers the note from the repeating pattern the harmonics share. Licklider proposed a timing-based explanation at MIT in 1951.

What is the KEMAR HRTF dataset?

It is a set of measurements made at the MIT Media Lab in 1994, recording how a standard dummy head changes sound arriving from 710 directions. It became a widely used free resource for creating 3D sound on headphones.

How does the MIT visual microphone work?

A high-speed camera films an object such as a crisp packet while sound plays nearby. Software measures vibrations smaller than a pixel and converts them back into audio, recovering speech and music from silent video.

Does 40 Hz sound treat Alzheimer's disease?

Not yet proven. MIT studies in mice found that 40 Hz light and sound reduced Alzheimer's-related proteins and improved memory, and small human trials suggest it is safe. Larger clinical trials are still needed before it can be considered a treatment.

Who is Josh McDermott?

Josh McDermott is an MIT professor in Brain and Cognitive Sciences who leads the Laboratory for Computational Audition. His group studies sound textures, music perception, reverb and AI models of hearing.

Sources

1. Licklider, J. C. R. (1951). A duplex theory of pitch perception. Experientia, 7, 128-134.
2. Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears. Journal of the Acoustical Society of America, 25(5), 975-979.
3. Gardner, B. and Martin, K. (1994). HRTF measurements of a KEMAR dummy-head microphone. MIT Media Lab.
4. Pompei, F. J. (1999). The use of airborne ultrasonics for generating audible sound beams. Journal of the Audio Engineering Society, 47(9), 726-731.
5. Csound, the sound and music computing system originally developed by Barry Vercoe at MIT.
6. McDermott, J. H. and Simoncelli, E. P. (2011). Sound texture perception via statistics of the auditory periphery. Neuron, 71(5), 926-940.
7. McDermott, J. H., Schemitsch, M. and Simoncelli, E. P. (2013). Summary statistics in auditory perception. Nature Neuroscience, 16, 493-498.
8. Traer, J. and McDermott, J. H. (2016). Statistics of natural reverberation enable perceptual separation of sound and space. PNAS, 113(48), E7856-E7865.
9. Norman-Haignere, S., Kanwisher, N. and McDermott, J. H. (2015). Distinct cortical pathways for music and speech revealed by hypothesis-free voxel decomposition. Neuron, 88(6), 1281-1296.
10. McDermott, J. H., Schultz, A. F., Undurraga, E. A. and Godoy, R. A. (2016). Indifference to dissonance in native Amazonians reveals cultural variation in music perception. Nature, 535, 547-550.
11. Davis, A., Rubinstein, M., Wadhwa, N., Mysore, G., Durand, F. and Freeman, W. T. (2014). The visual microphone: passive recovery of sound from video. ACM Transactions on Graphics (SIGGRAPH), 33(4).
12. Zhao, H., Gan, C., Rouditchenko, A., Vondrick, C., McDermott, J. and Torralba, A. (2018). The sound of pixels. ECCV 2018.
13. Francl, A. and McDermott, J. H. (2022). Deep neural network models of sound localization reveal how perception is adapted to real-world environments. Nature Human Behaviour, 6, 111-133.
14. Iaccarino, H. F. et al. (2016). Gamma frequency entrainment attenuates amyloid load and modifies microglia. Nature, 540, 230-235.
15. Martorell, A. J. et al. (2019). Multi-sensory gamma stimulation ameliorates Alzheimer's-associated pathology and improves cognition. Cell, 177(2), 256-271.
16. Chan, D. et al. (2022). Gamma frequency sensory stimulation in mild probable Alzheimer's dementia patients: results of feasibility and pilot studies. PLOS ONE, 17(12), e0278412.
17. Murdock, M. H. et al. (2024). Multisensory gamma stimulation promotes glymphatic clearance of amyloid. Nature, 627, 149-156.
18. MIT Laboratory for Computational Audition.
19. IHTFP Hack Gallery: Best of the Gallery, including the 1993 Green Building VU meter.

Try Spatial9 free