NASA Sonification and MIT Sound Hacks: The Science Behind the Effects

Part 1: Sonification

What sonification means

Sonification is the use of non-speech sound to carry information. The field took shape at the first International Conference on Auditory Display (ICAD) in 1992, organised by Gregory Kramer. His edited book *Auditory Display: Sonification, Audification, and Auditory Interfaces* (1994) is still the starting point. Later, Thomas Hermann, Andy Hunt and John G. Neuhoff edited *The Sonification Handbook* (2011), which is free to read online and covers almost everything in this chapter in more depth.

There are four common ways to turn data into sound:

Technique	How it works	Example
Audification	Play a wave-like signal directly, usually sped up or slowed down	Spacecraft radio waves played as audio
Parameter mapping	Map numbers to sound properties such as pitch, loudness, timing or position	Brighter stars play louder notes
Earcons and auditory icons	Short designed sounds that stand for events or objects	An alert tone when a value crosses a limit
Model-based sonification	Data becomes a virtual object you "play" or excite	Hermann's interactive data models

Parameter mapping is the most common, and also the easiest to get wrong. Bruce Walker and Michael Nees, in their chapter on the theory of sonification in *The Sonification Handbook*, show that listeners have expectations about polarity (should "more" sound higher or lower?) and scaling (how big a jump in pitch should a doubling of the data be?). A good mapping respects those expectations, so the listener does not have to fight their own ears.

How NASA turns the universe into sound

The Chandra "A Universe of Sound" project. Since 2020, NASA's Chandra X-ray Center has turned images from Chandra, Hubble, Webb and other telescopes into sound. The project is led by visualisation scientist Dr. Kimberly Arcand at the Chandra X-ray Center, with astrophysicist and musician Dr. Matt Russo and musician and sound engineer Andrew Santaguida of SYSTEM Sounds. Christine Malec, an accessibility consultant and member of the blind community, also joined the project. Most of their pieces scan an image like a playhead. Position sets the timing and pitch, brightness sets the loudness, and each telescope or wavelength gets its own instrument. Their sonifications include the Galactic Center, Cassiopeia A, the Bullet Cluster, Eta Carinae, the Crab Nebula and many more.

The Perseus black hole. In 2003, Chandra data showed ripples in the hot gas around the black hole at the centre of the Perseus galaxy cluster: pressure waves, which are real sound waves, about 57 octaves below middle C. In 2022, NASA released a sonification that pulled those waves out of the data and played them back 57 and 58 octaves higher than their true pitch. That is roughly 144 quadrillion and 288 quadrillion times their original frequency. This is audification of a real acoustic signal, not a made-up mapping.

Did people actually learn from it? In 2024, Arcand and colleagues published "A Universe of Sound: processing NASA data into sonifications to explore participant response" in *Frontiers in Communication*. They surveyed 3,184 sighted, blind and low-vision participants and found strong self-reported learning gains and positive responses. This is why sonification matters well beyond music: it makes science reachable for people who cannot see the images.

Radio waves from the planets. Space is mostly empty, so ordinary sound cannot travel through it. Spacecraft instead record plasma waves and radio emissions, which have frequencies in or near the audio range. In 1979, Voyager 1's plasma wave instrument detected "whistlers" at Jupiter: radio bursts from lightning that fall in pitch as they travel through charged gas. In 2018, the Juno mission's Waves instrument, built by the University of Iowa, reported more than 1,600 lightning whistlers. Ivana Kolmašová and colleagues described them in *Nature Astronomy*, almost ten times the number Voyager 1 recorded. Juno's Waves instrument also captured a sudden change in radio emissions during its 2021 flyby of Jupiter's moon Ganymede, which NASA released as audio.

Real sound on Mars. Mars has an atmosphere, so it has real sound. The Perseverance rover landed in February 2021 with microphones, including one on its SuperCam instrument. In 2022, Sylvestre Maurice and colleagues published "In situ recording of Mars soundscape" in *Nature*. Mars turned out to be extremely quiet, and sound travels at two speeds: about 240 metres per second for low pitches and about 250 metres per second for higher ones. On Earth it is about 343 metres per second. The same microphone heard the Ingenuity helicopter as a low hum at about 84 Hz.

The NASA Sonification collection in the classroom

The classroom collection holds 45 effects inspired by these projects. They process your own stems so that a pad or a transition takes on the character of the original sonification. They do not contain NASA's data. Each effect lists its mission and year under its name.

Effect	Source it recreates	Good on
"Perseus Black Hole"	Chandra, 2022	Sub pads, long transitions
"Bullet Cluster Dark Matter"	Chandra, Hubble, Magellan	Wide ambient beds
"Eta Carinae Outburst"	Chandra and Hubble	Risers and impacts
"Centaurus A Jet"	Chandra	Sweeping leads
"Jupiter Lightning Whistlers"	Voyager 1, 1979	Falling sweeps, ear candy
"Juno Ganymede Flyby"	Juno Waves instrument, 2021	Texture changes between sections
"Voyager Interstellar Hum"	Voyager 1, 2021	Drones
"Mars Wind" and "Mars Two Speeds of Sound"	Perseverance, 2021 and 2022	Air, breath, sparse intros
"Ingenuity Helicopter Hum"	Perseverance and Ingenuity, 2021	Low rhythmic texture
"Black Hole Remix"	Chandra Universe of Sound, 2022	Big drops

Spatial sonification

Most NASA sonifications are released in stereo. A 7.1.4 room adds the step NASA's images already imply: position in the sky can become direction around the listener, and height in the image can become the height speakers. That is the idea behind Spatial9's spatial sonification work, covered in our article Space Sonification and Spatial Audio: NASA, NAMM and Spatial9. In the classroom, you can do a simple version yourself: put a NASA effect on a stem, then place or move that stem in the Sphere so the sound comes from where the object would sit.

Part 2: Psychoacoustic sound hacks

Why psychoacoustics feels like hacking

Psychoacoustics studies the gap between the sound in the air and the sound you perceive. Your brain uses shortcuts to work out where a sound is, how far away it is and what made it. Each shortcut can be triggered on purpose. At MIT, a "hack" is a clever, harmless trick, and these discoveries fit that spirit: a 12 millisecond delay that makes a sound wider, two filters that make it rise, a tone that seems to climb forever. For the wider MIT story, from the missing fundamental to the visual microphone, read MIT Psychoacoustics Hacks: How MIT Hacked Human Hearing.

The MIT contributions

KEMAR HRTF measurements (Bill Gardner and Keith Martin, MIT Media Lab, 1994). A head-related transfer function (HRTF) describes how your head and outer ears filter sound arriving from each direction. In May 1994, Gardner and Martin measured a KEMAR dummy head at 710 positions, from 40 degrees below to directly overhead, with a loudspeaker 1.4 metres away. They published the data freely as MIT Media Lab Perceptual Computing Technical Report 280. It became one of the most used HRTF sets in the world, and it is the basis of the classroom's "Dummy Head Dome" effects, which render a sound through head-related filters at ten speaker positions.

Csound (Barry Vercoe, MIT, 1986). Vercoe's Csound became one of the most important languages for computer music, and it is still maintained today. Its classic unit generators include ring modulation, which multiplies two signals to create metallic, bell-like tones. That is the idea behind "Metal Mouth 180Hz Square Heavy".

Hyperinstruments (Tod Machover, MIT Media Lab, from 1986). Machover's hyperinstruments measure a performer's gestures and use them to extend the instrument, most famously in the hypercello piece *Begin Again Again* for Yo-Yo Ma (1991). "Hyper Gesture" turns that idea into a tempo-synced filter movement that sounds like a performer is shaping each note.

MIT Press books. Three essential books came out through MIT Press. Curtis Roads' *Microsound* (2001) is the reference for granular synthesis and inspires "Particle Storm 50ms x35" in Time Benders. Albert Bregman's *Auditory Scene Analysis* (1990) explains how the brain separates a mix into streams, which is why clean placement works. Jens Blauert's *Spatial Hearing* (revised edition, 1997) is the standard text on localisation.

The rest of the research map

Not every hack came from MIT, and getting credit right matters.

Effect	Idea	Paper	Where
"Phantom Width 12ms"	Precedence effect: the first arrival wins	Haas, 1951	Göttingen
"Fly-By 1/4 Wide Far"	Panning, distance and Doppler together	Chowning, "The Simulation of Moving Sound Sources", 1971	Stanford
"Gravity Well Overhead"	Frequency bands the brain reads as height	Blauert, 1969	Germany
"Infinite Staircase Ascending"	A pitch that rises forever	Shepard, 1964; Risset, 1969	Bell Labs
"Ghost String C3 Glass Bell"	Plucked string from a delay loop	Karplus and Strong, 1983	Stanford
"Robot Throat "Ah"", "Ghost Choir "Ee""	Vowel formants	Peterson and Barney, 1952	Bell Labs
"Hall of Mirrors"	First digital artificial reverb	Schroeder, 1962	Bell Labs
"Velvet Room"	Better reverb with early reflections	Moorer, "About This Reverberation Business", 1979	Computer Music Journal
"Liquid Chrome Plate"	Plate reverb design	Dattorro, "Effect Design", 1997	Journal of the Audio Engineering Society
"Cochlea Laser", "Cochlea Push"	Critical bands of the inner ear	Zwicker, Bark scale, 1961	Germany
"Midnight Loudness 40 phon"	Equal-loudness contours	ISO 226	International standard
"Time Stop"	Extreme time-stretch	Nasca, Paulstretch, 2006	Open source

Three of these matter most for immersive work:

• Precedence (Haas). When the same sound reaches you twice within about 1 to 30 milliseconds, you hear one sound from the direction of the first arrival. A short delay therefore widens a sound without moving it. Go too far and you hear an echo.
• Height bands (Blauert). Certain frequency regions make a sound seem to come from above, even from ordinary speakers. This is why the height speakers in 7.1.4 love bright, airy content.
• Moving sources (Chowning). A convincing fly-by needs more than panning: the level and brightness drop with distance, reverb grows, and the pitch bends as the source passes.

Hands-on in the classroom

1. Open Controls, then Effects Rack. It opens on NASA Sonification.
2. Press the headphones icon on "Perseus Black Hole". Set Preview on to Beep to hear the effect alone, then to Track with a sustained stem. Switch between Dry (A) and Effect (B).
3. Add it with Add to chain and set Mix to around 30% so the original stays readable.
4. Go to Mind Tricks, preview "Phantom Width 12ms" on a pluck, and compare A and B. Notice that it gets wider but does not move.
5. Try "Gravity Well Overhead" on a bright stem and listen for lift into the height speakers.
6. In Sound Hacks, preview "Ghost String C3 Glass Bell" with Beep. You are hearing the Karplus-Strong algorithm, a 1983 paper, in your browser.
7. Ask Niner, for example "nasa black hole on the pad", and check which effect it adds and to which stem.

Using science effects well

• Mix, do not drown. Research effects are strong. Start between 20% and 40% Mix and raise it only if the part still reads.
• One trick per stem. Stacking Haas width, a Dummy Head Dome and a fly-by on one stem confuses the very cues they rely on.
• Check in mono and on headphones. Precedence and HRTF tricks behave differently on speakers and headphones. Test both.
• Be honest in your credits. When you publish, say "inspired by NASA's Perseus sonification". Do not suggest that your track contains NASA data.

Reading list

Sonification and NASA

• Kramer, G. (ed.) (1994). *Auditory Display: Sonification, Audification, and Auditory Interfaces*. Addison-Wesley.
• Hermann, T., Hunt, A. and Neuhoff, J. G. (eds.) (2011). *The Sonification Handbook*. Logos. Free online.
• Walker, B. N. and Nees, M. A. (2011). "Theory of Sonification". In *The Sonification Handbook*.
• Arcand, K. K. et al. (2024). "A Universe of Sound: processing NASA data into sonifications to explore participant response". *Frontiers in Communication*, 9.
• Kolmašová, I. et al. (2018). "Discovery of rapid whistlers close to Jupiter implying lightning rates similar to those on Earth". *Nature Astronomy*, 2.
• Maurice, S. et al. (2022). "In situ recording of Mars soundscape". *Nature*, 605.
• NASA Chandra X-ray Center. "A Universe of Sound" sonification collection.

MIT and psychoacoustics

• Gardner, B. and Martin, K. (1994). "HRTF Measurements of a KEMAR Dummy-Head Microphone". MIT Media Lab Perceptual Computing Technical Report 280.
• Vercoe, B. Csound, MIT, 1986 onwards.
• Machover, T. Hyperinstruments, MIT Media Lab, 1986 onwards.
• Roads, C. (2001). *Microsound*. MIT Press.
• Bregman, A. S. (1990). *Auditory Scene Analysis*. MIT Press.
• Blauert, J. (1997). *Spatial Hearing*, revised edition. MIT Press.
• Haas, H. (1951). On the influence of a single echo on the audibility of speech. *Acustica*.
• Chowning, J. (1971). "The Simulation of Moving Sound Sources". *Journal of the Audio Engineering Society*.
• Shepard, R. N. (1964). "Circularity in Judgments of Relative Pitch". *Journal of the Acoustical Society of America*.
• Peterson, G. E. and Barney, H. L. (1952). "Control Methods Used in a Study of the Vowels". *Journal of the Acoustical Society of America*.
• Schroeder, M. R. (1962). "Natural Sounding Artificial Reverberation". *Journal of the Audio Engineering Society*.
• Zwicker, E. (1961). "Subdivision of the Audible Frequency Range into Critical Bands". *Journal of the Acoustical Society of America*.
• Moorer, J. A. (1979). "About This Reverberation Business". *Computer Music Journal*.
• Karplus, K. and Strong, A. (1983). "Digital Synthesis of Plucked-String and Drum Timbres". *Computer Music Journal*.
• Dattorro, J. (1997). "Effect Design, Part 1: Reverberator and Other Filters". *Journal of the Audio Engineering Society*.
• ISO 226. Acoustics: Normal equal-loudness-level contours.

Check yourself

1. What is the difference between audification and parameter mapping?
2. How many octaves higher did NASA play the Perseus black hole's pressure waves?
3. Why can Voyager and Juno "hear" Jupiter even though space carries no ordinary sound?
4. What two speeds of sound did Perseverance measure on Mars?
5. What did Gardner and Martin measure at MIT in 1994, and which effect uses that idea?
6. Which classroom effect comes from Karplus and Strong, and where did they work?
7. Why does a 12 millisecond delay make a sound wider instead of creating an echo?

Answers

1. Audification plays a wave-like signal directly. Parameter mapping assigns data values to sound properties such as pitch, loudness or position.
2. 57 and 58 octaves.
3. They record plasma waves and radio emissions, which can be converted straight into audio.
4. About 240 metres per second for low pitches and about 250 metres per second for higher ones.
5. KEMAR dummy-head HRTFs at 710 positions. The "Dummy Head Dome" effects use that idea.
6. "Ghost String C3 Glass Bell". Karplus and Strong worked at Stanford, not MIT.
7. Because of the precedence effect: within about 1 to 30 milliseconds, the brain fuses both arrivals into one sound.

Assignment

Create a 30-second "space interlude" for Echoes. Use one NASA Sonification effect on a sustained stem and one Mind Tricks effect on a second stem, then place both in the Sphere so their directions make sense. Write a short liner note of about 150 words that names each effect, the research or mission behind it (with year), the Mix percentage you chose, and one honest sentence on what is real data and what is a creative recreation.

Frequently asked questions

What is NASA sonification?

NASA sonification turns telescope and spacecraft data into sound, mapping brightness, position and wavelength to loudness, pitch, timing and instruments so the data can be heard.

Is it spelled sonification or soundification?

The scientific term is sonification, the use of non-speech sound to convey information, though people sometimes write soundification.

Who makes NASA's sonifications?

The Chandra X-ray Center's project is led by Dr. Kimberly Arcand with Dr. Matt Russo and Andrew Santaguida of SYSTEM Sounds.

Can you really hear a black hole?

The Perseus black hole creates real pressure waves in hot gas, and NASA made them audible in 2022 by playing them 57 and 58 octaves higher.

What does MIT have to do with spatial audio?

MIT Media Lab's 1994 KEMAR HRTF measurements by Gardner and Martin became one of the most used datasets for 3D sound on headphones and in renderers.

Do the classroom NASA effects contain real NASA data?

No, they are effects that recreate the character of NASA sonifications on your own stems, and each one names the mission that inspired it.

What are the most important books on psychoacoustics for spatial audio?

Start with Blauert's Spatial Hearing, Bregman's Auditory Scene Analysis, Roads' Microsound and The Sonification Handbook.

Next step

You now know where the science effects come from. Next, invent your own by describing it or making the sound with your voice in Classroom Lab 3: Design Audio Effects With Words and Your Voice. For the full rack, go back to The Effects Rack: 1,459 Immersive and Research Effects Explained, or return to the Spatial9 Academy course home.