How we hear in 3D

Interaural cues, spectral cues, distance, precedence and the psychoacoustic limits every mix must respect.

This page describes the auditory cues that the brain uses to locate sound in three dimensions: interaural time and level differences, spectral cues from the outer ear, distance cues and the precedence effect. It summarizes the limits of each cue, explains what they mean for immersive mixing and outlines the research that established them.

It is intended for engineers, producers, sound designers and students who work in Dolby Atmos, Eclipsa Audio, Ambisonics or binaural formats. Every immersive mix relies on these mechanisms, so knowing them lets you design for the way listeners actually hear instead of placing sounds by trial and error.

Overview of spatial hearing

Psychoacoustics is the study of how people perceive sound, linking the physics of sound waves to perceived pitch, loudness, direction and timbre. The ears do not record sound like a microphone: the cochlea splits sound into frequency bands, the brainstem compares timing differences between the ears and the cortex fills gaps using expectation. If you know which details the brain relies on, you can reproduce them, which is how headphones create 3D space.

Any position around a listener is described by three coordinates on the listener sphere:

• Azimuth: the angle on the horizontal plane.
• Elevation: the angle above or below ear level.
• Distance: the length of the line from the listener to the source.

Object-based formats describe sounds with these coordinates, and Spatial9's AI model, GravityLLM, generates spatial arrangements in the same terms.

Interaural cues

Interaural cues compare the signals at the two ears. They are the primary cues for left and right.

Interaural time difference

Definition. The interaural time difference (ITD) is the difference in arrival time of a sound at the two ears. A source to your right reaches the right ear first; the sound has to travel further, around the head, to reach the left ear.

Limits. For a human head, the largest possible ITD is less than one millisecond. The auditory system resolves timing differences much smaller than that, a fraction of a millisecond, and it does this for every sound you hear.

Frequency range. ITD is most useful for low frequencies, where the waveform is long enough for the brain to compare its phase at each ear.

Interaural level difference

Definition. The interaural level difference (ILD) is the difference in level between the two ears, caused mainly by the head shadow. High frequencies have short wavelengths and do not bend easily around the head, so the far ear receives a quieter, duller copy of the sound.

Frequency range. ILD is most useful for high frequencies, where the head shadow is strongest.

Duplex theory

The division of labor between ITD at low frequencies and ILD at high frequencies is known as the duplex theory. Lord Rayleigh first described it more than a century ago.

Spectral cues and the HRTF

Time and level differences say little about front versus back or up versus down. Spectral cues fill that gap.

Pinna. The pinna is the visible outer ear. Its folds reflect and delay incoming sound in small ways. These reflections combine with the direct sound and create peaks and notches in the frequency spectrum. The pattern changes with direction, so a sound from above has a different spectral fingerprint than a sound from behind. The head, shoulders and torso add their own small reflections.

Head-related transfer function. The head-related transfer function (HRTF) describes how the head, outer ears and torso filter sound arriving from a given direction, for one ear, before it reaches the eardrum. A full HRTF set contains measurements for many directions around the head and is, in effect, the acoustic signature of a person.

Learned and individual. Listeners learn their own HRTF over a lifetime; the brain associates a particular notch in the upper frequencies with a source above. Because every ear is shaped differently, HRTFs differ between people, so binaural audio is more convincing for some listeners than others.

Elevation. Elevation is mostly judged from pinna cues. Elevation perception is usually less precise than left-right perception. Research by Jens Blauert in 1969 showed that certain frequency regions make a sound seem to come from above, even from ordinary speakers.

Cone of confusion and head movement

Definition. For any given ITD and ILD there is a cone of possible directions that produce the same values. This set of directions is the cone of confusion. A source directly in front and a source directly behind reach both ears at the same time and at the same level, so their ITD and ILD are identical.

Spectral cues from the pinna help resolve the ambiguity, but they can be ambiguous themselves, especially for unfamiliar sounds.

Resolution by head movement. The brain resolves the cone of confusion with small head movements. If you turn your head slightly to the left, a sound in front moves toward the right ear and a sound behind moves toward the left ear.

Head tracking. Without head tracking, the whole headphone scene turns with the head, which never happens with real sound. With it, sounds stay anchored in space, front and back become clearer and the mix often appears outside the head.

Distance cues

Distance is not one cue but a set of cues that the brain combines.

The direct-to-reverberant ratio is one of the strongest distance cues indoors. Level alone is only one distance cue: lowering a fader makes a sound quieter but does not necessarily make it seem further away.

Precedence effect

Definition. The precedence effect is the tendency of the brain to localize a sound by its first arrival and to suppress the direction of early echoes. In a room, a direct sound is followed by many reflections from walls, floor and ceiling, each from a different direction. When the same sound arrives from several directions within a short window, the brain places it where the first arrival came from. The later copies add loudness and a sense of space but do not pull the location away.

Time window. The sources describe the window as roughly a few tens of milliseconds, and more specifically as about 1 to 30 milliseconds. Within this window, two arrivals fuse into one sound located in the direction of the first arrival. For example, when the same sound arrives from two directions 5 ms apart, you hear one sound from the direction of the first arrival.

Beyond the window. When delays get longer, the echoes separate and are heard as distinct repeats.

Origin. The effect is associated with Helmut Haas, who published on the influence of a single echo on the audibility of speech in 1951 in Göttingen. It is also called the Haas effect.

Summary of cues and limits

The following table lists each cue with the frequency range, values and limits given in the source material.

Binaural rendering

All sound reaches the listener at two points: the left and right eardrums. If the correct signal is delivered to each eardrum, with the right timing, level and spectral filtering, the brain cannot tell it apart from a real sound in a real place.

Binaural rendering applies this principle. A renderer takes a sound and a desired direction, applies the HRTF for that direction to create a left and a right signal and plays them over headphones. With distance cues and head tracking added, sounds can appear in front, behind and above the listener on ordinary headphones. Any decent pair of stereo headphones or earbuds can play binaural audio, because the 3D cues are encoded in the two channels. Head tracking requires a device and app that support it.

Generic HRTFs do not match every ear, so front and back confusions are more common than with personalized measurements. A mix can appear inside the head when the cues do not match the listener's ears, when there are few room or distance cues, or when there is no head tracking.

Implications for mixing

The cues above set limits that every immersive mix must respect. Apply the following principles when you place and move sources.

Placement and panning

• A classic stereo pan pot changes only level, so it uses one cue out of several. A panned sound can appear to sit on a line between two speakers rather than in the room.
• Use separation in space to improve clarity. When sources come from different directions, the brain separates them more easily, an effect called spatial release from masking. Separating instruments or voices in space makes each one clearer without raising its level.

Height

• Elevation is perceived less precisely than azimuth. Height elements need clear, bright content to read as above.
• The height speakers in a 7.1.4 layout suit bright, airy content, because certain frequency bands are heard as coming from above.

Distance

• Do not rely on level alone to push a source back. Combine lower level with a lower direct-to-reverberant ratio and, for very distant sources, reduced high frequencies.
• Use near-field cues sparingly for intimate, close effects.

Width and reflections

• A short delay inside the precedence window widens a sound without moving it. Beyond the window, the delay is heard as an echo.
• Early reflections in a reverb add size without blurring position.

Movement

• A convincing moving source needs more than panning. As John Chowning described in 1971, level and brightness drop with distance, reverb grows and the pitch bends as the source passes.

Translation and checking

• Precedence and HRTF techniques behave differently on speakers and headphones. Check both, and check in mono.
• Do not stack several cue-based effects on one stem, for example precedence width, HRTF rendering and a fly-by. They confuse the cues they rely on.
• HRTFs and hearing differ between listeners. Record what you hear in a listening test, not what you expect to hear.

Research background

The cues on this page come from more than a century of research. The following table lists the work cited in the source material that is most relevant to spatial hearing.

KEMAR HRTF measurements

In May 1994, Bill Gardner and Keith Martin at the MIT Media Lab placed microphones in the ears of KEMAR, a dummy head with average adult proportions. They measured 710 positions, from 40 degrees below to directly overhead, with a loudspeaker 1.4 metres away, sampled at 44.1 kHz. The data was published freely as MIT Media Lab Perceptual Computing Technical Report 280 and became one of the most used HRTF sets in the world.

Reverberation statistics

James Traer and Josh McDermott showed in 2016 that the brain recognizes a room by the statistical shape of its echoes and that real-world reverb follows predictable patterns, which is why well-designed artificial rooms can sound natural.

Machine models of localization

In 2022, Andrew Francl and Josh McDermott trained a deep neural network to localize sounds in a simulated world with realistic HRTFs and reverberant rooms. It made human-like mistakes, used the same cues and showed the precedence effect, but behaved less like a human when trained without echoes or background noise. Human hearing is adapted to real acoustic environments.

Next steps

• Mixing best practices: apply these cues as ten concrete rules for immersive mixes.
• Binaural and headphones: learn how Spatial9 renders HRTF-based binaural output.
• Creative thinking: use spatial cues for creative effects without causing listener fatigue.
• Core concepts: review the terms used across immersive formats and the Spatial9 platform.