Sonification Principles: Turning Data into Sound

Module A: Sonification in 3D. Lesson 1 of 20, 30 min.

Abstract

Sonification turns data relations into perceived relations in sound, but only rigorous mapping makes the result readable. This lesson covers the auditory display taxonomy, polarity and scaling, psychoacoustic limits and evaluation methods, with case studies from LIGO, NASA, the clinic and finance.

By the end you can

  • Distinguish audification, parameter-mapping, model-based sonification, earcons and auditory icons and choose the right one for a task
  • Specify mappings with justified polarity, scaling and range as reproducible, invertible functions
  • Identify psychoacoustic constraints such as parameter interaction, masking and streaming that undermine a sonification
  • Design a task-based evaluation with objective metrics and appropriate baselines

A chart asks you to look at data all at once. A sonification asks you to listen to it as it unfolds, and that one difference changes what you can notice, what you will miss, and how the display has to be designed. This lesson sets out the vocabulary, mapping theory and psychoacoustic limits that separate a rigorous auditory display from "data that happens to make noise", and it prepares the ground for the next lesson, where we use three-dimensional space as an additional data channel.

What sonification is, and why it is worth taking seriously

The 1999 Sonification Report prepared for the International Community for Auditory Display (ICAD) by Kramer and colleagues gave the field its working definition: sonification is the use of non-speech audio to convey information, and more specifically the transformation of data relations into perceived relations in an acoustic signal. Hermann (2008) later tightened this into criteria: the sound must reflect objective properties of the data, the transformation must be systematic and reproducible, and the same system must work with different data. A composer "inspired by" a climate dataset is not doing sonification; a designer who maps temperature to pitch with a documented, invertible function is.

There are three good reasons to use sound at all:

  • Temporal resolution and pattern sensitivity. Hearing excels at rhythm, periodicity and small changes in regularity, so anomalies buried in a dense plot can become obvious.
  • Eyes-free monitoring. Surgeons, pilots and operators already rely on sound while their eyes are busy elsewhere.
  • Accessibility. For blind and low-vision users, sonified graphs and data are a primary route to quantitative information.

Sound also has costs: it is transient, so the listener must hold the past in memory; it is easily fatiguing; and it reads absolute values worse than vision does. Good design treats these as constraints, not afterthoughts.

The rest of this lesson, its lab and self-check are part of Academy Advanced, a paid course. See who has access.

Key readings

  • Kramer, G., Walker, B. N., Bonebright, T., Cook, P., Flowers, J., Miner, N., & Neuhoff, J. (1999). Sonification Report: Status of the Field and Research Agenda. International Community for Auditory Display (ICAD).
  • Hermann, T., Hunt, A., & Neuhoff, J. G. (Eds.) (2011). The Sonification Handbook. Logos Publishing House, Berlin.
  • Walker, B. N. (2002). Magnitude Estimation of Conceptual Data Dimensions for Use in Sonification. Journal of Experimental Psychology: Applied, 8(4), 211-221.
  • Walker, B. N., & Kramer, G. (2005). Mappings and Metaphors in Auditory Displays: An Experimental Assessment. ACM Transactions on Applied Perception, 2(4), 407-412.
  • Dubus, G., & Bresin, R. (2013). A Systematic Review of Mapping Strategies for the Sonification of Physical Quantities. PLoS ONE, 8(12), e82491.