Data Exploration by Sonification

See for more up-to-date information

The way that we understand data is determined by the way we access data. As most data presentation techniques up to now are based on data visualization, researchers strongly developed an visual imagination of data spaces.

However, there are alternatives to visual data presentation, which might be superior for certain types of data. One of them is acoustic data presentation, or Sonification. Sonification is a new emerging field in HCI research, which closes a gap between the usually complex visual computer displays and the very poor acoustic interface to the user.

-Why to use other senses than vision?

We are capable in discerning subtle changes in acoustic patterns which is shown to a high degree by the medicine who routinely applies acoustic data analysis when applying the stethoscope to listen to breath noise and heart tones. We think there is a high potential in making use of acoustic data presentation to understand data in new ways which may accelerate, simplify and support the data mining process.
Just as a car mechanic listens to the sound of an engine to get information about causes of malfunction and hints what to do, the data miner may listen to the data and get cues what structures are found in the data: Sonification may give information about the clustering of data, the local intrinsic data dimensionality, class overlap, topological structures, etc.
Besides that, there are other interesting applications of sonification: e.g. it just can be applied to monitor the state of a learning neural network, or to monitor large amounts of data like longtime-EEG's.
In classification tasks, where a human expert evaluates the data, it can accelerate classification and improve performance.

-Some Sonification Types

Problems with Parameter Mapping

Parameter Mapping is the richest method for presentation of high dimensional data, but it has a couple of problems: the sonification is not invariant to spatial transformations, there is the need to assign a lot of attributes, which makes every sonification sound different. Interpretation is difficult, as the attributes have always to be looked up. Furthermore, the dimensionality is limited to the number of instrument parameters.

To overcome some of these problems a new approach was seeked and found with

-Model-Based Sonification

Here, the idea is to take the way that we use sound in our natural environment as a model for the usage in data sonification. Our acoustic senses are optimized to listen to natural sounds and are furthermore specialized to draw information from it. So, the idea is to build a sonification interface, in which  the sound is produced similar to the physical model and where the modes of interaction with the data takes the usual proceeding in inspecting objects into account.
Normally, real world objects are silent in a state of equibrilium. They produce sound when they are excited (struck, hit) Analogous, the data only produces sound if its excited by the user. The excitation happens in a visual interface. So to say, a artificial ,,dynamic data instrument`` is determined by the model and is played by the user. To determine a model based sonification, some things have to be determined:
  1. the system set up - what are the degrees of freedom
  2. the virtual physics - how are the d.o.f.'s coupled
  3. the acoustic observables - how do the d.o.f.'s contribute to an acoustic signal
  4. the modes of interaction - how might the user excite the system
  5. the listener attributes - how do acoustic waves propagate to the user and user location
Here are some advantages of this approach:

-Example: Data Sonograms

Here analougous to real sonograms, a shock wave is introduced to a fixed position in data space. The shock wave expands spherically in the data space, exciting all data points to a vibrational motion around their position if reached by the shock wave. The motion of the data points is determined by the local potential, the local friction forces. These properties are given by local attributes of the data, e.g. the local probability density and the local class entropy.
By listening to data sonograms, it can be perceived if distinct classes pentrate or if they are separated.
It may be perceived how much data contributes to a class. Furthermore, it may be perceived if a class has outliers.

An advantage is, that listening to sonifications does not interrupt the visual display. It might become an important interface in highly visual demanding tasks (e.g. surgery tasks).

-Current Research Projects

The current work consists in development of new sonification strategies for effective, easy-to-learn, easy-to-use acoustic data mining tools. Making our listening capabilities usable should extend visual displays rather than replace them.
Sonification is a strongly interdisciplinary field of research. Research of acoustic perception, cognition, sound metrics, sound representation, semantics of sound, efficient sound synthesis (e.g. with physical models), technical acoustics, psychoacoustics contribute to the design of sonification tools.

Some research projects are presented in greater detail on the project pages:



Thomas Hermann: thermann(at)

Last modified: Tue 07-14-1999