Real-Time Control of Sonification Models with a Haptic Interface

download postscript version

Authors: Thomas Hermann, Jan Krause and Helge Ritter
ICAD 2002, July 2002, Japan

Sound Demonstrations for the Audio-Haptic Ball Interface

Sounds for atomic collisions:

Collision between a A object with A object: sound (wood)
Collision between a B object and a B object: sound (plastic)
Collision between a A object and a B object: sound (glass)

Table 1: Sound examples for synthetic datasets from binary classification problems

File/Track:	A) two separated classes -- sound B) two classes showing little overlap along one axis -- sound C) two classes that overlap - inseparable classes -- sound
Description:	The sonification model was excited by shaking the interface device. The shaking activation is given by the acceleration a_x(t), a_y(t) . During the first half of the sound examples, the interface is shaken along the x axis, during the second half along the y axis.
Duration:	about 5 sec.

Table 2: Sound examples for Data solid sonifications using a GNG for the dataset (B) (see above) for different network complexities.

File/Track:	A: 5 neurons sound B: 10 neurons sound C: 15 neurons sound D: 20 neurons sound E: 25 neurons sound F: 30 neurons sound
Description:	Shaking excitation of the dataset with increasing network complexity.
Duration:	about 2 sec / example

Sound Example for shaking while GNG adaptation proceeds, and thus while the data-solid structure changes over time: sound example . It can be heard that with ongoing GNG growth more and more neurons exist (more collisions). From the pitch it can be perceived that the number of data points that have a neuron as its nearest neighbor decreases - new neurons are added between neurons that are frequently activated.