- Thomas Hermann, Helge Ritter (2004)
In Banissi, Ebad and Börner, Katy (Ed.) IV '04: Proceedings of the Information Visualisation, Eighth International Conference on (IV'04), p. 871--878, IEEE CNF, IEEE Computer Society, Washington, DC, USA
[BibTeX Entry]
[Download PDF]
Summary
In this paper we present an approach using incrementally constructed neural gas networks to 'grow' an intuitive interface for interactive exploratory sonification of high-dimensional data. The sonifications portray information about the intrinsic data dimensionality and its variation within the data space. The interface follows the paradigm of model-based sonification and consists of a graph of nodes that can be acoustically \'yexcited\'y with simple mouse actions. The sound generation process is defined in terms of the node parameters and the graph topology, following a physically motivated model of energy flow through the graph structure. The resulting sonification model is tied to the given data set by constructing both graph topology and node parameters by an adaptive, fully data-driven learning process, using a growing neural gas network. We report several examples of applying this method to static data sets and point out a generalization to the task of process analysis- d = 1 : Example_1a (mp3, 16k) Example_1b (mp3, 16k)
- d = 2 : Example_2a (mp3, 16k) Example_2b (mp3, 16k)
- d = 3 : Example_3a (mp3, 16k) Example_3b (mp3, 16k)
- d = 4 : Example_4a (mp3, 16k) Example_4b (mp3, 16k)
- d = 5 : Example_5a (mp3, 16k) Example_5b (mp3, 16k)
- d = 6 : Example_6a (mp3, 16k) Example_6b (mp3, 16k)
- d = 7 : Example_7a (mp3, 16k) Example_7b (mp3, 16k)
- d = 8 : Example_8a (mp3, 16k) Example_8b (mp3, 16k)
- N = 3: Example_1 (mp3, 16k) Example_1 (mp3, 16k)
- N = 20: Example_2a (mp3, 16k) Example_2b (mp3, 16k) Example_2c (mp3, 16k)
- N = 45: Example_3a (mp3, 16k) Example_3b (mp3, 16k) Example_3c (mp3, 16k)
- N = 150:Example_4a (mp3, 16k) Example_4b (mp3, 16k) Example_4c (mp3, 16k)
- The following examples are for a noisy spiral with only one rotation
- N = 20: Example_5a (mp3, 16k) Example_5b (mp3, 16k) Example_5c (mp3, 16k)
- N = 45: Example_6a (mp3, 16k) Example_6b (mp3, 16k) Example_6c (mp3, 16k)
- Noisy Spiral, 1 rotation Example_1 (mp3, 45k)
- Noisy Spiral, 2 rotation Example_2 (mp3, 45k)
- Gaussian Cluster of dimension 5 Example_3 (mp3, 45k)
- Mixture of 2d and 5d uniform distributions Example_4 (mp3, 45k)
- Handwritten digits, class '1', Probing: (a) (b) (c) (d) (mp3, 17k)
- Handwritten digits, class '1', Growth sonification: growth (mp3, 33k)
- Handwritten digits, class '2', Probing: (a) (b) (c) (d) (mp3, 17k)
- Handwritten digits, class '2', Growth sonification: growth (mp3, 33k)
Contact
Thomas Hermann
Helge Ritter
Neural Gas Sonification - Growing Adaptive Interfaces for Interacting with Data
Media files
Sound Examples: GNGS -- Probing Gaussian Distributions
The sonifications are currently computed offline: after a click in a
scatterplot of the data the nearest neuron of the GNG is searched and the
graph is excited at this location. Since energy propagates along
topological connections between the GNG graph, the sound is completely
determined from a connected subgraph, e.g. a cluster. The sonifications
last about 2 secs, and are presented to the listener as soon as they are
computed. Since this interrupts the exploratory flow only slightly, the
user is appearing an (discrete) interactive mode of exploration.
Sonifications for probing in the clusters of intrinsic dimension d:
Sound Examples: GNGS -- Probing Sonification for the Noisy Spiral dataset
A noisy spiral dataset is adapted by a GNG. This are GNGS probing sonifications from (a) the outer end of the spiral, (b) in the middle, (c) the inner end of the spiral. The examples are for differing network complexity, expressed by the number of neurons N.
Sound Examples: GNGS -- Process Monitoring Sonifications
The following for sonification present the dynamically changing auditory state of a GNG during the adaptive growth process.
Sound Examples: GNGS Probing and Process Monitoring for MNIST dataset
The MNIST dataset contains 24x24 pixel bitmaps of handwritten digits. 8x8 subsampling was performed to obtain 64-dimensional records, about 1000 records for each class. The examples are computed for the classes of '1' and '2'. The shape of '2' contains more internal degrees of freedom, resulting in a higher intrinsic dimensionality of the distribution, audible from the higher brilliance and complexity of the probing sonification.