- Thomas Hermann, Thomas Henning, Helge Ritter (2004)
In Camurri, Antonio and Volpe, Gualtiero (Ed.) Gesture-Based Communication in Human-Computer Interaction, 5th International Gesture Workshop, GW 2003 Genova, Italy, April 15-17, 2003, Selected Revised Papers, p. 369--379, Gesture Workshop, Springer, Berlin, Heidelberg
[BibTeX Entry]
Summary
This paper presents the gesture desk, a new platform for a human-computer interface at a regular computer workplace. It extends classical input devices like keyboard and mouse by arm and hand gestures, without the need to use any inconvenient accessories like data gloves or markers. A central element is a gesture box containing two infrared cameras and a color camera which is positioned under a glass desk. Arm and hand motions are tracked in three dimensions. A synchronizer board has been developed to provide an active glare-free IR-illumination for robust body and hand tracking. As a first application, we demonstrate interactive real-time browsing and querying of auditory self-organizing maps (AuSOMs). An AuSOM is a combined visual and auditory presentation of high-dimensional data sets. Moving the hand above the desk surface allows to select neurons on the map and to manipulate how they contribute to data sonification. Each neuron is associated with a prototype vector in high-dimensional space, so that a set of 2D-topologically ordered feature maps is queried simultaneously. The level of detail is selected by hand altitude over the table surface, allowing to emphasize or deemphasize neurons on the map.- Sound Example 1: AuSOM1.wav (580kB)
Contact
Thomas Hermann
Helge Ritter
Gesture Desk - An Integrated Multi-modal Gestural Workplace for Sonification
Media files
On this page, sound examples are provided for browsing AuSOM (Auditory Self-Organizing Maps) with our gesture desk.Section 5: Sound Example for browsing the AuSOM for the Iris data set
The sonification was rendered by moving the hand slowly over the map, with an constant altitude over the desk surface, thus the aura size is constant. The different rhythmical patterns represent the average feature values averaged over the contributing neurons in the aura.