- Jörg Martini, Thomas Hermann, Dario Anselmetti,
Helge Ritter (2004)
In Hermann, Thomas and Hunt, Andy (Ed.) Proceedings of the International Workshop on Interactive Sonification (ISon 2004), Bielefeld University, Interactive Sonification Community, Bielefeld, Germany, (peer-reviewed article)
[BibTeX Entry]
[Download PDF]
Summary
This paper presents an interactive audio-haptic human-computer interface for controlling an atomic force microscope (AFM) in force spectroscopy experiments on single molecules. The sensor data used are proportional to the force that is applied to a single molecule. These forces are measured in real-time by using the reflection of a laser beam from a cantilever. We present a system that involves (a) a visual display of the data (b) a force-feedback joystick for navigating the sample and providing a tactile feedback of the forces, and (c) an auditory display to monitor the measured data while interactively moving the sample. The sonification we have developed integrates information at various levels of detail, including audifications of the high-frequency cantilever movement, and an auditory stream that communicates the instantaneous deviation of forces between the approach and retract phase of the sample. The sonification design and offline-computed sonifications will be presented and discussed. We further report on our first experiences with this interactive multi-modal control interface for manipulation of individual DNA molecules.
- (a) data files where DNA actually binds to the cantilever
S_a1 (mp3, k) S_a2 (mp3, k) S_a3 (mp3, k) S_a4 (mp3, k)
S_a5 (mp3, k) S_a6 (mp3, k) S_a7 (mp3, k) S_a8 (mp3, k) - (b) data files where nothing happens (no binding)
S_b1 (mp3, k) S_b2 (mp3, k) S_b3 (mp3, k) S_b4 (mp3, k)
S_b5 (mp3, k) S_b6 (mp3, k) S_b7 (mp3, k) S_b8 (mp3, k) - (c) audifications of unknown things going on at the cantilever
S_c1 (mp3, k) S_c2 (mp3, k) S_c3 (mp3, k) S_c4 (mp3, k)
S_c5 (mp3, k) S_c6 (mp3, k) S_c7 (mp3, k) S_c8 (mp3, k) - (d) audifications of sticktip events
S_d1 (mp3, k) S_d2 (mp3, k) S_d3 (mp3, k) S_d4 (mp3, k)
S_d5 (mp3, k) S_d6 (mp3, k) S_d7 (mp3, k) S_d8 (mp3, k) - Sonification example 1 - wind sound layer
S_mbs_1 (mp3, k) - Sonification example 2 - as above plus tone sound layer
S_mbs_2 (mp3, k) - Sonification example 3 - as above plus marker sound layer
S_mbs_3 (mp3, k) Contact
Thomas Hermann
Interactive Sonification for exploring Single Molecule Properties with AFM-based Force Spectroscopy
Media files
Audifications of DNA measurements:
Interactive Sonifications using the Simulator System in Neo/NST
The sounds contain 3 layers: (a) a wind-sound pattern that corresponds to
the absolute force measured, (b) a repeating tone, whose pitch is
determined by the net binding force, (c) marker sounds that are played on
certain events connected to a specific physical meaning, like e.g. the
rupture of a binding. The sound examples represent typical interactions
using the simulator, a picture shows a screenshot of the system.
Currently, the slider is used to navigate the cantilever, in the real AFM
system, here the force feedback joystick comes into play.