With a background in Machine Learning (s. diploma thesis), my current research interest is in computational modeling of human cognition (s. Ph.D. thesis).
Imitation Mechanisms and Motor Cognition for Social Embodied Agents |
In social interactions we are continuously confronted with a variety of verbal and nonverbal behavior, such as speech, facial expressions and gestures. In doing so, we need to be able to both recognize and generate such behavior. If computers could also recognize and understand natural communication (e.g. speech or gesture), humans would be able to interact with complex systems more naturally and intuitively. The other way around, when these complex systems are embodied in robots or virtual characters, they can generate social behavior that can be intuitively interpreted. This capability of performing embodied social reactions to recognized social cues closes the loop of natural communication. As a result, complicated tasks can be performed with minimal effort.
In the cognitive science of social interaction, human social capabilities and characteristics are studied from both a neurobiological and a psychological point of view. For this purpose, human social behaviors are analyzed with respect to different social stimuli. Given the same or similar stimuli, if a computer algorithm could generate the same output as a human would, this algorithm would be able to simulate the corresponding human social behavior. In this way, the human form of social interaction could be used to design a model of the interaction between computers and humans. Such a cognitive model can be designed based on psychological and neurobiological evidence, as well as on cognitive theories. As a result, on the one hand, robots or virtual characters could be endowed with social capabilities so that they can interact with humans in different assistance scenarios. On the other hand, successful implementation of cognitive theories would validate their explanation efficiency.
In this context, recent technological advances in motion tracking systems have opened up new perspectives for recognizing non-verbal behavior. These recent technological advances draw attention to new research focus on hand-arm gestures. We intend to give humanoid virtual agents the capability to cope with iconic gestures while interacting with humans. This aim raises the following research questions:
- How can human iconic gestures be perceived, processed, recognized and understood as social cues?
- How can a virtual agent learn to perform iconic gestures through imitation from a human user?
A computational model that fulfills these requirements would enable virtual humanoid agents, which can then be engaged in gestural interaction with humans.
The developed computational model for embodied gesture perception and generation, both grounded in a shared motor knowledge.
See publications for results how the developed computational model endows a virtual agent with social capabilities during social interaction with human interlocutors.
In the following videos, Vince our virtual humanoid agent shows different aspects of the developed computational model in two different demonstration scenarios.
The virual agent, Vince, in interaction with a human user
Vince recognizes and learns to generate gestures for his toys
Algorithms of Automatic Reconstruction of Neurons from the Confocal Images |
The morphology of neurons plays a decisive role in information processing carried out by the entire nervous system. Therefore, the three-dimensional reconstruction of neurons gains in importance for neurobiological analysis. The large quantity of proposed fully- and semi-automated reconstruction methods indicates the demand for the 3d modeling of neurons. We developed a method to generate a fully automated 3d reconstruction of neurons with the aid of algorithms of machine learning and computer vision.
The input for the algorithms is image stacks of the neurons, consisting of several slices, which are recorded by means of confocal microscopy. The automatic reconstruction process, which consists of several successive algorithms, constructs a 3D geometrical model, that will ultimately represent the displayed neuron's structure in the image slices, with high accuracy. The model should possess the topological and metric properties of the neuron structure so as to enable further morphological analysis. The variety of neuron structures and the noise of microscopy images require, on the one hand, flexible models, and on the other hand, models that exhibit robustness against noise. Due to these conflicting requirements, a perfect reconstruction is impossible. Hence, the automatic reconstructed model can be corrected and enhanced through the use of developed semi-automated fitting algorithms and an implemented graphical user interface.
Download the diploma thesis in PDF format (4.8 MB).