Universität Bielefeld Universität Bielefeld - Technische Fakultät - AG Wissensbasierte Systeme

Jahresübersicht 2002

Veröffentlichungen des Jahres 2002 inklusive aller verfügbaren Abstracts

- Download PDF - BibTeX

P. Biermann, B. Jung, M. Latoschik & I. Wachsmuth:
Virtuelle Werkstatt: A Platform for Multimodal Assembly in VR

In: Proceedings Fourth Virtual Reality International Conference (VRIC 2002),
Laval, France, 19-21 June 2002, 53-62.

- Download PDF - BibTeX

In this paper we describe ongoing research that aims at the development of a generic demonstration platform for virtual prototype modeling by utilizing multimodal ­ speech and gesture ­ interactions in Virtual Reality. Particularly, we concentrate on two aspects. First, a knowledge- based approach for assembling CAD-based parts in VR is introduced. This includes a system to generate meta-information from geometric models as well as accompanying task-level algorithms for virtual assembly. Second, a framework for modeling multimodal interaction using gesture and speech is presented that facilitates its generic adaptation to scene-graph-based applications. The chosen decomposition of the required core modules is exemplified by an example of a typical object rotation interaction.

Virtual Reality, Virtual Assembly, Multimodal Interaction, Interaction Decomposition, Task-Level Interfaces

T. Jörding, I. Wachsmuth:
An Anthropomorphic Agent for the Use of Spatial Language.

In K.R. Coventry & P. Olivier (eds.):
Spatial Language: Cognitive and Computational Aspects.
(pp. 69-86), Dordrecht: Kluwer, 2002.

- Download PDF - BibTeX

In this paper we describe communication with a responsive virtual environment with the main emphasis on the processing of spatial expressions in natural language instructions. This work is part of the VIENA project in which we chose interior design as an example domain. A multi- agent system acts as an intelligent mediator between the user and a graphics system. To make the communication about spatial relations more intuitive, we developed an anthropomorphic agent, which is graphically visualised in the scene. With reference to the human-like figure we explain the use of qualitative spatial expressions, like "right of" and "there".

B. Jung, M. Latoschik, P. Biermann & I. Wachsmuth:
Virtuelle Werkstatt.

In: 1. Paderborner Workshop Augmented Reality / Virtual Reality in der Produktentstehung,
(pp. 185-196), Paderborn: HNI, 2002.

- Download PDF - BibTeX

Das Projekt "Virtuelle Werkstatt" will Forschungsarbeiten aus den Bereichen Multimodale Interaktion und Virtuelles Konstruieren erweitern und derart zusammenführen, daß ihre realitätsnahe Erprobung in der Virtuellen Realität (VR) demonstrierbar wird. Multimodale Interaktion betrifft die unmittelbare Umsetzung von Benutzereingriffen in einer visualisierten 3D-Szene aufgrund von sprach- begleiteten Gesteneingaben. Virtuelles Konstruieren betrifft die Erstellung und Erprobung computergraphisch visualisierter 3D-Modelle geplanter mechanischer Konstruktionen (sog. virtueller Prototypen), um eine realistische Vorabexploration von Entwürfen per Simulation in der Virtuellen Realität zu ermöglichen. Der Einsatz eines Cave-artigen VR-Großdisplays macht hierbei gleichzeitig Benutzer- interaktionen mit sprachbegleiteten Gesteneingaben im Greifraum wie auch im Fernraum erforschbar.

B. Jung, T. Pfeiffer & J. Zakotnik:
Natural Language Based Virtual Prototyping on the Web.

In C. Geiger et al. (eds.):
Proceedings Structured Design of Virtual Environments and 3D-Components.
Aachen: Shaker, 2002.

- Download PDF - BibTeX

This contribution describes a WWW-based multi-user system for concurrent virtual prototyping. A 3D scene of CAD parts is presented to the users in the web browser. By instructing the system using simple natural language commands, complex aggregates can be assembled from the basic parts. The current state of the assembly is instantly published to all system users who can discuss design choices in a chat area. The implementation builds on an existing system for virtual assembly made available as a web service. The client side components are fully implemented as Java applets and require no plugin for visualization of 3D content. Http tunneled messaging between web clients and server ensures system accessibility from any modern web browser even behind firewalls. The system is first to demonstrate natural language based virtual prototyping on the web.

S. Kopp, I. Wachsmuth:
Model-based Animation of Coverbal Gesture

Proceedings of Computer Animation 2002, pp. 252-257, IEEE Press, Los Alamitos, CA, 2002.

- Download PDF - BibTeX

Virtual conversational agents are supposed to combine speech with nonverbal modalities for intelligible and believeable utterances. However, the automatic synthesis of coverbal gestures still struggles with several problems like naturalness in procedurally generated animations, flexibility in pre-defined movements, and synchronization with speech. In this paper, we focus on generating complex multimodal utterances including gesture and speech from XML-based descriptions of their overt form. We describe a coordination model that reproduces co-arcticulation and transition effects in both modalities. In particular, an efficient kinematic approach to creating gesture animations from shape specifications is presented, which provides fine adaptation to temporal constraints that are imposed by cross-modal synchrony.

A. Kranstedt, S. Kopp & I. Wachsmuth:
MURML: A Multimodal Utterance Representation Markup Language for Conversational Agents

In Proceedings of the Workshop Embodied Conversational Agents - let's specify and evaluate them,
Autonomous Agents & Multi-Agent Systems (AAMAS02), Bologna, Italy, July 2002.
Also available as technical report 2002/05 of the SFB 360 at the University of Bielefeld.

- Download PDF - BibTeX

This paper presents work on an artificial anthropomorphic agent with multimodal interaction abilities. It focuses on the development of a markup language, MURML, that bridges between the planning and the animation tasks in the production of multimodal utterances. This hierarchically structured notation provides flexible means of describing gestures in a form-based way and of explicitly expressing their relations to accompanying speech.

M. E. Latoschik:
Designing Transition Networks for Multimodal VR-Interactions Using a Markup Language

In: Proceedings of the IEEE fourth International Conference on Multimodal Interfaces, ICMI 2002,
Pittsburgh, USA, October 2002, 411-416.


T. Sowa & I. Wachsmuth:
Interpretation of Shape-Related Iconic Gestures in Virtual Environments.

In I. Wachsmuth & T. Sowa (eds.): Gesture and Sign Language in Human-Computer Interaction
(pp. 21-33). Berlin: Springer (LNAI 2298), 2002.

- Download PDF - BibTeX

So far, approaches towards gesture recognition focused mainly on deictic and emblematic gestures. Iconics, viewed as iconic signs in the sense of Peirce, are different from deictics and emblems, for their relation to the referent is based on similarity. In the work reported here, the breakdown of the complex notion of similarity provides the key idea towards a computational model of gesture semantics for iconic gestures. Based on an empirical study, we describe first steps towards a recognition model for shape-related iconic gestures and its implementation in a prototype gesture recognition system. Observations are focused on spatial concepts and their relation to features of iconic gestural expressions. The recognition model is based on a graph-matching method which compares the decomposed geometrical structures of gesture and object.

I. Voss:
Gedächtnismodelle für die virtuelle Konstruktion in der virtuellen Realität

In C. Vorwerg & I. Voss (eds.): Gedächtnisprozesse in Interaktionen,
pp. 41-46, SFB Report 03/02, Universität Bielefeld. 2002.


I. Wachsmuth:
Communicative Rhythm in Gesture and Speech

In P. Mc Kevitt, S. O'Nuállain & C. Mulvihill (eds.): Language, Vision and Music,
(pp. 117-132), Amsterdam: Benjamins, 2002 (reprinted by permission of Springer-Verlag).


I. Wachsmuth & S. Kopp:
Lifelike Gesture Synthesis and Timing for Conversational Agents.

In I. Wachsmuth & T. Sowa (eds.): Gesture and Sign Language in Human-Computer Interaction (pp. 120-133).
Berlin: Springer (LNAI 2298), 2002.

- Download PDF - BibTeX

Synchronization of synthetic gestures with speech output is one of the goals for embodied conversational agents which have become a new paradigm for the study of gesture and for human-computer interface. In this context, this contribution presents an operational model that enables lifelike gesture animations of an articulated figure to be rendered in real-time from representations of spatiotemporal gesture knowledge. Based on various findings on the production of human gesture, the model provides means for motion representation, planning, and control to drive the kinematic skeleton of a figure which comprises 43 degrees of freedom in 29joints for the main body and 20 DOF for each hand. The model is conceived to enable cross-modal synchrony with respect to the coordination of gestures with the signal generated by a text-to-speech system.

I. Wachsmuth & N. Leßmann:
Eine kognitiv motivierte Architektur für einen anthropomorphen Künstlichen Kommunikator.

In: Tagungsbeiträge "Human Centered Robotic Systems 2002",
Karlsruhe, Dezember 2002, 141-148.

- Download PDF - BibTeX

Für den im Teilprojekt D3 des SFB 360 verfolgten Auftrag der Systemintegration stellt sich mit der Zusammenführung perzeptiver, kognitiver und aktorischer Komponenten die Aufgabe der Gesamt- konzeption der Architektur eines Situierten Künstlichen Kommunikators. In dem vorliegenden Beitrag wird dies am Beispiel des anthropomorphen Agenten Max" exploriert, der in der virtuellen Realität verkörpert ist. Dabei wird ein besonderer Schwerpunkt auf die kommunikativen Fähigkeiten des Agenten unter Berücksichtigung seiner Körperlichkeit gelegt. Vorgestellt werden die Kernideen einer kognitiv motivierten Hybrid-Architektur für einen kommunizierenden Agenten und erste Realisierungsansätze.

A. Kranstedt, 25.07.2003