Virtuelle Werkstatt A demonstration platform for virtual-reality-based prototyping using gesture and speech

Die gleichen Informationen in Deutsch

Started in November 2001, the project "Virtuelle Werkstatt" aimed to combine and extend research results in multimodal human-computer interaction and immersive CAD systems to develop a generic demonstration platform for virtual prototyping based on Virtual Reality (VR) technology. Multimodal interaction is concerned with the processing of gesture and speech based user input to drive the modifications of a 3D-visualized scene. Immersive CAD is concerned with the design, exploration and evaluation of virtual prototypes of planned mechanical assemblies using VR simulations. The virtual prototypes are displayed in realistic size by a CAVE-like multi-sided projection system that facilitates research on both manipulative and communicative gestures for interaction with close and distant virtual parts. In the application domain of variant design of "Citymobile" vehicles, the project pursued the following goals:

• Development of an integrated framework for recognition, interpretation, and rendition of gesture and speech based interactions in VR.
• Development of knowledge based methods supporting the modeling, variation, and functional evaluation of virtual prototypes.
• Proof-of-concept implementation and sample applications e.g. involving the collaboration of spatially distributed design teams ("concurrent engineering").

Project team:
Prof. Dr. Ipke Wachsmuth, Dr. Marc Erich Latoschik, Dr. Peter Biermann, Dipl.-Inform. Christian Fröhlich, and (till 2003) Dr.-Ing Bernhard Jung

Student staff (2006/07):
Felix Rabe, Nils Peuser

Demo: [mpg; 71 MB] – TV broadcast (in German) "Projekt Zukunft", Deutsche Welle Fernsehen weltweit, 15 Jan 2006 (first showing)

Project related publications:

• Fröhlich, C., Wachsmuth, I., & Latoschik, M. E. (2009). Virtuelle Werkstatt – Multimodale Interaktion für intelligente virtuelle Konstruktion. In J. Gausemeier, M. Grafe (Eds.), 8. Paderborner Workshop Augmented & Virtual Reality in der Produktentstehung (pp. 241-255). Paderborn: HNI.

• Fröhlich, C., Biermann, P., Latoschik, M. E., & Wachsmuth, I. (2009). Processing Iconic Gestures in a Multimodal Virtual Construction Environment. In: M. Dias, S. Gibet, M. M. Wanderley, R. Bastos (Eds.), Gesture-Based Human-Computer Interaction and Simulation. Springer LNAI 5085, 187-192.

• Fröhlich, C. & Latoschik, M. E. (2008). Incorporating the Actor Model into SCIVE on a Semantic Level. In: Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), Proceedings of the IEEE Virtual Reality Workshop, Reno, USA, (pp. 61-64), Aachen: Shaker.

• Biermann, P., Fröhlich, C., Latoschik, M. E. & Wachsmuth, I. (2007) . Semantic Information and Local Constraints for Parametric Parts in Interactive Virtual Construction In: Proceedings of the 8th International Symposium on Smart Graphics 2007, SG2007 , Kyoto, Japan, June 2007 (pp. 124-134)

• Latoschik, M. E. (2007). Semantic Reflection – Knowledge Based Design of Intelligent Simulation Environments. In J. Hertzberg, M. Beetz, R. Englert (Eds.), KI 2007: Advances in Artificial Intelligence. LNCS 4667, Springer , 481-484.

• Latoschik, M. E. & Fröhlich, C. (2007). Semantic Reflection for Intelligent Virtual Environments. In: Proceedings of the IEEE VR2007, Charlotte (NC), USA, March 2007, pages 305-306.

• Latoschik, M. E. & Fröhlich, C. (2007). Towards Intelligent VR: Multi -layered Semantic Reflection for Intelligent Virtual Environments. In: Proceedings GRAPP 2007 – 2nd International Conference on Computer Graphics Theory and Applications, Barcelona, Spain, pages 249-259.

• Latoschik, M. E., Fröhlich, C. & Wendler, A. (2006). Scene Synchronization in Close-Coupled World Representations using SCIVE. International Journal of Virtual Reality, IJVR 5(3), 47-52.

• Heumer, G., Schilling, M. & Latoschik, M. E. (2005). Automatic Data Exchange and Synchronization for Knowledge-Based Intelligent Virtual Environments. In: Proceedings of the IEEE VR2005 Bonn, Germany, March 2005 (pp. 43-50).

• Latoschik, M. E. (2005). A User Interface Framework for Multimodal VR Interactions In: Proceedings of the IEEE Seventh International Conference on Multimodal Interfaces, ICMI 2005 Trento, Italy, October 2005 (pp. 76-83).

• Latoschik, M. E., Biermann, P. & Wachsmuth, I. (2005). High-level Semantics Representation for Intelligent Simulative Environments. In: Proceedings of the IEEE VR2005 Bonn, Germany, March 2005 (pp. 283-284).

• Latoschik, M. E., Biermann, P., & Wachsmuth, I. (2005). Knowledge in the Loop: Semantics Representation for Multimodal Simulative Environments. In: Proceedings of the 5th International Symposium on Smart Graphics, Springer LNCS 3638, pages 25-39.

• Pfeiffer, T., Weber, M. & Jung, B. (2005). Ubiquitous Virtual Reality: Accessing Shared Virtual Environments through Videoconferencing Technology. In: Proc. Theory and Practice of Computer Graphics 2005, Eurographics UK Chapter. (Best Paper Award)

• P. Biermann, & B. Jung: Variant Design in Immersive Virtual Reality: A Markup Language for Scalable CSG Parts. Articulated Motion and Deformable Objects, Proceedings AMDO-2004, Palma de Mallorca, Spain, Springer (LNCS 3179), 123-133.

• P. Biermann, & I. Wachsmuth: Non-Physical Simulation of Gears and Modifiable Connections in Virtual Reality. Proceedings 6th Virtual Reality International Conference (IEEE VRIC 2004), Laval, France, 159-164.

• T. Pfeiffer, & M. E. Latoschik: Resolving Object References in Multimodal Dialogues for Immersive Virtual Environments. Proceedings of the IEEE VR2004, Chicago, USA, March 2004, 35-42.

• M. Weber & B. Jung: Implementierung kollaborativer Mixed-Reality-Systeme: Ein Vergleich von Open-Source Videokonferenz-Software. In: Virtuelle und Erweiterte Realität - 1. Workshop der GI-Fachgruppe AR/VR. Shaker-Verlag, 2004, 173-180.

• P. Biermann & I. Wachsmuth: An Implemented Approach for a Visual Programming Environment in VR. In Proceedings Fifth Virtual Reality International Conference (VRIC 2003), Laval, France, 14-16 May 2003, 229-234.

• B. Jung. Task-Level Assembly Modeling in Virtual Environments. In Computational Science and Its Applications - ICCSA 2003, International Conference, Montreal, Canada, May 18-21, 2003. Proceedings, Part III. LNCS 2669. Springer. 2003. 721-730.

• M.E. Latoschik, M. Schilling: Incorporating VR Databases into AI Knowledge Representations: A Framework for Intelligent Graphics Applications. Proceedings of the 6th IASTED International Conference on Computer Graphics and Imaging. Honolulu, Hawaii, 2003.

• P. Biermann, B. Jung, M. Latoschik & I. Wachsmuth: Virtuelle Werkstatt: A Platform for Multimodal Assembly in VR. In Proceedings Fourth Virtual Reality International Conference (VRIC 2002), Laval, France, 19-21 June 2002, 53-62.

• B. Jung, M. Latoschik, P. Biermann & I. Wachsmuth: Virtuelle Werkstatt. In 1. Paderborner Workshop Augmented Reality / Virtual Reality in der Produktentstehung (pp. 185-196), Paderborn: HNI, 2002.

• M. E. Latoschik: Designing Transition Networks for Multimodal VR-Interactions Using a Markup Language. In Proceedings of the IEEE fourth International Conference on Multimodal Interfaces, ICMI 2002, Pittsburgh, USA, October 2002, 411-416.

• M. E. Latoschik: A Gesture Processing Framework for Multimodal Interaction in Virtual Reality. ACM AFRIGRAPH 2001, 1st Internat. Conference on Computer Graphics, Virtual Reality and Visualization in Africa, 95-100.

Supported by the Deutsche Forschungsgemeinschaft (DFG) 2001-2004; 2006-2007.