UniBi Logo Bielefeld University - Faculty of Technology - AI Group - Home

QuickSearch:   Number of matching entries: 0.


Abstract: This paper introduces a semantic representation for virtual prototyping in interactive virtual construction applications. The representation reflects semantic information about dynamic constraints to define objects’

modification and construction behavior as well as knowledge structures supporting multimodal interaction utilizing speech and gesture. It is conveniently defined using XML-based markup for virtual building parts. The semantic information is processed during runtime in two ways: Constraint graphs are mapped to a generalized data-flow network and scene-graph. Interaction knowledge is accessed and matched during multimodal analysis.

BibTeX:

@inproceedings{biermann:2007:Semantic,

  author = {Peter Biermann and Christian Fr{\"o}hlich and Marc Erich Latoschik and Ipke Wachsmuth},

  title = {{S}emantic {I}nformation and {L}ocal {C}onstraints for {P}arametric {P}arts in {I}nteractive {V}irtual {C}onstruction},

  booktitle = {Proceedings of the 8th International Symposium on Smart Graphics 2007, SG2007},

  address = {Kyoto, Japan},

  month = {june},

  year = {2007},

  pages = {124-134},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/semantic-info-and-constraints-SG07.pdf}

}

Abstract: Für die Mensch-Maschine-Interaktion ist die Erfassung der Aufmerksamkeit des Benutzers von großem Interesse. Für Anwendungen in der Virtuellen Realität (VR) gilt dies insbesondere, nicht zuletzt dann, wenn Virtuelle Agenten als Benutzerschnittstelle eingesetzt werden. Aktuelle Ansätze zur Bestimmung der visuellen Aufmerksamkeit verwenden meist monokulare Eyetracker und daher auch nur zweidimensionale bedeutungstragende Blickfixationen relativ zu einer Projektionsebene. Für typische Stereoskopie-basierte VR Anwendungen ist aber eine zusätzliche Berücksichtigung der Fixationstiefe notwendig, um so den Tiefenparameter für die Interaktion nutzbar zu machen, etwa für eine höhere Genauigkeit bei der Objektauswahl (Picking). Das in diesem Beitrag vorgestellte Experiment zeigt, dass bereits mit einem einfacheren binokularen Gerät leichter zwischen sich teilweise verdeckenden Objekten unterschieden werden kann. Trotz des positiven Ergebnisses kann jedoch noch keine uneingeschränkte Verbesserung der Selektionsleistung gezeigt werden. Der Beitrag schließt mit einer Diskussion weiterer Schritte mit dem Ziel, die vorgestellte Technik weiter zu verbessern.
BibTeX:

@inproceedings{donner:2007:tiefenbestimmung,

  author = {Matthias Donner and Thies Pfeiffer and Marc Erich Latoschik and Ipke Wachsmuth},

  title = {{B}lickfixationstiefe in stereoskopischen {VR}-{U}mgebungen: {E}ine vergleichende {S}tudie},

  booktitle = {Virtuelle und Erweiterte Realit{\"a}t, 4. Workshop of the GI special interest group VR/AR},

  year = {2007},

  pages = {113-124},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Eyetracking-VRAR07.pdf}

}

Abstract: In diesem Artikel wird die Anbindung eines low-cost Eingabegerätes für Virtuelle Umgebungen auf Basis des Nintendo Wii Controllers vorgestellt. Mit der WiiMote und dem Nunchuk ist es möglich, Beschleunigungen beider Hände zu erfassen. Durch Vibrieren und die integrierten LEDs stehen zudem Feedbackkanäle zur Verfügung. Dieser Artikel befasst sich mit den Möglichkeiten und Grenzen des Controllers. Er stellt eine Schnittstelle vor, mit der es möglich ist, den Wii-Controller sowohl an Einzelplatz- als auch in VR/AR-Systemen (z.B. CAVEs) effizient zu nutzen. Auch werden verschiedene Anwendungsbeispiele für die sinnvolle Nutzung vorgestellt.
BibTeX:

@inproceedings{hammerl:2007:WiiNC,

  author = {Sebastian Hammerl and Tim Preuss and Marc Erich Latoschik},

  title = {{WiiNC} - {Wii} {N}etwork {C}ontrol - {E}insatz des {W}ii-{C}ontrollers f{\"u}r {VR}-{A}nwendungen},

  booktitle = {Virtuelle und Erweiterte Realit{\"a}t, 4. Workshop of the GI special interest group VR/AR},

  year = {2007},

  pages = {141-148},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/WiiNC-GIVRAR07.pdf}

}

Abstract: Die Fachgruppe Virtuelle und Erweiterte Realität der Gesellschaft für Informatik wurde im Jahr 2003 als Informationsplattform und Interessengemeinschaft gegründet. Seit 2004 organisiert die Gruppe einen jährlich stattfindenden Workshop, der am 15. Juli 2007 an der Bauhaus-Universität Weimar zu Gast ist. Dieser Band enthält die 22 ausgewählten Beiträge. Das Themenspektrum reicht von Mensch-Maschine-Interation über Simulations-, Projektions- und Rendertechnologien zu VR/AR-Anwendungszenarien. Wir sind besonders froh über die aktive Beteiligung vieler Nachwuchsforscher und Studenten -- ein Indiz für die anhaltende Attraktivität und Aktualität der Zukunftstechnologien Virtual Reality und Augmented Reality. Besonderer Dank gilt dem Programmkomitee und den vielen Helfern bei der lokalen Organisation, ohne deren Unterstützung die Organisation und Durchführung dieser Veranstaltung nicht möglich gewesen wäre. Wir freuen uns auf eine interessante Veranstaltung mit lebhaften Diskussionen.
BibTeX:

@inproceedings{latoschik:ARVR:2007,

  author = {Marc Erich Latoschik and Bernd Fr{\"o}hlich},

  title = {Virtuelle und Erweiterte Realit{\"a}t, 4. Workshop of the GI special interest group VR/AR},

  publisher = {Shaker Verlag},

  year = {2007}

}

Abstract: This paper introduces semantic reflection, a novel concept for a modular design of intelligent applications. SCIVE, a simulation core for intelligent Virtual Environments (IVEs), provides semantic reflection on multiple layers: SCIVE’s architecture grants semantic driven uniform access to low-level simulation core logic, to specific simulation modules’ application definitions, as well as to high-level semantic environment descriptions. It additionally provides a frame to conveniently interconnect various simulation modules, e.g., for graphics, physics, audio, haptics, or AI etc. SCIVE’s Knowledge Representation Layer’s base formalism provides the central organizing structure for the diverse modules’ data representations. It allows bidirectional knowledge driven access between the modules since their specific data structures and functions are transitively reflected by the semantic layer. Hence SCIVE preserves, integrates and provides unified access to the development paradigms of the interconnected modules, e.g., scene graph metaphors or field route concepts etc. well known from todays Virtual Reality systems. SCIVE’s semantic reflection implementation details are illustrated following a complex example application. We illustrate how semantic reflection and modularity support extensibility and maintainability of VR applications, potential for automatic system configuration and optimization, as well as the base for comprehensive knowledge driven access for IVEs.
BibTeX:

@inproceedings{latoschik:2007:intelligentVR,

  author = {Marc Erich Latoschik and Christian Fr{\"o}hlich},

  title = {Towards Intelligent {VR}: Multi-Layered Semantic Reflection for {I}ntelligent {V}irtual {E}nvironments},

  booktitle = {Proceedings of the Graphics and Applications GRAPP 2007},

  address = {Barcelona, Spain},

  year = {2007},

  pages = {249-259},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Towards-IVR-grapp07-latoschik.pdf}

}

Abstract: We introduce semantic reflection as an architectural concept for Intelligent Virtual Environments (IVEs). SCIVE, a dedicated IVE simulation core, combines modularity with close coupled integrative aspects to provide semantic reflection on multiple layers from low-level simulation core logic, specific simulation modules’ appli-

cation definitions, to high-level semantic environment descriptions. SCIVE’s Knowledge Representation Layer provides the central organizing structure which ties together data representations of simulation modules, e.g., for graphics, physics, audio, haptics, or AI etc., while it additionally allows bidirectional knowledge driven ac-

cess between the modules.

BibTeX:

@inproceedings{latoschik:2007:semantic_reflection,

  author = {Marc Erich Latoschik and Christian Fr{\"o}hlich},

  title = {{S}emantic {R}eflection for {I}ntelligent {V}irtual {E}nvironments},

  booktitle = {Proceedings of the IEEE VR 2007},

  address = {North Carolina, USA},

  year = {2007},

  pages = {305-306},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/semantic-reflection-VR07.pdf}

}

Abstract: The mediation of social presence is one of the most interesting challenges of modern communication technology. The proposed metaphor of Interactive Social Displays describes new ways of interactions with multi-/crossmodal interfaces prepared for a psychologically augmented communication. A first prototype demonstrates the application of this metaphor in a teleconferencing scenario.
BibTeX:

@inproceedings{pfeiffer:2007:InteractiveSDs,

  author = {Thies Pfeiffer and Marc Erich Latoschik},

  title = {{I}nteractive {S}ocial {D}isplays},

  booktitle = {13th Eurographics Symposium on Virtual Environments/10th Workshop on Immersive Projection Technology IPT-EGVE 2007},

  year = {2007},

  pages = {41-42},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/ISD-EGVE2007.pdf}

}

Abstract: The mediation of social presence is one of the most interesting challenges of modern communication technology. The proposed metaphor of Interactive Social Displays describes new ways of interactions with multi-/crossmodal interfaces prepared for a psychologically augmented communication. A first prototype demonstrates the application of this metaphor in a teleconferencing scenario.
BibTeX:

@inproceedings{pfeiffer:2007:ISD1,

  author = {Thies Pfeiffer and Marc Erich Latoschik},

  title = {{I}nteractive {S}ocial {D}isplays},

  booktitle = {Proceedings of the IEEE Symposium on 3D User Interfaces 2007},

  address = {North Carolina, USA},

  year = {2007},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/ISD-3DUI2007.pdf}

}

Abstract: This paper describes the configuration of a cost-efficient monolithic render server aimed at multi-screen Virtual Reality display devices. The system uses common Of-The-Shelf (OTS) PC components and feeds up to 6 independent screens via 3 graphics pipes with the potential to feed up to 12 screens. The internal graphics accelerators each use at least 8 PCIe lanes which results in sufficient bandwidth. Performance measurements are provided for several benchmarks which compare the system’s performance to well established network based render clusters.
BibTeX:

@inproceedings{rabe:2007:low-cost,

  author = {Felix Rabe and Christian Fr{\"o}hlich and Marc Erich Latoschik},

  title = {Low-Cost Image Generation for {I}mmersive {M}ulti-{S}creen {E}nvironments},

  booktitle = {Virtuelle und Erweiterte Realit{\"a}t, 4. Workshop of the GI special interest group VR/AR},

  year = {2007},

  pages = {65-76},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/low-cost-ig-VRAR07.pdf}

}

Abstract: This article illustrates the detection of 6 degrees of freedom (DOF) by utilizing a simple laser pointer device and a camera. The laser pointer device is augmented by a diffraction grating to project a unique laser grid onto the projection planes used in projection based immersive VR-setups. The distortion of the projected grid is then used to calculate the additional degrees of freedom as required for human-computer interaction purposes.
BibTeX:

@inproceedings{bomberg:2006:monoscopic,

  author = {Elmar Bomberg and Marc Erich Latoschik},

  title = {Monoscopic {6DOF} Detection using a Laser Pointer},

  booktitle = {Virtuelle und Erweiterte Realit{\"a}t, 3. Workshop of the GI special interest group VR/AR},

  year = {2006},

  pages = {143-154},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Mono6DofDetection-GIVRAR06.pdf}

}

Abstract: This paper introduces SCIVE, a Simulation Core for Intelligent Virtual Environments. SCIVE provides a Knowledge Representation Layer (KRL) as a central organizing structure. Based on a semantic net, it ties together the data representations of the various simulation modules, e.g., for graphics, physics, audio, haptics or Artificial Intelligence (AI) representations. SCIVE's open architecture allows a seamless integration and modification of these modules. Their data synchronization is widely customizable to support extensibility and maintainability. Synchronization can be controlled through filters which in turn can be instantiated and parametrized by any of the modules, e.g., the AI component can be used to change an object's behavior to be controlled by the physics instead of the interaction- or a keyframe-module. This bidirectional inter-module access is mapped by, and routed through, the KRL which semantically reflects all objects or entities the simulation comprises. Hence, SCIVE allows extensive application design and customization from low-level core logic, module configuration and flow control, to the simulated scene, all on a high-level unified representation layer while it supports well known development paradigms commonly found in Virtual Reality applications.
BibTeX:

@article{latoschik:2006:scenesync,

  author = {Marc Erich Latoschik and Christian Fr{\"o}hlich and Alexander Wendler},

  title = {{S}cene {S}ynchronization in {C}lose {C}oupled {W}orld {R}epresentations using {SCIVE}},

  journal = {{T}he {I}nternational {J}ournal of {V}irtual {R}eality},

  year = {2006},

  volume = {5},

  number = {3},

  pages = {47-52},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/SCIVE-IJVR06.pdf}

}

Abstract: Advanced VR simulation systems are composed of several components with independent and heterogeneously structured databases. To guarantee a closed and consistent world simulation, flexible and robust data exchange between these components has to be realized. This multiple database problem is well known in many distributed application domains, but it is central for VR setups composed of diverse simulation components. Particularly complicated is the exchange between object-centered and graph-based representation formats, where entity attributes may be distributed over the graph structure. This article presents an abstract declarative attribute representation concept, which handles different representation formats uniformly and enables automatic data exchange and synchronization between them. This mechanism is tailored to support the integration of a central knowledge component, which provides a uniform representation of the accumulated knowledge of the several simulation components involved. This component handles the incoming–possibly conflicting–world changes propagated by the diverse components. It becomes the central instance for process flow synchronization of several autonomous evaluation loops.
BibTeX:

@inproceedings{heumeretal:automaticdataexchange:VR2005,

  author = {Heumer, Guido and Schilling, Malte and Latoschik, Marc Erich},

  title = {Automatic Data Exchange and Synchronization for Knowledge-Based {I}ntelligent {V}irtual {E}nvironments},

  booktitle = {Proceedings of the IEEE VR2005},

  address = {Bonn, Germany},

  year = {2005},

  pages = {43-50},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Automatic_Data_Exchange.pdf}

}

Abstract: This article presents a User Interface (UI) framework for multimodal interactions targeted at immersive virtual environments. Its configurable input and gesture processing components provide an advanced behavior graph capable of routing continuous data streams asynchronously. The framework introduces a Knowledge Representation Layer which augments objects of the simulated environment with Semantic Entities as a central object model that bridges and interfaces Virtual Reality (VR) and Artificial Intelligence (AI) representations. Specialized node types use these facilities to implement required processing tasks like gesture detection, preprocessing of the visual scene for multimodal integration, or translation of movements into multimodally

initialized gestural interactions. A modified Augmented Transition Nettwork (ATN) approach accesses the knowledge layer as well as the preprocessing components to integrate linguistic, gestural, and context information in parallel. The overall framework emphasizes extensibility, adaptivity and reusability, e.g., by utilizing persistent and interchangeable XML-based formats to describe its processing stages.

BibTeX:

@inproceedings{latoschik:UIF:2005,

  author = {Latoschik, Marc Erich},

  title = {A User Interface Framework for Multimodal {VR} Interactions},

  booktitle = {Proceedings of the IEEE seventh International Conference on Multimodal Interfaces, ICMI 2005},

  address = {Trento, Italy},

  month = {October},

  year = {2005},

  pages = {76-83},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/pp217-latoschik.pdf}

}

Abstract: This article describes the integration of knowledge based techniques into simulative Virtual Reality (VR) applications. The approach is motivated using multimodal Virtual Construction as an example domain. An abstract Knowledge Representation Layer (KRL) is proposed which is expressive enough to define all necessary data for diverse simulation tasks and which additionally provides a base formalism for the integration of Artificial Intelligence (AI) representations. The KRL supports two different implementation methods. The first method uses XSLT processing to transform the external KRL format into the representation formats of the diverse target systems. The second method implements the KRL using a functionally extendable semantic network. The semantic net library is tailored for real time simulation systems where it interconnects the required simulation modules and establishes access to the knowledge representations inside the simulation loop. The KRL promotes a novel object model for simulated objects called Semantic Entities which provides a uniform access to the KRL and which allows extensive system modularization. The KRL approach is demonstrated in two simulation areas. First, a generalized scene graph representation is presented

which introduces an abstract definition and implementation of geometric node interrelations. It supports scene and application structures which can not be expressed using common scene hierarchies or field route concepts. Second, the KRL’s expressiveness is demonstrated in the design of multimodal interactions. Here, the KRL defines the knowledge particularly required during the semantic analysis of multimodal user utterances.

BibTeX:

@inproceedings{latoschik:2005:knowledge_in_loop,

  author = {Marc Erich Latoschik and Peter Biermann and Ipke Wachsmuth},

  title = {Knowledge in the Loop: Semantics Representation for Multimodal Simulative Environments},

  booktitle = {Proceedings of the 5th International Symposium on Smart Graphics 2005},

  address = {Frauenwoerth Cloister, near Munich, Germany},

  year = {2005},

  pages = {25-39},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/kitl-print-springer-final.pdf}

}

Abstract: This article describes an integration of knowledge based techniques into simulative Virtual Reality (VR) applications motivated using a virtual construction task. An abstract Knowledge Representation Layer (KRL) provides a base formalism for the integration of simulation semantics. The KRL approach is demonstrated using a generalized scene graph representation which introduces an abstract implementation of geometric node interrelations.
BibTeX:

@inproceedings{latoschik:2005:semantics_rep,

  author = {Marc Erich Latoschik and Peter Biermann and Ipke Wachsmuth},

  title = {High-level Semantics Representation for {I}ntelligent {S}imulative {E}nvironments},

  booktitle = {Proceedings of the IEEE VR2005},

  address = {Bonn, Germany},

  year = {2005},

  pages = {283-284},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/high_level_semantics_representation.pdf}

}

Abstract: This paper describes the underlying concepts and the technical implementation of a system for resolving multimodal references in Virtual Reality (VR). In this system the temporal and semantic relations intrinsic to referential utterances are expressed as a constraint satisfaction problem, where the propositional value of each referential unit during a multimodal dialogue updates incrementally the active set of constraints. As the system is based on findings of human cognition research it also regards, e.g., constraints implicitly assumed by human communicators. The implementation takes VR related real-time and immersive conditions into account and adapts its architecture to well known scene-graph based design patterns by introducing a so-called reference resolution engine. Regarding the conceptual work as well as regarding the implementation, special care has been taken to allow further refinements and modifications to the underlying resolving processes on a high level basis.
BibTeX:

@inproceedings{pfeiffer_latoschik:Resolving:VR2004,

  author = {Pfeiffer, Thies and Latoschik, Marc Erich},

  title = {Resolving {O}bject {R}eferences in multimodal {D}ialogues for {I}mmersive {V}irtual {E}nvironments},

  booktitle = {Proceedings of the IEEE Virtual Reality conference 2004},

  year = {2004},

  pages = {35-42},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Resolving_Object_References.pdf}

}

Abstract: Systeme der Virtuellen Realität (VR) stellen neue Herausforderungen an die Art und Weise der Systembedienung. Die Übertragung herkömmlicher 2D orientierter WIMP-Schnittstellen (Windows, Icon, Menue, Pointer) erweist sich hier häufig als ungeeignet. In diesem Zusammenhang verfolgt die vorgestellte Arbeit den Ansatz, VR-Interaktionen über natürliche menschliche Kommunikationsmöglichkeiten mit Gestik und Sprache zu realisieren. Zu diesem Ziel wurden Kernkomponenten entwickelt, welche in den Echtzeit-getriebenen Programmfluss von VR Systemen eingebettet sind und Funktionen für die Gestikerkennung und -analyse sowie die multimodale Integration und Auswertung bereitstellen. Diese Komponenten werden in einer Reihe von aktuellen Projekten etwa im Kontext der Virtuellen Konstruktion, der Deixisforschung oder zur Interaktion mit MAX, einem antropomorphen Gegenüber in virtuellen Umgebungen eingesetzt.
BibTeX:

@article{latoschik:2003:mmivr,

  author = {Marc Erich Latoschik},

  title = {{M}ultimodale {I}nteraktion in {V}irtueller {R}ealit{\"a}t am {B}eispiel der virtuellen {K}onstruktion},

  journal = {KI-K{\"u}nstliche Intelligenz: Embodied Conversational Agents},

  year = {2003},

  volume = {4},

  number = {03},

  pages = {37-38},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/mmivr-am-bsp-KI-ECA03.pdf}

}

Abstract: This article presents a framework for incorporating commonly used VR (Virtual Reality) databases for

graphics and physics simulation into an AI (Artificial Intelligence) knowledge base using a unifying

semantic net (SN) representation. Several examples in the area of multimodal interaction for AI based

graphics applications are given to motivate this approach. An evaluation of the chosen SN knowledge representation (KR) regarding five roles suitable to analyze a given KR is followed by a discussion about

resulting conceptual and technical requirements for the underlying DB/KBMS (database/knowledge base management system) which supports the chosen KR as well as mediating layers for external simulation relevant modules.

BibTeX:

@inproceedings{latoschik:Incorporating:CGIM03,

  author = {Latoschik, Marc Erich and Schilling, Malte},

  title = {Incorporating {VR} {D}atabases into {AI} {K}nowledge {R}epresentations: {A} {F}ramework for {I}ntelligent {G}raphics {A}pplications},

  booktitle = {Proceedings of the {S}ixth {I}nternational {C}onference on {C}omputer {G}raphics and {I}maging},

  publisher = {ACTA {P}ress},

  year = {2003},

  pages = {79-84},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/incorporating_VR_into_AI.pdf}

}

Abstract: In the Collaborative Research Center (SFB 360) at the University of Bielefeld we are concerned wih situated

artificial communicators. In one application scenario the user is involved in a task-oriented discourse with

the embodied conversational agent MAX. He guides him through an assembly process by means of task descriptions uttered using both German natural language and gestures. The work presented is on a system for identifying referent objects for deictic references in a real-time Virtual Reality (VR) Environment.

BibTeX:

@inproceedings{pfeiffer_etal:Resolution:EuroCogSci03,

  author = {Thies Pfeiffer and Ian Voss and Marc Erich Latoschik},

  title = {Resolution of Multimodal Object References using Conceptual Short Term Memory},

  booktitle = {Proceedings of the EuroCogSci03},

  publisher = {Lawrence Erlbaum Associates Inc},

  year = {2003},

  pages = {462},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Resolution_of_Multimodal_Object_References.pdf}

}

Abstract: In this paper we describe ongoing research that aims at the development of a generic demonstration platform for virtual prototype modeling by utilizing multimodal speech and gesture interactions in Virtual Reality. Particularly, we concentrate on two aspects. First, a knowledge-based approach for assembling CAD-based parts in VR is introduced. This includes a system to generate meta-information from geometric models

as well as accompanying task-level algorithms for virtual assembly. Second, a framework for modeling multimodal interaction using gesture and speech is presented that facilitates its generic adaptation to

scene-graph-based applications. The chosen decomposition of the required core modules is exemplified by an example of a typical object rotation interaction.

BibTeX:

@inproceedings{biermann_etal:2002:VWerk,

  author = {Peter Biermann and Bernhard Jung and Marc Erich Latoschik and Ipke Wachsmuth},

  title = {{V}irtuelle {W}erkstatt: A Platform for Multimodal Assembly in {VR}},

  booktitle = {Proceedings Fourth Virtual Reality International Conference (VRIC 2002), Laval, France},

  month = {jun},

  year = {2002},

  pages = {53-62},

  url = {http://www.techfak.uni-bielefeld.de/~ipke/download/VWerkDruck.pdf}

}

Abstract: Das Projekt "Virtuelle Werkstatt" will Forschungsarbeiten aus den Bereichen Multimodale Interaktion und Virtuelles Konstruieren erweitern und derart zusammenführen, daß ihre realitätsnahe Erprobung in der Virtuellen Realität (VR) demonstrierbar wird. Multimodale Interaktion betrifft die unmittelbare Umsetzung

von Benutzereingriffen in einer visualisierten 3D-Szene aufgrund von sprachbegleiteten Gesteneingaben. Virtuelles Konstruieren betrifft die Erstellung und Erprobung computergraphisch visualisierter 3D-Modelle geplanter mechanischer Konstruktionen (sog. virtueller Prototypen), um eine realistische Vorabexploration

von Entwürfen per Simulation in der Virtuellen Realität zu ermöglichen. Der Einsatz eines Cave-artigen VR-Großdisplays macht hierbei gleichzeitig Benutzerinteraktionen mit sprachbegleiteten Gesteneingaben im Greifraum wie auch im Fernraum erforschbar.

BibTeX:

@inproceedings{jung_etal:2002:VWerk,

  author = {Bernhard Jung and Marc Erich Latoschik and Peter Biermann and Ipke Wachsmuth},

  title = {{V}irtuelle {W}erkstatt},

  booktitle = {1. Paderborner Workshop Augmented Reality / Virtual Reality in der Produktentstehung},

  address = {Paderborn},

  publisher = {HNI},

  year = {2002},

  pages = {185-196},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/jlbw02.pdf}

}

Abstract: This article presents one core component for enabling multimodal-speech and gesture-driven interaction in and for Virtual Environments. A so-called temporal Augmented Transition Network (tATN) is introduced. It allows to integrate and evaluate information from speech, gesture, and a given application context using a combined syntactic/semantic parse approach. This tATN represents the target structure for a multimodal integration markup language (MIML). MIML centers around the specification of multimodal interactions by letting an application designer declare temporal and semantic relations between given input utterance percepts and certain application states in a declarative and portable manner. A subsequent parse pass translates MIML into corresponding tATNs which are directly loaded and executed by a simulation engines scripting facility.
BibTeX:

@inproceedings{latoschik:Designing:ICMI02,

  author = {Marc Erich Latoschik},

  title = {Designing {T}ransition {N}etworks for {M}ultimodal {VR}-{I}nteractions {U}sing a {M}arkup {L}anguage},

  booktitle = {Proceedings of the {F}ourth {IEEE} {I}nternational {C}onference on {M}ultimodal {I}nterfaces {ICMI}'02, Pittsburgh, Pennsylvania},

  publisher = {IEEE},

  year = {2002},

  pages = {411-416},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/dtn_for_multimodal.pdf}

}

Abstract: This article presents a modular approach to incorporate multimodal gesture and speech driven interaction into virtual reality systems. Based on existing techniques for modelling VR-applications, the overall task is separated into different problem categories: from sensor synchronisation to a high-level description of crossmodal temporal and semantic coherences, a set of solution concepts is presented that seamlessly fit into both the static (scenegraph-based) representation and into the dynamic (renderloop and immersion) aspects of a realtime application. The developed framework establishes a connecting layer between raw sensor data and a general functional description of multimodal and scenecontext related evaluation procedures for VR-setups.

As an example for the concepts, their implementation in a system for virtual construction is described.

BibTeX:

@inproceedings{lat:PrOSA-framework,

  author = {Marc Erich Latoschik},

  title = {A General Framework for Multimodal Interaction in {V}irtual {R}eality Systems: Pr{OSA}},

  booktitle = {The Future of VR and AR Interfaces - Multimodal, Humanoid, Adaptive and Intelligent. Proceedings of the Workshop at IEEE Virtual Reality 2001},

  year = {2001},

  series = {GMD report},

  number = {138},

  pages = {21-25},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/A_General_Framework_for_Multimodal.pdf}

}

Abstract: This article presents a gesture detection and analysis framework for modelling multimodal interactions. It is particulary designed for its use in Virtual Reality (VR) applications and contains an abstraction layer for different sensor hardware. Using the framework, gestures are described by their characteristic spatio-temporal features which are on the lowest level calculated by simple predefined detector modules or nodes. These nodes can be connected by a data routing mechanism to perform more elaborate evaluation functions, therewith establishing complex detector nets. Typical problems that arise from the time-dependent invalidation of multimodal utterances under immersive conditions lead to the development of pre-evaluation concepts that as well support their integration into scene graph based systems to support traversal-type access. Examples of realized interactions illustrate applications which make use of the described concepts.
BibTeX:

@inproceedings{latoschik:gestureprocessing:01,

  author = {Marc Erich Latoschik},

  title = {A Gesture Processing Framework for Multimodal Interaction in {V}irtual {R}eality},

  booktitle = {Proceedings of the 1st International Conference on Computer Graphics, Virtual Reality and Visualisation in Africa, AFRIGRAPH 2001},

  publisher = {ACM SIG{\-}GRAPH},

  year = {2001},

  pages = {95-100},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/A_gesture_processing_framework.pdf}

}

Abstract: Virtuelle Realität (VR) stellt durch die Abkehr von Bildschirm- und Desktop-zentrierten Metaphern einen Umbruch in der Multimedia- technik dar. Die Systemhandhabung mit herkömmlichen Eingabe- geräten bei Erhalt der Bewegungsfreiheit und der Einbettung des Benutzers wird allerdings zusehends unnatürlich und erfordert ein Überdenken bisheriger Bedienungskonzepte. Im vorliegenden Band wird ein Ansatz ausgearbeitet, der neuartige - multimodale - Interaktionsformen in virtuellen Umgebungen er- möglicht. Dafür werden modulare Komponenten für eine Gestener- kennung (PrOSA - Patterns on Sequences of Attributes) und für ihre multimodale Integration mit Sprache bereitgestellt (erweiterter ATN-Formalismus). Besonderes Augenmerk gilt der Übertragbar- keit auf heutige high-end VR-Systeme.
BibTeX:

@phdthesis{latoschik:MIVR,

  author = {Latoschik, Marc Erich},

  title = {Multimodale {I}nteraktion in {V}irtueller {R}ealit\"{a}t am {B}eispiel der virtuellen {K}onstruktion},

  publisher = {infix, Berlin},

  school = {Technische Fakult\"{a}t, Universit\"{a}t Bielefeld},

  year = {2001},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/l01MMIVR.pdf}

}

Abstract: This paper presents work on multimodal communication with an anthropomorphic agent. It focuses on processing of multimodal input and output employing natural language and gestures in virtual environments. On the input side, we describe our approach to recognize and interpret co-verbal gestures used for pointing, object manipulation, and object description. On the output side, we present the utterance generation module of the agent which is able to produce coordinated speech and gestures.
BibTeX:

@inproceedings{sow:a_communicative_mediator01,

  author = {Sowa, T. and Kopp, S. and Latoschik, M. E.},

  title = {A Communicative Mediator in a {V}irtual {E}nvironment: Processing of Multimodal Input and Output},

  booktitle = {Proceedings of the International Workshop on Information Presentation and Natural Multimodal Dialogue},

  year = {2001},

  pages = {71-74},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Verona.pdf}

}

Abstract: Virtuelle Realität oder Virtual Reality (VR) bezeichnet ein neuartiges Kommunikationsmedium, das die

unmittelbare Wechselwirkung des Menschen mit räumlich organisierten rechnergenerierten Darstellungen

erlaubt. Verbunden mit körperlich verankerter Interaktion finden insbesondere gestische Eingaben starkes

Interesse. Dieser Beitrag gibt einen Überblick über Forschungsarbeiten im Labor für Künstliche Intelligenz

und Virtuelle Realität an der Universität Bielefeld, mit denen Grundlagen für den Einsatz gestischer und

sprachlicher Interaktionstechniken entwickelt werden; als Erprobungsdomäne dient ein Szenario des virtuellen

Konstruierens. Für die schnelle Erfassung komplexer Hand-Armgesten werden derzeit Datenhandschuhe und

Körper-Tracker eingesetzt. Die Auswertung erfolgt mit wissensbasierten Ansätzen, die atomare Formelemente

der Gestik symbolisch beschreiben und zu größeren Einheiten zusammensetzen. Ein zweites Thema ist die

multimodale Interaktion durch sprachlich-gestische Eingaben, z.B. wenn auf einen Gegenstand gezeigt

("dieses Rohr") oder eine Drehrichtung ("so herum") signalisiert wird. Schließlich wird dargestellt, wie die

Ansätze zur Formbeschreibung von Gesten für die Synthese natürlich wirkender gestischer Ausgaben mit

einer artikulierten, anthropomorphen Figur übertragen werden können, die in laufenden Arbeiten mit

Sprachausgaben koordiniert werden.

BibTeX:

@inproceedings{wachsmuth:MMIVR,

  author = {Wachsmuth, Ipke and Voss, Ian and Sowa, Timo and Latoschik, Marc Erich and Kopp, Stefan and Jung, Bernhard},

  title = {Multimodale {I}nteraktion in der {V}irtuellen {R}ealit\"{a}t},

  booktitle = {Mensch \& {C}omputer 2001},

  publisher = {B.G. Teubner Stuttgart},

  year = {2001},

  series = {Berichte des German chapter of the ACM},

  number = {55},

  pages = {265-274},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/wachsmuth_mc2001.pdf}

}

Abstract: Im Bielefelder Labor für Künstliche Intelligenz und Virtuelle Realität liegt der Forschungsschwerpunkt auf der Integration von gestischer und sprachlicher Kommunikation in einem Szenario des Virtuellen Konstruierens. Dabei werden hochaufgelöste räumliche Visualisierungen CAD-basierter Bauteilmodelle in realistischer Größe auf einer Großbildleinwand präsentiert und über Eingabegeräte der Virtuellen Realität (Datenhandschuhe, Positionssensoren, Spracherkennungssystem) zu komplexen Aggregaten zusammengebaut. Wissensbasierte Techniken kommen dabei einerseits bei der Montagesimulation mit den computergraphischen Bauteilmodellen und andererseits bei der Auswertung der sprachlich-gestischen Eingaben zum Einsatz.
BibTeX:

@article{jun:VirtuellesKonstruieren,

  author = {Jung, Bernhard and Kopp, Stefan and Latoschik, Marc and Sowa, Timo and Wachsmuth, Ipke},

  title = {{V}irtuelles {K}onstruieren mit {G}estik und {S}prache},

  journal = {K\"unstliche {I}ntelligenz},

  publisher = {aren{DT}a{P} Verlag, Bremen},

  year = {2000},

  volume = {2/00},

  pages = {5-11},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Virtuelles-Konstruieren-KI00.pdf}

}

Abstract: Dieser Beitrag stellt ein System für die sprachlich-gestische Interaktion zur Steuerung eines Systems zur Virtuellen Konstruktion vor. Eine Übersicht über verschiedene Manipulationsaufgaben in dieser Domäne dient als Grundlage, um Interaktionsbeispiele zu erläutern. Neben deiktischen Gesten des Benutzers werden mimetische Gesten, die gewünschte Veränderungen "vormachen", betrachtet. Diese werden durch sprachliche oder gestische Trigger eingeleitet und bewirken eine Anpassung in den Funktionsmodi der Auswertung, wobei zwischen diskreten und kontinuierlichen Interaktionen unterschieden wird. Um kontinuierliche Modifikationen in der virtuellen Szene umzusetzen, werden neben dem Konzept der Manipulatoren sogenannte Aktuatoren als Repräsentanten für Benutzermodalitäten sowie Motion-Modifikatoren zur Korrektur unscharfer Sensor-Eingaben eingeführt.
BibTeX:

@inproceedings{lat:MIS,

  author = {Latoschik, Marc Erich and Jung, Bernhard and Wachsmuth, Ipke},

  title = {{M}ultimodale {I}nteraktion mit einem {S}ystem zur {V}irtuellen {K}onstruktion},

  booktitle = {Proceedings der 29. Jahrestagung der Gesellschaft f\"{u}r Informatik - Informatik'99, Informatik \"{u}berwindet Grenzen},

  address = {Berlin Heidelberg New York},

  publisher = {Springer-Verlag},

  year = {1999},

  pages = {88-97},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/mis_konstruktion.pdf}

}

BibTeX:

@inproceedings{lat:SGI,

  author = {Latoschik, Marc Erich and Wachsmuth, Ipke},

  title = {{S}prachgest\"{u}tzte gestische {I}nteraktion zur {S}teuerung {V}irtueller {K}onstruktion},

  booktitle = {Tagungsband zum Workshop des Forschungsverbundes NRW -- Die Virtuelle Wissensfabrik -- vom 23./24. September 1999},

  address = {Schlo\ss{} Birlinghoven, St. Augustin},

  publisher = {GMD},

  year = {1999}

}

Abstract: This paper presents a technical approach for temporal symbol integration aimed to be generally applicable in unimodal and multimodal user interfaces. It draws its strength from symbolic data representation and an underlying rulebased system, and is embedded in a multi-agent system. The core method for temporal integration is motivated by findings from cognitive science research. We discuss its application for a gesture recognition task and speech-gesture integration in a Virtual Construction scenario. Finally an outlook of an empirical evaluation is given.
BibTeX:

@inproceedings{sow:TSI,

  author = {Sowa, Timo and Fr\"{o}hlich, Martin and Latoschik, Marc Erich},

  title = {Temporal Symbolic Integration Applied to a Multimodal System Using Gestures and Speech},

  booktitle = {Gesture-{B}ased {C}ommunication in {H}uman-{C}omputer {I}nteraction - Proceedings of the International Gesture Workshop (Gif-sur-Yvette, France, March 1999)},

  address = {Berlin Heidelberg New York},

  publisher = {Springer-Verlag},

  year = {1999},

  series = {Lecture Notes in Artificial Intelligence},

  pages = {291-302},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/Temporal-GW99.pdf}

}

Abstract: The idea of Virtual Prototyping is the use of realistic digital product models for design and functionality analysis in early stages of the product development cycle. The goal of our research is to make modeling of virtual prototypes more intuitive and powerful by using knowledge enhanced Virtual Reality techniques for interactive construction of virtual prototypes from 3D-visualized, CAD-based parts. To this end, a knowledge-based approach for real-time assembly simulation has been developed that draws on dynamically updated representations of part matings and assembly structure. The approach has been implemented in an experimental system, the CODY Virtual Constructor, that supports a variety of interaction modalities, such as direct manipulation, natural language, and gestures.
BibTeX:

@inproceedings{jun:KBA,

  author = {Jung, Bernhard and Latoschik, Marc Erich and Wachsmuth, Ipke},

  title = {Knowledge-Based Assembly Simulation for Virtual Prototype Modeling},

  booktitle = {IECON'98: Proceedings of the 24th annual Conference of the IEEE Industrial Electronics Society},

  year = {1998},

  volume = {4},

  pages = {2152-2157},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/KBbased-assembly-IECON98.pdf}

}

Abstract: Virtual environments are a new means for human-computer interaction. Whereas techniques for visual presentation have reached a high level of maturity in recent years, many of the input devices and interaction techniques still tend to be awkward for this new media. Where the borders between real and artificial environments vanish, a more natural way of interaction is desirable. To this end, we investigate the benefits of integrated speech- and gesture-based interfaces for interacting with virtual environments. Our research results are applied within a virtual construction scenario, where 3D visualized mechanical objects can be spatially rearranged and assembled using speech- and gesture-based communication.
BibTeX:

@inproceedings{lat:USG,

  author = {Latoschik, Marc Erich and Fr\"{o}hlich, Martin and Jung, Bernhard and Wachsmuth, Ipke},

  title = {Utilize Speech and Gestures to Realize Natural Interaction in a {V}irtual {E}nvironment},

  booktitle = {IECON'98: Proceedings of the 24th annual Conference of the IEEE Industrial Electronics Society},

  year = {1998},

  volume = {4},

  pages = {2028-2033},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/usg_to_realize.pdf}

}

BibTeX:

@incollection{lat:sprachbegleitete,

  author = {Latoschik, Marc Erich and Wachsmuth, Ipke},

  title = {Sprachbegleitete {K}{{\"o}}rper-{G}estik vor multimedialen {G}ro{{\ss}}displays},

  booktitle = {Forschung an der Universit{{\"a}}t Bielefeld},

  publisher = {Universit\"{a}t {B}ielefeld, {I}nformations- und {P}ressestelle},

  year = {1998},

  volume = {17},

  pages = {7-9}

}

Abstract: Developing state of the art multimedia applications nowadays calls for the use of sophisticated visualisation and immersion techniques, commonly referenced as Virtual Reality. While Virtual Reality meanwhile reaches good results both in image quality and in fast user feedback using parallel computation techniques, the methods for interacting with these systems need to be improved. In this paper we introduce a multimedia application that uses a gesture-driven interface and, secondly, the architecture for an expandable gesture recognition system. After different gesture types for interaction in a virtual environment are discussed with respect to a required functionality, the implementation of a specific gesture detection module for distant pointing recognition is described, and the whole system design is tested for its task adequacy.
BibTeX:

@inproceedings{lat:EDP,

  author = {Latoschik, Marc Erich and Wachsmuth, Ipke},

  title = {Exploiting Distant Pointing Gestures for Object Selection in a {V}irtual {E}nvironment},

  booktitle = {Gesture and Sign Language in Human-Computer Interaction},

  publisher = {Springer-Verlag},

  year = {1997},

  series = {Lecture Notes in Artificial Intelligence},

  volume = {1371},

  pages = {185-196},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/exploiting_distant_pointing_gestures.pdf}

}

Abstract: In the VIENA Project ("Virtual Environments and Agents") we develop easy-to-use virtual environments for interactive design and exploration. We have modeled and implemented a synthetic human-like agent, Hamilton, that inhabits a simulated office environment and acts as an embodied virtual interface agent (VIA). To explore or change the simulated environment, people can instruct Hamilton by way of verbal input and simple hand gestures. Hamilton has a variety of functionalities which are put in effect by its agency, a multi-agent system. In mediating an instruction, invisible agents track exact object locations and colorings, and they negotiate alternative ways of acting. Hamilton's agency is also able to adapt to individual users' preferences during run time. As the VIA is present in the synthetic scene, users can take advantage of its anthropomorphic features, and they can choose to communicate with the agent from an external or an immersed view.
BibTeX:

@inproceedings{wac:avirtualinterface,

  author = {Wachsmuth, Ipke and Lenzmann, Britta and J{{\"o}}rding, Tanja and Jung, Bernhard and Latoschik, Marc and Fr{{\"o}}hlich, Martin},

  title = {A {V}irtual {I}nterface {A}gent und its Agency},

  booktitle = {Proceedings of the First International Conference on Autonomous Agents},

  year = {1997},

  pages = {516-517},

  url = {http://www.techfak.uni-bielefeld.de/~marcl/publications/download/A_Virtual_Interface_Agent.pdf}

}

Created by JabRef on 03/08/2007.