|
Pfeiffer, T., Pfeiffer, J. & Meißner, M. (2013). Mobile Recommendation Agents Making Online Use of Visual Attention Information at the Point of Sale. In Proceedings of the NeuroIS. |
| Abstract: We aim to utilize online information about visual attention for developing mobile recommendation agents (RAs) for use at the point of sale. Up to now, most RAs are focussed exclusively at personalization in an e-commerce setting. Very little is known, however, about mobile RAs that offer information and assistance at the point of sale based on individual-level feature based preference models (Murray and Häubl 2009). Current attempts provide information about products at the point of sale by manually scanning barcodes or using RFID (Kowatsch et al. 2011, Heijden 2005), e.g. using specific apps for smartphones. We argue that an online access to the current visual attention of the user offers a much larger potential. Integrating mobile eye tracking into ordinary glasses would yield a direct benefit of applying neuroscience methods in the user’s everyday life. First, learning from consumers’ attentional processes over time and adapting recommendations based on this learning allows us to provide very accurate and relevant recommendations, potentially increasing the perceived usefulness. Second, our proposed system needs little explicit user input (no scanning or navigation on screen) making it easy to use. Thus, instead of learning from click behaviour and past customer ratings, as it is the case in the e-commerce setting, the mobile RA learns from eye movements by participating online in every day decision processes. We argue that mobile RAs should be built based on current research in human judgment and decision making (Murray et al. 2010). In our project, we therefore follow a two-step approach: In the empirical basic research stream, we aim to understand the user’s interaction with the product shelf: the actions and patterns of user’s behaviour (eye movements, gestures, approaching a product closer) and their correspondence to the user’s informational needs. In the empirical system development stream, we create prototypes of mobile RAs and test experimentally the factors that influence the user’s adoption. For example, we suggest that a user’s involvement in the process, such as a need for exact nutritional information or for assistance (e.g., reading support for elderly) will influence the user’s intention to use such as system. The experiments are conducted both in our immersive virtual reality supermarket presented in a CAVE, where we can also easily display information to the user and track the eye movement in great accuracy, as well as in real-world supermarkets (see Figure 1), so that the findings can be better generalized to natural decision situations (Gidlöf et al. 2013). In a first pilot study with five randomly chosen participants in a supermarket, we evaluated which sort of mobile RAs consumers favour in order to get a first impression of the user’s acceptance of the technology. Figure 1 shows an excerpt of one consumer’s eye movements during a decision process. First results show long eye cascades and short fixations on many products in situations where users are uncertain and in need for support. Furthermore, we find a surprising acceptance of the technology itself throughout all ages (23 – 61 years). At the same time, consumers express serious fear of being manipulated by such a technology. For that reason, they strongly prefer the information to be provided by trusted third party or shared with family members and friends (see also Murray and Häubl 2009). Our pilot will be followed by a larger field experiment in March in order to learn more about factors that influence the user’s acceptance as well as the eye movement patterns that reflect typical phases of decision processes and indicate the need for support by a RA. |
|
BibTeX:
@inproceedings{Pfeiffer2013b,
author = {Thies Pfeiffer and Jella Pfeiffer and Martin Meißner},
title = {Mobile Recommendation Agents Making Online Use of Visual Attention Information at the Point of Sale},
booktitle = {Proceedings of the NeuroIS},
year = {2013},
note = {accepted, to appear}
}
|
|
|
|
Pfeiffer, T. & Essig, K. (2013). Analysis of Eye Movements in Situated Natural Interactions. In Abstracts of the European Eye Movement Conference. |
| Abstract: Studying and analyzing eye movements in natural interaction situations in the field – as opposed to restrictive laboratory settings – is a dream coming true with recent developments in hard- and software. Areas of interest include situated communication, sports, economics or human factors. These new opportunities, however, also challenge current methodological approaches in basic research. Traditional eye-tracking parameters as indices for visual attention processes and established methods for scientific visualization, such as heatmaps or scanpaths, have to be carefully reconsidered when everything is dynamically in motion – as it is often the case in natural interaction situations. Many measures do not consider variations in applications fields, stimuli types, environments and multi-modal recordings, such as 3D object occlusions. The analysis of situated eye movements is even more challenging, as the flexibility gained with mobile eye tracking goes along with a significant increase of annotation time and complexity. Based on our own experiences drawn from various studies using mobile eye tracking, the talk discusses the (new) challenges and reviews state-of-the-art approaches for the analysis of eye movements in natural behavior, including manual and automatic annotations of situated eye movements, as well as combinations of eye tracking and motion capturing both in the field and lab. |
|
BibTeX:
@inproceedings{Pfeiffer2013a,
author = {Thies Pfeiffer and Kai Essig},
title = {Analysis of Eye Movements in Situated Natural Interactions},
booktitle = {Abstracts of the European Eye Movement Conference},
year = {2013},
note = {accepted, to appear}
}
|
|
|
|
Pfeiffer-Lessmann, N., Pfeiffer, T. & Wachsmuth, I. (2013). A Model of Joint Attention for Humans and Machines. In Abstracts of the European Eye Movement Conference. |
| Abstract: Joint attention is the simultaneous allocation of attention to a target as a consequence of attending to each other’s attentional states. It is an important prerequisite for successful interaction and supports the grounding of future actions. Cognitive modeling of joint attention in a virtual agent requires an operational model for behavior recognition and production. To this end, we created a declarative four-phase-model (initiate / respond / feedback / focus) of joint attention based on a literature review. We applied this model to gaze communication and implemented it in the cognitive architecture of our virtual agent Max. To substantiate the model regarding the natural timing of gaze behavior, we conducted a study on human-agent interactions in immersive virtual reality. The results show that participants preferred the agent to exhibit a timing behavior similar to their own. Building on these insights, we now aim at a process model of joint attention. We are interested in patterns of joint attention emerging in natural interactions. In the preliminary results of a human-human study, we find patterns of fixation targets and fixation durations that allow us to identify the four phases and infer the current state of joint attention. |
|
BibTeX:
@inproceedings{Pfeiffer-Lessmann2013,
author = {Nadine Pfeiffer-Lessmann and Thies Pfeiffer and Ipke Wachsmuth},
title = {A Model of Joint Attention for Humans and Machines},
booktitle = {Abstracts of the European Eye Movement Conference},
year = {2013},
note = {accepted, to appear}
}
|
|
|
|
Pfeiffer, T. (2013, in press). Visuelle Aufmerksamkeit in Virtueller und Erweiterter Realität: Integration und Nutzung im Szenengraphen. In 12. Paderborner Workshop Augmented and Virtual Reality in der Produktentstehung. Paderborn, Germany: Heinz Nixdorf Institut, Universität Paderborn. |
| Abstract: Die synthetische Stimulation der visuellen Wahrnehmung ist seit jeher im Fokus von Virtueller und Erweiterter Realität und die möglichst exakte Bestimmung der Nutzerperspektive auf die dreidimensionale Welt eine der Kernaufgaben. Bislang gibt es jedoch nur einige exemplarische Ansätze, in denen die Blickrichtung des Nutzers oder gar die Verteilung der visuellen Aufmerksamkeit im Raum genauer bestimmt wird. Macht man diese Informationen der Anwendungslogik verfügbar, könnten existierende Verfahren zur Visualisierung weiter optimiert und neue Verfahren entwickelt werden. Darüber hinaus erschließen sich damit Blicke als Interaktionsmodalität. Aufbauend auf langjährigen Erfahrungen mit der Blickinteraktion in der Virtuellen Realität beschreibt der Artikel Komponenten für einen Szenengraph, mit dem sich blickbasierte Interaktionen leicht und entlang gewohnter Prinzipien realisieren lassen. |
|
BibTeX:
@inproceedings{Pfeiffer2013,
author = {Thies Pfeiffer},
title = {Visuelle Aufmerksamkeit in Virtueller und Erweiterter Realität: Integration und Nutzung im Szenengraphen},
booktitle = {12. Paderborner Workshop Augmented and Virtual Reality in der Produktentstehung},
publisher = {Heinz Nixdorf Institut, Universität Paderborn},
year = {2013},
note = {in press}
}
|
|
|
|
Pfeiffer, T. (2013, in press). Documentation with data gloves. In Body-Language-Communication: An International Handbook on Multimodality in Human Interaction. Berlin, New York: Mouton de Gruyter. |
BibTeX:
@incollection{Pfeiffer:Documentationwithdata:2013,
author = {Thies Pfeiffer},
title = {Documentation with data gloves},
booktitle = {Body-Language-Communication: An International Handbook on Multimodality in Human Interaction},
publisher = {Mouton de Gruyter},
year = {2013},
note = {in press}
}
|
|
|
|
Pfeiffer, T. (2013, in press). Documentation with motion capture. In Body-Language-Communication: An International Handbook on Multimodality in Human Interaction. Berlin, New York: Mouton de Gruyter. |
BibTeX:
@incollection{Pfeiffer:Documentationwithmotion:2012,
author = {Thies Pfeiffer},
title = {Documentation with motion capture},
booktitle = {Body-Language-Communication: An International Handbook on Multimodality in Human Interaction},
publisher = {Mouton de Gruyter},
year = {2013},
note = {in press}
}
|
|
|
|
Lücking, A., Pfeiffer, T. & Rieser, H. (2013, in press). Pointing and Reference Reconsidered. International Journal of Corpus Linguistics. |
| Abstract: The semiotic function of co-verbal pointing is a much debated issue. We investigate the semantics and pragmatics of co-verbal pointing from a foundational perspective combining experiments, statistical investigation, computer simulation and theoretical modelling techniques in a novel manner. At the outset, an overview of approaches to pointing in various Cognitive Sciences is given. Using inter alia assumptions of Ludwig Wittgenstein, David Kaplan, and Donald Davidson, we establish semantic and pragmatic hypotheses concerning the role of pointing in multi-modal expressions. The semantic hypotheses say: Pointing uniquely picks out the object referred to. In contrast, the first pragmatic hypothesis maintains that the object pointed at is identified by a default inference: Since “picking out” implements the idea of an assignment or an interpretation function that is how we determine the difference between semantics and pragmatics. The second pragmatic hypothesis only attributes a global pragmatic function to pointing, namely, that of shifting the focus of attention, which can be viewed as a precondition for triggering the default inference. We evaluate the hypotheses with an annotated and rated corpus of object identification games set up in experiments in which body movement tracking techniques have been extensively used to generate precise pointing measurements. Discussion of hypotheses is based on two models to conceptualize pointing, a ‘pointing ray’ model and a ‘pointing cone’ model. Statistical investigation and computer simulations show that especially distal areas in the pointing domain falsify the semantic hypotheses. The pragmatic hypotheses win: reference involving pointing rests on a default inference which we specify using the empirical data. The results raise numerous problems for classical semantics-pragmatics interfaces brought to the fore in the penultimate section, where we argue for pre-semantic pragmatics as well as post-semantic Gricean pragmatics. Finally, we discuss how our empirical pointing model fits into non-classical (i.e., “situated”) semantics-pragmatics interfaces. It provides an account for certain reference conceptions held in Relevance Theory as well as in Situation Semantics. |
|
BibTeX:
@article{Luecking.etal:PointingandReference:2013,
author = {Andy Lücking and Thies Pfeiffer and Hannes Rieser},
title = {Pointing and Reference Reconsidered},
journal = {International Journal of Corpus Linguistics},
year = {2013},
note = {in press}
}
|
|
|
|
Pfeiffer, T. (2013, in press). Gaze-based Assistive Technologies. In Assistive Technologies and Computer Access for Motor Disabilities: IGI Global. |
BibTeX:
@incollection{,
author = {Thies Pfeiffer},
title = {Gaze-based Assistive Technologies},
booktitle = {Assistive Technologies and Computer Access for Motor Disabilities},
publisher = {IGI Global},
year = {2013},
note = {in press}
}
|
|
|
|
Kousidis, S., Pfeiffer, T., Malisz, Z., Wagner, P. & Schlangen, D. (2012). Evaluating a minimally invasive laboratory architecture for recording multimodal conversational data., 39-42. |
| Abstract: This paper presents ongoing work on the design, deployment and evaluation of a multimodal data acquisition architecture which utilises minimally invasive motion, head, eye and gaze tracking alongside high-quality audiovisual recording of human interactions. The different data streams are centrally collected and visualised at a single point and in real time by means of integration in a virtual reality (VR) environment. The overall aim of this endeavour is the implementation of a multimodal data acquisition facility for the purpose of studying non-verbal phenomena such as feedback gestures, hand and pointing gestures and multi-modal alignment. In the first part of this work that is described here, a series of tests were performed in order to evaluate the feasibility of tracking feedback head gestures using the proposed architecture. |
|
BibTeX:
@inproceedings{2510937,
author = {Kousidis, Spyridon and Pfeiffer, Thies and Malisz, Zofia and Wagner, Petra and Schlangen, David},
title = {Evaluating a minimally invasive laboratory architecture for recording multimodal conversational data.},
year = {2012},
pages = {39--42}
}
|
|
|
|
Kim, E.-S., Kim, J., Pfeiffer, T., Wachsmuth, I. & Zhang, B.-T. (2012). ‘Is this right?’ or ‘Is that wrong?’: Evidence from Dynamic Eye-Hand Movement in Decision Making [Abstract]. In Proceedings of the 34th Annual Meeting of the Cognitive Science Society, 2723, August. Austin, TX. |
| Abstract: Eye tracking and hand motion (or mouse) tracking are complementary techniques to study the dynamics underlying human cognition. Eye tracking provides information about attention, reasoning, mental imagery, but figuring out the dynamics of cognition is hard. On the other hand, hand movement reveals the hidden states of high-level cognition as a continuous trajectory, but the detailed process is difficult to infer. Here, we use both eye and hand tracking while the subject watches a video drama and plays a multimodal memory game (MMG), a memory recall task designed to investigate the mechanism of recalling the contents of dramas. Our experimental results show that eye tracking and mouse tacking provide complementary information on cognitive processes. In particular, we found that, when humans make difficult decisions, they tend to ask ’Is the distractor wrong?’, rather than ’Is the decision right?’. |
|
BibTeX:
@inproceedings{Kim.etal:Isthisright:2012,
author = {Eun-Sol Kim and Jiseob Kim and Thies Pfeiffer and Ipke Wachsmuth and Byoung-Tak Zhang},
title = {‘Is this right?’ or ‘Is that wrong?’: Evidence from Dynamic Eye-Hand Movement in Decision Making [Abstract]},
booktitle = {Proceedings of the 34th Annual Meeting of the Cognitive Science Society},
year = {2012},
pages = {2723}
}
|
|
|
|
Liguda, C. & Pfeiffer, T. (2012). Modeling Math Word Problems with Augmented Semantic Networks. In Natural Language Processing and Information Systems, 247-252: Springer. |
| Abstract: Modern computer-algebra programs are able to solve a wide range of mathematical calculations. However, they are not able to understand and solve math text problems in which the equation is described in terms of natural language instead of mathematical formulas. Interestingly, there are only few known approaches to solve math word problems algorithmically and most of employ models based on frames. To overcome problems with existing models, we propose a model based on augmented semantic networks to represent the mathematical structure behind word problems. This model is implemented in our Solver for Mathematical Text Problems (SoMaTePs) [1], where the math problem is extracted via natural language processing, transformed in mathematical equations and solved by a state-of-the-art computer-algebra program. SoMaTePs is able to understand and solve mathematical text problems from German primary school books and could be extended to other languages by exchanging the language model in the natural language processing module. | |
BibTeX:
@inproceedings{Liguda.Pfeiffer:ModelingMathWord:2012,
author = {Christian Liguda and Thies Pfeiffer},
title = {Modeling Math Word Problems with Augmented Semantic Networks},
booktitle = {Natural Language Processing and Information Systems},
publisher = {Springer},
year = {2012},
pages = {247-252},
url = {http://www.springerlink.com/content/k47u238x4r5402qv/}
}
|
|
|
|
Lücking, A. & Pfeiffer, T. (2012). Framing Multimodal Technical Communication. With Focal Points in Speech-Gesture-Integration and Gaze Recognition. In Handbook of Technical Communication, 8, 591-644. Berlin/Boston: Mouton de Gruyter. |
BibTeX:
@incollection{Luecking.Pfeiffer:FramingMultimodalTechnical:2012,
author = {Andy Lücking and Thies Pfeiffer},
title = {Framing Multimodal Technical Communication. With Focal Points in Speech-Gesture-Integration and Gaze Recognition},
booktitle = {Handbook of Technical Communication},
publisher = {Mouton de Gruyter},
year = {2012},
volume = {8},
pages = {591--644}
}
|
|
|
|
Pfeiffer, T. (2012). 3D Attention Volumes for Usability Studies in Virtual Reality. In Proceedings of the IEEE Virtual Reality 2012, 117-118, March 4-8. Orange County, CA, USA: IEEE. |
| Abstract: The time course and the distribution of visual attention are powerful measures for the evaluation of the usability of products. Eye tracking is thus an established method for evaluating websites, software ergonomy or modern cockpits for cars or airplanes. In most cases, however, the point of regard is measured on 2D products. This article presents work that uses an approach to measure the point of regard in 3D to generate 3D Attention Volumes as a qualitative 3D visualization of the distribution of visual attention. This visualization can be used to evaluate the design of virtual products in an immersive 3D setting, similar as heatmaps are used to assess the design of websites. | |
BibTeX:
@inproceedings{Pfeiffer:3DAttentionVolumes:2012,
author = {Thies Pfeiffer},
title = {3D Attention Volumes for Usability Studies in Virtual Reality},
booktitle = {Proceedings of the IEEE Virtual Reality 2012},
publisher = {IEEE},
year = {2012},
pages = {117-118}
}
|
|
|
|
Pfeiffer, T. (2012). Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects. In Gestures and Sign Language in Human-Computer Interaction and Embodied Communication, 9th International Gesture Workshop, GW 2011(7206), 238-249, May 2011. Athens, Greece: Springer-Verlag GmbH. |
| Abstract: Referring to objects using multimodal deictic expressions is an important form of communication. This work addresses the question on how pragmatic factors affect content distribution between the modalities speech and gesture. This is done by analyzing a study on deictic pointing gestures to objects under two conditions: with and without speech. The relevant pragmatic factor was the distance to the referent object. As one main result two strategies were identified which were used by participants to adapt their gestures to the condition. This knowledge can be used, e.g., to improve the naturalness of pointing gestures employed by embodied conversational agents. | |
BibTeX:
@inproceedings{Pfeiffer:InteractionbetweenSpeech:2012,
author = {Thies Pfeiffer},
title = {Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects},
booktitle = {Gestures and Sign Language in Human-Computer Interaction and Embodied Communication, 9th International Gesture Workshop, GW 2011},
publisher = {Springer-Verlag GmbH},
year = {2012},
number = {7206},
pages = {238--249},
doi = {http://dx.doi.org/10.1007/978-3-642-34182-3}
}
|
|
|
|
Pfeiffer, T. (2012). Measuring and Visualizing Attention in Space with 3D Attention Volumes. In Proceedings of the Symposium on Eye Tracking Research and Applications, 29-36, March 28-30. New York, NY, USA: ACM. |
| Abstract: Knowledge about the point of regard is a major key for the analysis of visual attention in areas such as psycholinguistics, psychology, neurobiology, computer science and human factors. Eye tracking is thus an established methodology in these areas, e.g. for investigating search processes, human communication behavior, product design or human-computer interaction. As eye tracking is a process which depends heavily on technology, the progress of gaze use in these scientific areas is tied to the advancements of eye-tracking technology. It is thus not surprising that in the last decades, research was primarily based on 2D stimuli and rather static scenarios, regarding both content and observer. Only with the advancements in mobile and robust eye-tracking systems, the observer is freed to physically interact in a 3D target scenario. Measuring and analyzing the point of regards in 3D space, however, requires additional techniques for data acquisition and scientific visualization. We describe the process for measuring the 3D point of regard and provide our own implementation of this process, which extends recent approaches of combining eye tracking with motion capturing, including holistic estimations of the 3D point of regard. In addition, we present a refined version of 3D attention volumes for representing and visualizing attention in 3D space. |
|
BibTeX:
@inproceedings{Pfeiffer:MeasuringandVisualizing:2012,
author = {Thies Pfeiffer},
title = {Measuring and Visualizing Attention in Space with 3D Attention Volumes},
booktitle = {Proceedings of the Symposium on Eye Tracking Research and Applications},
publisher = {ACM},
year = {2012},
pages = {29--36},
url = {http://doi.acm.org/10.1145/2168556.2168560},
doi = {http://dx.doi.org/10.1145/2168556.2168560}
}
|
|
|
|
Pfeiffer, T. (2012). Towards a Linguistically Motivated Model for Selection in Virtual Reality. In Proceedings of the IEEE Virtual Reality 2012, 89-90, March 4-8. Orange County, CA, USA: IEEE. |
| Abstract: Swiftness and robustness of natural communication is tied to the redundancy and complementarity found in our multimodal communication. Swiftness and robustness of human-computer interaction (HCI) is also a key to the success of a virtual reality (VR) environment. The interpretation of multimodal interaction signals has therefore been considered a high goal in VR research, e.g. following the visions of Bolt's put-that-there in 1980. It is our impression that research on user interfaces for VR systems has been focused primarily on finding and evaluating technical solutions and thus followed a technology-oriented approach to HCI. In this article, we argue to complement this by a human-oriented approach based on the observation of human-human interaction. The aim is to find models of human-human interaction that can be used to create user interfaces that feel natural. As the field of Linguistics is dedicated to the observation and modeling of human-human communication, it could be worthwhile to approach natural user interfaces from a linguistic perspective. We expect at least two benefits from following this approach. First, the human-oriented approach substantiates our understanding of natural human interactions. Second, it brings about a new perspective by taking the interaction capabilities of a human addressee into account, which are not often explicitly considered or compared with that of the system. As a consequence of following both approaches to create user interfaces, we expect more general models of human interaction to emerge. |
|
BibTeX:
@inproceedings{Pfeiffer:TowardsLinguisticallyMotivated:2012,
author = {Thies Pfeiffer},
title = {Towards a Linguistically Motivated Model for Selection in Virtual Reality},
booktitle = {Proceedings of the IEEE Virtual Reality 2012},
publisher = {IEEE},
year = {2012},
pages = {89--90}
}
|
|
|
|
Pfeiffer, T. (2012). Using Virtual Reality Technology in Linguistic Research. In Proceedings of the IEEE Virtual Reality 2012, 83-84, March 4-8. Orange County, CA, USA: IEEE. |
| Abstract: In this paper, we argue that empirical research on genuine linguistic topics, such as on the production of multimodal utterances in the speaker and the interpretation of the multimodal signals in the interlocutor, can greatly benefit from the use of virtual reality technologies. Established methodologies for research on multimodal interactions, such as the presentation of pre-recorded 2D videos of interaction partners as stimuli and the recording of interaction partners using multiple 2D video cameras have crucial shortcomings regarding ecological validity and the precision of measurements that can be achieved. In addition, these methodologies enforce restrictions on the researcher. The stimuli, for example, are not very interactive and thus not as close to natural interactions as ultimately desired. Also, the analysis of 2D video recordings requires intensive manual annotations, often frame-by-frame, which negatively affects the feasible number of interactions which can be included in a study. The technologies bundled under the term virtual reality offer exciting possibilities for the linguistic researcher: gestures can be tracked without being restricted to fixed perspectives, annotation can be done on large corpora (semi-)automatically and virtual characters can be used to produce specific linguistic stimuli in a repetitive but interactive fashion. Moreover, immersive 3D visualizations can be used to recreate a simulation of the recorded interactions by fusing the raw data with theoretic models to support an iterative data-driven development of linguistic theories. This paper discusses the potential of virtual reality technologies for linguistic research and provides examples for the application of the methodology. |
|
BibTeX:
@inproceedings{Pfeiffer:UsingVirtualReality:2012,
author = {Thies Pfeiffer},
title = {Using Virtual Reality Technology in Linguistic Research},
booktitle = {Proceedings of the IEEE Virtual Reality 2012},
publisher = {IEEE},
year = {2012},
pages = {83--84}
}
|
|
|
|
Pfeiffer-Leßmann, N., Pfeiffer, T. & Wachsmuth, I. (2012). An Operational Model of Joint Attention - Timing of Gaze Patterns in Interactions between Humans and a Virtual Human. In Proceedings of the 34th Annual Meeting of the Cognitive Science Society, 851-856, August. Austin, TX. |
| Abstract: Joint attention has been identified as a foundational skill in human-human interaction. If virtual humans are to engage in joint attention, they have to meet the expectations of their human interaction partner and provide interactional signals in a natural way. This requires operational models of joint attention with precise information on natural gaze timing. We substantiate our model of the joint attention process by studying human-agent interactions in immersive virtual reality and present results on the timing of referential gaze during the initiation of joint attention |
|
BibTeX:
@inproceedings{Pfeiffer-Lessmann.etal:OperationalModelJoint:2012,
author = {Nadine Pfeiffer-Leßmann and Thies Pfeiffer and Ipke Wachsmuth},
title = {An Operational Model of Joint Attention - Timing of Gaze Patterns in Interactions between Humans and a Virtual Human},
booktitle = {Proceedings of the 34th Annual Meeting of the Cognitive Science Society},
year = {2012},
pages = {851--856},
url = {http://wsu.mindmodeling.org/cogsci2012/cogsci12_proceedings.pdf}
}
|
|
|
|
Pfeiffer-Lessmann, N., Pfeiffer, T. & Wachsmuth, I. (2012). An Operational Model of Joint Attention - Timing of the Initiate-Act in Interactions with a Virtual Human. In Proceedings of the KogWis 2012, 96-97, October. Bamberg, Germany: University of Bamberg Press. |
| Abstract: Joint attention has been identified as a foundational skill in human-human interaction. In our research, we use virtual humans to evaluate cognitive models, here, enabling them to engage in joint attention. This requires an operational model with precise information on natural gaze timing. We aim at substantiating our model of joint attention (Pfeiffer-Leßmann & Wachsmuth 2009) by studying human-agent interactions and present results on the timing of referential gaze during the initiation of joint attention. Our operational model of joint attention covers four phases: initiate-act (1), respond-act (2), feedback phase (3), and focus-state (4). It is implemented in the cognitive architecture of our virtual agent Max. For the evaluation of the model in interaction with humans, investigations on timeframes, human expectations and insights on how humans actually perceive Max’ behaviors are indispensable. In the present study we focus on phase 1. We let the participants engage in joint attention with Max in an immersive virtual reality environment, which provides a highly controlled experimental setup. The experiment had two conditions: in one the participants took the role of the initiator, in the other Max was the initiator. We found that participants accepted the same kind of gaze patterns produced by Max using our model as natural as they themselves performed when they had the initiative. Thus, under both conditions we found a mean dwell time of 1.9s for the referential gaze act, which is compatible with related findings in human-human interaction. |
|
BibTeX:
@inproceedings{Pfeiffer-Lessmann.etal:OperationalModelJoint:2012b,
author = {Nadine Pfeiffer-Lessmann and Thies Pfeiffer and Ipke Wachsmuth},
title = {An Operational Model of Joint Attention - Timing of the Initiate-Act in Interactions with a Virtual Human},
booktitle = {Proceedings of the KogWis 2012},
publisher = {University of Bamberg Press},
year = {2012},
pages = {96--97},
url = {http://www.cogsys.wiai.uni-bamberg.de/kogwis2012/files/prelimproc.pdf}
}
|
|
|
|
Essig, K., Frank, S., Sand, N., Künsemöller, J., Pfeiffer, T., Ritter, H. & Schack, T. (2011). JVideoGazer - Towards an Automatic Annotation of Gaze Videos from Natural Scenes. Journal of Eye Movement Research, 4(3), 48, 21 - 25 August. |
| Abstract: Modern mobile eye-tracking systems record participants' gaze behavior while they interact in an unrestricted environment, often haptically manipulating complex objects of interest. These mobile systems can be easily set up and operated. However, the gained freedom is paid for with a difficult and cumbersome analysis of the stored gaze data, as the recorded scene videos are individually different. They have to be manually annotated, sometimes frame-by-frame, to mark regions of interest. This very time consuming and error-prone process prevents the use of eye-tracking in various areas of application. In order to speed up the annotation, we developed a new software, JVideoGazer, for fully- and semi-automatic annotation of gaze videos. This software provides two alternative algorithms for object tracking: a SURF (Speeded-Up Robust Features) interest point detector and descriptor, and a Kernel-Based Bayesian Filtering (KBBF) algorithm. We evaluated the JVideoGazer by comparing the results of the object trackers among each other and to those of a manual annotation based on scene videos of a typical day-by-day task. Results show that the JVideoGazer provides a reliable automatic video analysis even under challenging recording conditions and thus significantly speeds up the annotation process. Furthermore, the tracking algorithms show different performance advantages under diverse experimental conditions, making our modular approach with various object tracking algorithms a suitable tool for the annotation of gaze videos from natural scenes. |
|
BibTeX:
@article{EssigFrankEtAl:JVideoGazer-Towards:2011,
author = {Essig, Kai and Frank, Seifert and Sand, Norbert and Künsemöller, Jörn and Pfeiffer, Thies and Ritter, Helge and Schack, Thomas},
title = {JVideoGazer - Towards an Automatic Annotation of Gaze Videos from Natural Scenes},
booktitle = {Abstracts of the 16th European Conference on Eye Movements},
journal = {Journal of Eye Movement Research},
year = {2011},
volume = {4(3)},
pages = {48},
url = {http://www.jemr.org/online/4/3/1}
}
|
|
|
|
Essig, K., Frank, S., Sand, N., Jörn, K., Pfeiffer, T., Ritter, H. & Schack, T. (2011). JVideoGazer - Towards an Automatic Annotation of Gaze Videos from Natural Scenes. In Proceedings of the World Congress on Engineering and Technology (CET). Shanghai. |
| Abstract: Modern mobile eye-tracking systems record participants' gaze behavior while they interact in an unrestricted environment, often haptically manipulating complex objects of interest. These mobile systems can be easily set up and operated. However, the gained freedom is paid for with a difficult and cumbersome analysis of the stored gaze data, as the recorded scene videos are individually different. They have to be manually annotated, sometimes frame-by-frame, to mark regions of interest. This very time consuming and error-prone process prevents the use of eye-tracking in various areas of application. In order to speed up the annotation, we developed a new software, JVideoGazer, for fully- and semi-automatic annotation of gaze videos. This software provides two alternative algorithms for object tracking: a SURF (Speeded-Up Robust Features) interest point detector and descriptor, and a Kernel-Based Bayesian Filtering (KBBF) algorithm. We evaluated the JVideoGazer by comparing the results of the object trackers among each other and to those of a manual annotation based on scene videos of a typical day-by-day task. Results show that the JVideoGazer provides a reliable automatic video analysis even under challenging recording conditions and thus significantly speeds up the annotation process. Furthermore, the tracking algorithms show different performance advantages under diverse experimental conditions, making our modular approach with various object tracking algorithms a suitable tool for the annotation of gaze videos from natural scenes. |
|
BibTeX:
@inproceedings{EssigFrankEtAl:JVideoGazer-Towards:2011a,
author = {Essig, Kai and Frank, Seifert and Sand, Norbert and Jörn, Künsemöller and Pfeiffer, Thies and Ritter, Helge and Schack, Thomas},
title = {JVideoGazer - Towards an Automatic Annotation of Gaze Videos from Natural Scenes},
booktitle = {Proceedings of the World Congress on Engineering and Technology (CET)},
year = {2011},
note = {to appear}
}
|
|
|
|
Hülsmann, F., Dankert, T. & Pfeiffer, T. (2011). Gaze-Based Interaction - Eyesteroids – Input Methods for Immersive Games. In Proceedings of the Postersession at the Interdisciplinary College 2011. |
BibTeX:
@inproceedings{HuelsmannDankertEtAl:Gaze-BasedInteraction-:2011,
author = {Felix Hülsmann and Timo Dankert and Thies Pfeiffer},
title = {Gaze-Based Interaction - Eyesteroids – Input Methods for Immersive Games},
booktitle = {Proceedings of the Postersession at the Interdisciplinary College 2011},
year = {2011},
note = {(poster)}
}
|
|
|
|
Hülsmann, F., Dankert, T. & Pfeiffer, T. (2011). Comparing Gaze-based and Manual Interaction in a Fast-paced Gaming Task in Virtual Reality. In Virtuelle & Erweiterte Realität, 8. Workshop der GI-Fachgruppe VR/AR, 1-12. Aachen: Shaker Verlag. |
| Abstract: The idea of using gaze as an interaction modality has been put forward by the famous work of Bolt in 1981. In virtual reality (VR), gaze has been used for several means since then: view-dependent optimization of rendering, intelligent information visualization, reference communication in distributed telecommunication settings and object selection. Our own research aims at improving gaze-based interaction methods in general. In this paper, gaze-based interaction is examined in a fast-paced selection task to identify current usability problems of gaze-based interaction and to develop best practices. To this end, an immersive Asteroids-like shooter called Eyesteroids was developed to support a study comparing manual and gaze-based interaction methods. Criteria for the evaluation were interaction performance and user immersion. The results indicate that while both modalities (hand and gaze) work well for the task, manual interaction is easier to use and often more accurate than the implemented gaze-based methods. The reasons are discussed and the best practices as well as options for further improvements of gaze-based interaction methods are presented. |
|
BibTeX:
@inproceedings{Hulsmann.etal:ComparingGazebased:2011,
author = {Felix Hülsmann and Timo Dankert and Thies Pfeiffer},
title = {Comparing Gaze-based and Manual Interaction in a Fast-paced Gaming Task in Virtual Reality},
booktitle = {Virtuelle & Erweiterte Realität, 8. Workshop der GI-Fachgruppe VR/AR},
publisher = {Shaker Verlag},
year = {2011},
pages = {1--12}
}
|
|
|
|
Liguda, C. & Pfeiffer, T. (2011). A Question Answer System for Math Word Problems. In First International Workshop on Algorithmic Intelligence. |
| Abstract: Solving word problems is an important part in school education in primary as well as in high school. Although, the equations that are given by a word problem could be solved by most computer algebra programs without problems, there are just few systems that are able to solve word problems. In this paper we present the ongoing work on a system, that is able to solve word problems from german primary school math books. | |
BibTeX:
@inproceedings{Liguda.Pfeiffer:QuestionAnswerSystem:2011,
author = {Christian Liguda and Thies Pfeiffer},
title = {A Question Answer System for Math Word Problems},
booktitle = {First International Workshop on Algorithmic Intelligence},
year = {2011},
note = {to appear}
}
|
|
|
|
Mattar, N. & Pfeiffer, T. (2011). Interactive 3D Graphs for Web-based Social Networking Platforms. International Journal of Computer Information Systems and Industrial Management Applications, 3, 427-434. |
| Abstract: Social networking platforms (SNPs) are meant to reflect the social relationships of their users. Users typically enter very personal information and should get useful feedback about their social network. They should indeed be empowered to exploit the information they have entered. In reality, however, most SNPs actually hide the structure of the user’s rich social network behind very restricted text-based user interfaces and large parts of the potential information which could be extracted from the entered data lies fallow. This article presents results from a user study showing that 3D visualizations of social graphs can be utilized more effectively – and moreover – are preferred by users compared to traditional text-based interfaces. Subsequently, the article addresses the problem of how to deploy interactive 3D graphical interfaces to large user communities. This is demonstrated on the social graph app¬lication FriendGraph3D for Facebook. | |
BibTeX:
@article{MattarPfeiffer:Interactive3DGraphs:2011,
author = {Nikita Mattar and Thies Pfeiffer},
title = {Interactive 3D Graphs for Web-based Social Networking Platforms},
journal = {International Journal of Computer Information Systems and Industrial Management Applications},
year = {2011},
volume = {3},
pages = {427--434},
url = {http://www.mirlabs.org/ijcisim/regular_papers_2011/Paper48.pdf}
}
|
|
|
|
Pfeiffer, T. (2011). Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects. In Gestures in Embodied Communication and Human-Computer Interaction, 9th International Gesture Workshop, GW 2011, 109-112, Mai. Athens, Greece: National and Kapodistrian University of Athens. |
| Abstract: Referring to objects using multimodal deictic expressions is an important form of communication. This work addresses the question on how content is distributed between the modalities speech and gesture by comparing deictic pointing gestures to objects with and without speech. As a result, two main strategies used by participants to adapt their gestures to the condition were identified. This knowledge can be used, e.g., to improve the naturalness of pointing gestures employed by embodied conversational agents. | |
BibTeX:
@inproceedings{Pfeiffer:InteractionbetweenSpeech:2011,
author = {Thies Pfeiffer},
title = {Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects},
booktitle = {Gestures in Embodied Communication and Human-Computer Interaction, 9th International Gesture Workshop, GW 2011},
publisher = {National and Kapodistrian University of Athens},
year = {2011},
pages = {109--112}
}
|
|
|
|
Pfeiffer, T. (2011). Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces, December. Aachen, Germany: Shaker Verlag. |
BibTeX:
@book{Pfeiffer:UnderstandingMultimodalDeixis:2011,
author = {Thies Pfeiffer},
title = {Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces},
publisher = {Shaker Verlag},
year = {2011},
url = {http://www.shaker.de/shop/978-3-8440-0592-9}
}
|
|
|
|
Pfeiffer, T., Liguda, C., Wachsmuth, I. & Stein, S. (2011). Living with a Virtual Agent: Seven Years with an Embodied Conversational Agent at the Heinz Nixdorf MuseumsForum. In Proceedings of the International Conference Re-Thinking Technology in Museums 2011 - Emerging Experiences, 121 - 131. Ireland: thinkk creative & the University of Limerick. |
| Abstract: Since 2004 the virtual agent Max is living at the Heinz Nixdorf MuseumsForum – a computer science museum. He is welcoming and entertaining visitors ten hours a day, six days a week, for seven years. This article brings together the experiences made by the staff of the museum, the scientists who created and maintained the installation, the visitors and the agent himself. It provides insights about the installation’s hard- and software and presents highlights of the agent’s ontogenesis in terms of the features he has gained. A special focus is on the means Max uses to engage with visitors and the features which make him attractive. | |
BibTeX:
@inproceedings{PfeifferLigudaEtAl:LivingwithVirtual:2011,
author = {Thies Pfeiffer and Christian Liguda and Ipke Wachsmuth and Stefan Stein},
title = {Living with a Virtual Agent: Seven Years with an Embodied Conversational Agent at the Heinz Nixdorf MuseumsForum},
booktitle = {Proceedings of the International Conference Re-Thinking Technology in Museums 2011 - Emerging Experiences},
publisher = {thinkk creative & the University of Limerick},
year = {2011},
pages = {121 -- 131}
}
|
|
|
|
Pfeiffer, T. & Wachsmuth, I. (2011). Dreidimensionale Erfassung visueller Aufmerksamkeit für Usability-Bewertungen an virtuellen Prototypen. In 10. Paderborner Workshop Augmented and Virtual Reality in der Produktentstehung(295), 39-51. Paderborn, Germany: Heinz Nixdorf Institut, Universität Paderborn. |
| Abstract: Die Messung visueller Aufmerksamkeit mittels Eye-Tracking ist eine etablierte Methode in der Bewertung von Ergonomie und Usability. Ihr Gegenstandsbereich beschränkt sich jedoch primär auf 2D-Inhalte wie Webseiten, Produktfotos oder –videos. Bewegte Interaktion im dreidimensionalen Raum wird selten erfasst, weder am realen Objekt, noch am virtuellen Prototyp. Mit einer Aufmerksamkeitsmessung im Raum könnte der Gegenstandsbereich um diese Fälle deutlich erweitert werden. Der vorliegende Artikel arbeitet den aktuellen Stand der Forschung zur Messung visueller Aufmerksamkeit im Raum auf. Dabei werden insbesondere die zu bewältigenden Schwierigkeiten herausgearbeitet und Lösungsansätze aufgezeigt. Als Schwerpunkt werden drei Themen an eigenen Arbeiten diskutiert: Aufbau und Kalibrierung der Systeme, Bestimmung des betrachteten Volumens und Visualisierung der Aufmerksamkeit im Raum. | |
BibTeX:
@inproceedings{PfeifferWachsmuth:DreidimensionaleErfassungvisueller:2011,
author = {Thies Pfeiffer and Ipke Wachsmuth},
title = {Dreidimensionale Erfassung visueller Aufmerksamkeit für Usability-Bewertungen an virtuellen Prototypen},
booktitle = {10. Paderborner Workshop Augmented and Virtual Reality in der Produktentstehung},
publisher = {Heinz Nixdorf Institut, Universität Paderborn},
year = {2011},
number = {295},
pages = {39--51}
}
|
|
|
|
Renner, P., Lüdike, N., Wittrowski, J. & Pfeiffer, T. (2011). Towards Continuous Gaze-Based Interaction in 3D Environments - Unobtrusive Calibration and Accuracy Monitoring. In Virtuelle & Erweiterte Realität, 8. Workshop der GI-Fachgruppe VR/AR, 13-24. Aachen: Shaker Verlag. |
| Abstract: Gaze-based interaction in virtual reality promises to have interactional advantages, but the current state-of-art is still faced with usability issues. Two of them, the decrease in accuracy (drifts) under continuous usage, e.g. due to slippages of the gear, and the obtrusive standard calibration procedure are addressed in this work. We propose a new calibration procedure that blends smoothly in the virtual environment: an eye-catching moving object enters the field of view and while the user follows this object, the tracking is calibrated. In addition, we designed two heuristics to detect drifts automatically and thus trigger calibrations only when necessary. The applicability of both approaches was positively verified in a user study. |
|
BibTeX:
@inproceedings{Renner.etal:TowardsContinuousGaze:,
author = {Patrick Renner and Nico Lüdike and Jens Wittrowski and Thies Pfeiffer},
title = {Towards Continuous Gaze-Based Interaction in 3D Environments - Unobtrusive Calibration and Accuracy Monitoring},
booktitle = {Virtuelle & Erweiterte Realität, 8. Workshop der GI-Fachgruppe VR/AR},
publisher = {Shaker Verlag},
year = {2011},
pages = {13--24}
}
|
|
|
|
Renner, P., Lüdike, N., Wittrowski, J. & Pfeiffer, T. (2011). Gaze-Based Interaction - Unobtrusive Re-Calibration for Continuous Interaction in 3D. In Proceedings of the Poster Session at the Interdisciplinary College 2011. |
BibTeX:
@inproceedings{RennerLuedikeEtAl:Gaze-BasedInteraction-:2011,
author = {Patrick Renner and Nico Lüdike and Jens Wittrowski and Thies Pfeiffer},
title = {Gaze-Based Interaction - Unobtrusive Re-Calibration for Continuous Interaction in 3D},
booktitle = {Proceedings of the Poster Session at the Interdisciplinary College 2011},
year = {2011},
note = {(poster)}
}
|
|
|
|
Mattar, N. & Pfeiffer, T. (2010). Relationships in Social Networks Revealed: A Facebook App for Social Graphs in 3D Based on X3DOM and WebGL. In Proceedings of the IADIS International Conference Web Virtual Reality and Three-Dimensional Worlds 2010, 269-276: IADIS Press. |
| Abstract: From the perspective of individual users, social networking platforms (SNPs) are meant to reflect their social relationships. SNPs should provide feedback allowing users to exploit the information they have entered. In reality, however, most SNPs actually hide the rich social network constructed by the users in their databases behind simple user interfaces. These interfaces reduce the complexity of a user's social network to a text-based list in HTML. This article presents results from a user study showing that 3D visualizations of social graphs can be utilized more effectively – and moreover – are preferred by users compared to traditional text-based interfaces. Subsequently, the article addresses the problem of deployment of rich interfaces. A social graph application for Facebook is presented, demonstrating how WebGL and HTML5/X3D can be used to implement rich social applications based on upcoming web standards. | |
BibTeX:
@inproceedings{MattarPfeiffer:RelationshipsinSocial:2010,
author = {Nikita Mattar and Thies Pfeiffer},
title = {Relationships in Social Networks Revealed: A Facebook App for Social Graphs in 3D Based on X3DOM and WebGL},
booktitle = {Proceedings of the IADIS International Conference Web Virtual Reality and Three-Dimensional Worlds 2010},
publisher = {IADIS Press},
year = {2010},
pages = {269--276}
}
|
|
|
|
Pfeiffer, T. (2010). Object Deixis: Interaction Between Verbal Expressions and Manual Pointing Gestures. In Proceedings of the KogWis 2010, 221-222. Potsdam: Universitätsverlag Potsdam. |
| Abstract: Object deixis is at the core of language and an ideal example of multimodality. Speech, gaze and manual gestures are used by interlocutors to refer to objects in their 3D environment. The interplay of verbal expressions and gestures during deixis is an active research topic in linguistics as well as in human-computer interaction. Previously, we conducted a study on manual pointing during dialogue games using state-of-the art tracking technologies to record gestures with high spatial precision (Kranstedt, Lücking, Pfeiffer, Rieser and Wachsmuth, 2006), To reveal strategies in manual pointing gestures, we present an analysis of this data with a new visualization technique. |
|
BibTeX:
@inproceedings{Pfeiffer:ObjectDeixis_Interaction:2010,
author = {Thies Pfeiffer},
title = {Object Deixis: Interaction Between Verbal Expressions and Manual Pointing Gestures},
booktitle = {Proceedings of the KogWis 2010},
publisher = {Universitätsverlag Potsdam},
year = {2010},
pages = {221--222},
url = {http://nbn-resolving.de/urn:nbn:de:kobv:517-opus-46055}
}
|
|
|
|
Pfeiffer, T. (2010). Tracking and Visualizing Visual Attention in Real 3D Space. In Proceedings of the KogWis 2010, 220-221. Potsdam: Universitätsverlag Potsdam. |
| Abstract: Humans perceive, reason and act within a 3D environment. In empirical methods, however, researchers often restrict themselves to 2D, either in using 2D content or relying on 2D recordings for analysis, such as videos or 2D eye movements. Regarding, e.g., multimodal deixis, we address the open question of the morphology of the referential space (Butterworth and Itakura, 2000), For modeling the referential space of gaze pointing, precise knowledge about the target of our participants’ visual attention is crucial. To this ends, we developed methods to assess the location of the point of regard, which are outlined here. |
|
BibTeX:
@inproceedings{Pfeiffer:TrackingandVisualizing:2010,
author = {Thies Pfeiffer},
title = {Tracking and Visualizing Visual Attention in Real 3D Space},
booktitle = {Proceedings of the KogWis 2010},
publisher = {Universitätsverlag Potsdam},
year = {2010},
pages = {220--221},
url = {http://nbn-resolving.de/urn:nbn:de:kobv:517-opus-46055}
}
|
|
|
|
Pfeiffer, T. (2010). Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces, School: Bielefeld University,, April. Bielefeld, Germany. |
| Abstract: When humans communicate, we use deictic expressions to refer to objects in our surrounding and put them in the context of our actions. In face to face interaction, we can complement verbal expressions with gestures and, hence, we do not need to be too precise in our verbal protocols. Our interlocutors hear our speaking; see our gestures and they even read our eyes. They interpret our deictic expressions, try to identify the referents and -- normally -- they will understand. If only machines could do alike. The driving vision behind the research in this thesis are multimodal conversational interfaces where humans are engaged in natural dialogues with computer systems. The embodied conversational agent Max developed in the A.I. group at Bielefeld University is an example of such an interface. Max is already able to produce multimodal deictic expressions using speech, gaze and gestures, but his capabilities to understand humans are not on par. If he was able to resolve multimodal deictic expressions, his understanding of humans would increase and interacting with him would become more natural. Following this vision, we as scientists are confronted with several challenges. First, accurate models for human pointing have to be found. Second, precise data on multimodal interactions has to be collected, integrated and analyzed in order to create these models. This data is multimodal (transcripts, voice and video recordings, annotations) and not directly accessible for analysis (voice and video recordings). Third, technologies have to be developed to support the integration and the analysis of the multimodal data. Fourth, the created models have to be implemented, evaluated and optimized until they allow a natural interaction with the conversational interface. To this ends, this work aims to deepen our knowledge of human non-verbal deixis, specifically of manual and gaze pointing, and to apply this knowledge in conversational interfaces. At the core of the theoretical and empirical investigations of this thesis are models for the interpretation of pointing gestures to objects. These models address the following questions: When are we pointing? Where are we pointing to? Which objects are we pointing at? With respect to these questions, this thesis makes the following three contributions: First, gaze-based interaction technology for 3D environments: Gaze plays an important role in human communication, not only in deictic reference. Yet, technology for gaze interaction is still less developed than technology for manual interaction. In this thesis, we have developed components for real-time tracking of eye movements and of the point of regard in 3D space and integrated them in a framework for DRIVE. DRIVE provides viable information about human communicative behavior in real-time. This data can be used to investigate and to design processes on higher cognitive levels, such as turn-taking, check- backs, shared attention and resolving deictic reference. Second, data-driven modeling: We answer the theoretical questions about timing, direction, accuracy and dereferential power of pointing by data-driven modeling. As empirical basis for the simulations, we created a substantial corpus with high-precision data from an extensive study on multimodal pointing. Two further studies complemented this effort with substantial data on gaze pointing in 3D. Based on this data, we have developed several models of pointing and successfully created a model for the interpretation of manual pointing that achieves a human-like performance level. Third, new methodologies for research on multimodal deixis in the fields of linguistics and computer science: The experimental-simulative approach to modeling -- which we follow in this thesis -- requires large collections of heterogeneous data to be recorded, integrated, analyzed and resimulated. To support the researcher in these tasks, we developed the Interactive Augmented Data Explorer. IADE is an innovative tool for research on multimodal interaction based on virtual reality technology. It allows researchers to literally immerse into multimodal data and interactively explore them in real-time and in virtual space. With IADE we have also extended established approaches for scientific visualization of linguistic data to 3D, which previously existed only for 2D methods of analysis (e.g. video recordings or computer screen experiments). By this means, we extended McNeill's 2D depiction of the gesture space to gesture space volumes expanding in time and space. Similarly, we created attention volumes, a new way to visualize the distribution of attention in 3D environments. | |
BibTeX:
@phdthesis{Pfeiffer:UnderstandingMultimodalDeixis:2010,
author = {Thies Pfeiffer},
title = {Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces},
school = {Bielefeld University},
year = {2010},
url = {http://pub.ub.uni-bielefeld.de/pub?func=drec&id=2308111}
}
|
|
|
|
Renner, P., Dankert, T., Schneider, D., Mattar, N. & Pfeiffer, T. (2010). Navigating and Selecting in the Virtual Supermarket: Review and Update of Classic Interaction Techniques. In Virtuelle und Erweiterte Realität: 7. Workshop der GI-Fachgruppe VR/AR, 71-82. Aachen, Germany: Shaker Verlag GmbH. |
| Abstract: Classic techniques for navigation and selection such as Image-Plane and World in Miniature have been around for more than 20 years. In the course of a seminar on interaction in virtual reality we reconsidered five methods for navigation and two for selection. These methods were significantly extended by the use of up-to-date hardware such as Fingertracking devices and the Nintendo Wii Balance Board and evaluated in a virtual supermarket scenario. Two user studies, one on experts and one on novices, revealed information on usability and efficiency. As an outcome, the combination of Ray-Casting and Walking in Place turned out to be the fastest. | |
BibTeX:
@inproceedings{RennerDankertEtAl:NavigatingandSelecting:2010,
author = {Patrick Renner and Timo Dankert and Dorothe Schneider and Nikita Mattar and Thies Pfeiffer},
title = {Navigating and Selecting in the Virtual Supermarket: Review and Update of Classic Interaction Techniques},
booktitle = {Virtuelle und Erweiterte Realität: 7. Workshop der GI-Fachgruppe VR/AR},
publisher = {Shaker Verlag GmbH},
year = {2010},
pages = {71--82}
}
|
|
|
|
Bluhm, A., Eickmeyer, J., Feith, T., Mattar, N. & Pfeiffer, T. (2009). Exploration von sozialen Netzwerken im 3D Raum am Beispiel von $SoNforall R $ für Last.fm. In Virtuelle und Erweiterte Realität - Sechster Workshop der GI-Fachgruppe VR/AR, 269-280. Aachen: Shaker Verlag. |
| Abstract: Die Portale für soziale Netzwerke im Internet gehen mittlerweile deutlich über die Verwaltung einfacher Bekanntschaftsbeziehungen hinaus. Ihnen liegen immer reichhaltigere Datenmodelle zu Grunde. Darstellung und Exploration dieser Netzwerke sind eine grosse Herausforderung für die Entwickler, wenn beides nicht zu einer solchen für die Benutzer werden soll. Im Rahmen eines studentischen Projektes wurde die dritte Dimension für die Darstellung des komplexen sozialen Netzwerkes von Last.fm nutzbar gemacht. Durch die entwickelte Anwendung SoNAR wird das Netzwerk interaktiv und intuitiv sowohl am Desktop, als auch in der Immersion einer dreiseitigen CAVE explorierbar. Unterschiedliche Relationen des sozialen Graphen können parallel exploriert und damit Zusammenhänge zwischen Individuen intuitiv erfahren werden. Eine Suchfunktion erlaubt dabei die fexible Komposition verschiedener Startknoten für die Exploration. | |
BibTeX:
@inproceedings{Bluhm:Sonar:2009,
author = {Andreas Bluhm and Jens Eickmeyer and Tobias Feith and Nikita Mattar and Thies Pfeiffer},
title = {Exploration von sozialen Netzwerken im 3D Raum am Beispiel von $SoNforall R $ für Last.fm},
booktitle = {Virtuelle und Erweiterte Realität - Sechster Workshop der GI-Fachgruppe VR/AR},
publisher = {Shaker Verlag},
year = {2009},
pages = {269--280}
}
|
|
|
|
Pfeiffer, T., Latoschik, M.E. & Wachsmuth, I. (2009). Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments. Journal of Virtual Reality and Broadcasting, 5 (16), jan. |
| Abstract: Tracking user’s visual attention is a fundamental aspect in novel human-computer interaction paradigms found in Virtual Reality. For example, multimodal interfaces or dialogue-based communications with virtual and real agents greatly benefit from the analysis of the user’s visual attention as a vital source for deictic references or turn-taking signals. Current approaches to determine visual attention rely primarily on monocular eye trackers. Hence they are restricted to the interpretation of two-dimensional fixations relative to a defined area of projection. The study presented in this article compares precision, accuracy and application performance of two binocular eye tracking devices. Two algorithms are compared which derive depth information as required for visual attention-based 3D interfaces. This information is further applied to an improved VR selection task in which a binocular eye tracker and an adaptive neural network algorithm is used during the disambiguation of partly occluded objects. |
|
BibTeX:
@article{Pfeiffer:Evaluation:2009,
author = {Thies Pfeiffer and Marc Erich Latoschik and Ipke Wachsmuth},
title = {Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments},
journal = {Journal of Virtual Reality and Broadcasting},
year = {2009},
volume = {5},
number = {16},
pdf = {http://www.jvrb.org/past-issues/5.2008/1660/5200816.pdf},
url = {http://www.jvrb.org/past-issues/5.2008/1660}
}
|
|
|
|
Pfeiffer, T. & Mattar, N. (2009). Benefits of Locating Overt Visual Attention in Space Using Binocular Eye Tracking for Mixed Reality Applications. In Workshop-Proceedings der Tagung Mensch & Computer 2009: Grenzenlos frei!?, 272-274. Berlin: Logos Berlin. |
| Abstract: The "Where?" is quite important for Mixed Reality applications: Where is the user looking at? Where should augmentations be displayed? The location of the overt visual attention of the user can be used both to disambiguate referent objects and to inform an intelligent view management of the user interface. While the vertical and horizontal orientation of attention is quite commonly used, e.g. derived from the orientation of the head, only knowledge about the distance allows for an intrinsic measurement of the location of the attention. This contribution reviews our latest results on detecting the location of attention in 3D space using binocular eye tracking. | |
BibTeX:
@inproceedings{Pfeiffer:MixedReality:2009,
author = {Thies Pfeiffer and Nikita Mattar},
title = {Benefits of Locating Overt Visual Attention in Space Using Binocular Eye Tracking for Mixed Reality Applications},
booktitle = {Workshop-Proceedings der Tagung Mensch & Computer 2009: Grenzenlos frei!?},
publisher = {Logos Berlin},
year = {2009},
pages = {272--274}
}
|
|
|
|
Breuing, A., Pfeiffer, T. & Kopp, S. (2008). Conversational Interface Agents for the Semantic Web - a Case Study. In Proceedings of the Poster and Demonstration Session at the 7th International Semantic Web Conference (ISWC 2008), October. Karlsruhe, Germany. |
| Abstract: The Semantic Web is about to become a rich source of knowledge whose potential will be squandered if it is not accessible for everyone. Intuitive interfaces like conversational agents are needed to better disseminate this knowledge, either on request or even proactively in a context-aware manner. This paper presents work on extending an existing conversational agent, Max, with abilities to access the Semantic Web via natural language communication. | |
BibTeX:
@inproceedings{Breuing:Conversational:2008,
author = {Alexa Breuing and Thies Pfeiffer and Stefan Kopp},
title = {Conversational Interface Agents for the Semantic Web - a Case Study},
booktitle = {Proceedings of the Poster and Demonstration Session at the 7th International Semantic Web Conference (ISWC 2008)},
year = {2008},
url = {http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-401/}
}
|
|
|
|
Meißner, M., Essig, K., Pfeiffer, T., Decker, R. & Ritter, H. (2008). Eye-tracking decision behaviour in choice-based conjoint analysis. In ECVP Abstract Supplement, 37 , 97-97: PION LTD. |
| Abstract: In a novel approach we investigated choice processes using eye tracking to improve research instruments in marketing research. Choice-based conjoint analysis (CBC) is the most widely-used tool for investigating consumer preferences on the basis of choice tasks. While CBC is highly appreciated for its realism (Haaijer and Wedel, 2007), marketing researchers have highlighted that respondents are easily exposed to the problem of information overload (Green et al, 2001). The question how much information is being processed during choice processes and how preference measurement is affected remains an open research issue. We investigated choice processes using eye tracking in a CBC on-line consumer survey. We showed (i) that the extent to which information is processed is decreasing in later choice tasks, (ii) in how far information overload changes the pattern of eye movements, and (iii) how the difficulty of a choice task influences information processing. | |
BibTeX:
@inproceedings{MeissnerEssigEtAl:Eye-trackingdecisionbehaviour:2008,
author = {Meißner, Martin and Essig, Kai and Pfeiffer, Thies and Decker, Reinhold and Ritter, Helge},
title = {Eye-tracking decision behaviour in choice-based conjoint analysis},
booktitle = {ECVP Abstract Supplement},
publisher = {PION LTD},
year = {2008},
volume = {37},
pages = {97--97},
note = {Conference Abstract},
url = {http://www.perceptionweb.com/abstract.cgi?id=v080344}
}
|
|
|
|
Pfeiffer, T., Latoschik, M. & Wachsmuth, I. (2008). Conversational Pointing Gestures for Virtual Reality Interaction: Implications from an Empirical Study. In Proceedings of the IEEE Virtual Reality 2008, 281-282, March 8 - 12. Reno, Nevada. |
| Abstract: Interaction in conversational interfaces strongly relies on the system's capability to interpret the user's references to objects via deictic expressions. Deictic gestures, especially pointing gestures, provide a powerful way of referring to objects and places, e.g., when communicating with an Embodied Conversational Agent in a Virtual Reality Environment. We highlight results drawn from a study on pointing and draw conclusions for the implementation of pointing-based conversational interactions in partly immersive Virtual Reality. | |
BibTeX:
@inproceedings{Pfeiffer:Conversational:2008,
author = {Pfeiffer, T. and Latoschik, M.E. and Wachsmuth, I.},
title = {Conversational Pointing Gestures for Virtual Reality Interaction: Implications from an Empirical Study},
booktitle = {Proceedings of the IEEE Virtual Reality 2008},
journal = {Virtual Reality Conference},
year = {2008},
pages = {281--282}
}
|
|
|
|
Pfeiffer, T. & Wachsmuth, I. (2008). Social Presence: The Role of Interpersonal Distances in Affective Computer-Mediated Communication. In Proceedings of the 11th. International Workshop on Presence, 275-279, October. Padova, Italy: CLEUP Cooperativa Libraria Universitaria Padova. |
| Abstract: Emotions and interpersonal distances are identified as key aspects in social interaction. A novel Affective Computer-Mediated Communication (ACMC) framework has been developed making the interplay of both aspects explicit to facilitate social presence. In this ACMC framework, the displays can be arranged in virtual space manually or automatically. We expect that, according to empirical findings, the social relation as well as momentarily affective appraisals will influence this arrangement. The proposed concept extends from desktop devices to fully immersive Virtual Reality interfaces. | |
BibTeX:
@inproceedings{Pfeiffer:Social:2008,
author = {Thies Pfeiffer and Ipke Wachsmuth},
title = {Social Presence: The Role of Interpersonal Distances in Affective Computer-Mediated Communication},
booktitle = {Proceedings of the 11th. International Workshop on Presence},
publisher = {CLEUP Cooperativa Libraria Universitaria Padova},
year = {2008},
pages = {275--279}
}
|
|
|
|
Pfeiffer, T. (2008). Towards Gaze Interaction in Immersive Virtual Reality: Evaluation of a Monocular Eye Tracking Set-Up. In Virtuelle und Erweiterte Realität - Fünfter Workshop der GI-Fachgruppe VR/AR, 81-92. Aachen: Shaker Verlag GmbH. |
| Abstract: Of all senses, it is visual perception that is predominantly deluded in Virtual Realities. Yet, the eyes of the observer, despite the fact that they are the fastest perceivable moving body part, have gotten relatively little attention as an interaction modality. A solid integration of gaze, however, provides great opportunities for implicit and explicit human-computer interaction. We present our work on integrating a lightweight head-mounted eye tracking system in a CAVE-like Virtual Reality Set-Up and provide promising data from a user study on the achieved accuracy and latency. | |
BibTeX:
@inproceedings{Pfeiffer:Towards:2008,
author = {Thies Pfeiffer},
title = {Towards Gaze Interaction in Immersive Virtual Reality: Evaluation of a Monocular Eye Tracking Set-Up},
booktitle = {Virtuelle und Erweiterte Realität - Fünfter Workshop der GI-Fachgruppe VR/AR},
publisher = {Shaker Verlag GmbH},
year = {2008},
pages = {81--92}
}
|
|
|
|
Weiß, P., Pfeiffer, T., Schaffranietz, G. & Rickheit, G. (2008). Coordination in dialog: Alignment of object naming in the Jigsaw Map Game. In Cognitive Science 2007. Proceedings of the 8th Annual Conference of the Cognitive Science Society of Germany, 4-20. |
| Abstract: People engaged in successful dialog have to share knowledge, e.g., when naming objects, for coordinating their actions. According to Clark (1996), this shared knowledge, the common ground, is explicitly established, particularly by negotiations. Pickering and Garrod (2004) propose with their alignment approach a more automatic and resource-sensitive mechanism based on priming. Within the collaborative research center (CRC) “Alignment in Communication” a series of experimental investigations of natural face-to-face dialogs should bring about vital evidence to arbitrate between the two positions. This series should ideally be based on a common setting. In this article we review experimental settings in this research line and refine a set of requirements. We then present a flexible design called the Jigsaw Map Game and demonstrate its applicability by reporting on a first experiment on object naming. | |
BibTeX:
@inproceedings{Weiss:Coordination:2008,
author = {P. Weiß and T. Pfeiffer and G. Schaffranietz and G. Rickheit},
title = {Coordination in dialog: Alignment of object naming in the Jigsaw Map Game},
booktitle = {Cognitive Science 2007. Proceedings of the 8th Annual Conference of the Cognitive Science Society of Germany},
year = {2008},
pages = {4-20}
}
|
|
|
|
Pfeiffer, T., Donner, M., Latoschik, M.E. & Wachsmuth, I. (2007). 3D fixations in real and virtual scenarios. Journal of Eye Movement Research, Special issue: Abstracts of the ECEM 2007, 13. |
BibTeX:
@article{Pfeiffer:3D:2007,
author = {Thies Pfeiffer and Matthias Donner and Marc Erich Latoschik and Ipke Wachsmuth},
title = {3D fixations in real and virtual scenarios},
journal = {Journal of Eye Movement Research},
year = {2007},
volume = {Special issue: Abstracts of the ECEM 2007},
pages = {13}
}
|
|
|
|
Pfeiffer, T., Donner, M., Latoschik, M.E. & Wachsmuth, I. (2007). Blickfixationstiefe in stereoskopischen VR-Umgebungen: Eine vergleichende Studie. In Vierter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR, 113-124. Aachen, Germany: Shaker Verlag GmbH. |
| Abstract: Für die Mensch-Maschine-Interaktion ist die Erfassung der Aufmerksamkeit des Benutzers von großem Interesse. Für Anwendungen in der Virtuellen Realität (VR) gilt dies insbesondere, nicht zuletzt dann, wenn Virtuelle Agenten als Benutzerschnittstelle eingesetzt werden. Aktuelle Ansätze zur Bestimmung der visuellen Aufmerksamkeit verwenden meist monokulare Eyetracker und interpretieren daher auch nur zweidimensionale bedeutungstragende Blickfixationen relativ zu einer Projektionsebene. Für typische Stereoskopie-basierte VR Anwendungen ist aber eine zusätzliche Berücksichtigung der Fixationstiefe notwendig, um so den Tiefenparameter für die Interaktion nutzbar zu machen, etwa für eine höhere Genauigkeit bei der Objektauswahl (Picking). Das in diesem Beitrag vorgestellte Experiment zeigt, dass bereits mit einem einfacheren binokularen Gerät leichter zwischen sich teilweise verdeckenden Objekten unterschieden werden kann. Trotz des positiven Ergebnisses kann jedoch noch keine uneingeschränkte Verbesserung der Selektionsleistung gezeigt werden. Der Beitrag schließt mit einer Diskussion nächster Schritte mit dem Ziel, die vorgestellte Technik weiter zu verbessern. | |
BibTeX:
@inproceedings{Pfeiffer:Blickfixationstiefe:2007,
author = {Thies Pfeiffer and Matthias Donner and Marc Erich Latoschik and Ipke Wachsmuth},
title = {Blickfixationstiefe in stereoskopischen VR-Umgebungen: Eine vergleichende Studie},
booktitle = {Vierter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR},
publisher = {Shaker Verlag GmbH},
year = {2007},
pages = {113-124}
}
|
|
|
|
Pfeiffer, T. & Latoschik, M.E. (2007). Interactive Social Displays. In IPT-EGVE 2007, Virtual Environments 2007, Short Papers and Posters, 41-42: Eurographics Association. |
| Abstract: The mediation of social presence is one of the most interesting challenges of modern communication technology. The proposed metaphor of Interactive Social Displays describes new ways of interactions with multi-/crossmodal interfaces prepared for a psychologically augmented communication. A first prototype demonstrates the application of this metaphor in a teleconferencing scenario. | |
BibTeX:
@inproceedings{Pfeiffer:Interactive:2007,
author = {Thies Pfeiffer and Marc Erich Latoschik},
title = {Interactive Social Displays},
booktitle = {IPT-EGVE 2007, Virtual Environments 2007, Short Papers and Posters},
publisher = {Eurographics Association},
year = {2007},
pages = {41-42}
}
|
|
|
|
Pfeiffer, T. & Latoschik, M.E. (2007). Interactive Social Displays, Poster at the 3D UI 2007. |
| Abstract: The mediation of social presence is one of the most interesting challenges of modern communication technology. The proposed metaphor of Interactive Social Displays describes new ways of interactions with multi-/crossmodal interfaces prepared for a psychologically augmented communication. A first prototype demonstrates the application of this metaphor in a teleconferencing scenario. | |
BibTeX:
@misc{Pfeiffer:Interactive:2007a,
author = {Thies Pfeiffer and Marc E. Latoschik},
title = {Interactive Social Displays},
howpublished = {Poster at the 3D UI 2007},
year = {2007}
}
|
|
|
|
Pfeiffer, T. & Wachsmuth, I. (2007). Interpretation von Objektreferenzen in multimodalen Äußerungen. In Kognitionsforschung 2007 - Beiträge zur 8. Jahrestagung der Gesellschaft für Kognitionswissenschaft, 109-110, March. Aachen: Shaker Verlag. |
| Abstract: Im Rahmen der Entwicklung einer multimodalen Schnittstelle für die Mensch-Maschine Kommunikation konzentriert sich diese Arbeit auf die Interpretation von Referenzen auf sichtbare Objekte. Im Vordergrund stehen dabei Fragen zur Genauigkeit von Zeigegesten und deren Interaktion mit sprachlichen Ausdrücken. Die Arbeit spannt dabei methodisch einen Bogen von Empirie über Simulation und Visualisierung zur Modellbildung und Evaluation. In Studien zur deiktischen Objektreferenz wurden neben sprachlichen Äußerungen unter dem Einsatz moderner Motion Capturing Technik umfangreiche Daten zum deiktischen Zeigen erhoben. Diese heterogenen Daten, bestehend aus Tracking Daten, sowie Video und Audio Aufzeichnungen, wurden annotiert und mit eigens entwickelten interaktiven Werkzeugen unter Einsatz von Techniken der Virtuellen Realität integriert und aufbereitet. Die statistische Auswertung der Daten erfolgte im Anschluß mittels der freien Statistik-Software R. Die datengetriebene Modellbildung bildet die Grundlage für die Weiterentwicklung eines unscharfen, fuzzy-basierten, Constraint Satisfaction Ansatzes zur Interpretation von Objektreferenzen. Wesentliches Ziel ist dabei eine inkrementelle, echtzeitfähige Verarbeitung, die den Einsatz in direkter Mensch-Maschine Interaktion erlaubt. Die Ergebnisse der Studie haben über die Fragestellung hinaus Einfluss auf ein Modell zur Produktion von deiktischen Ausdrücken und direkte Konsequenzen für einschlägige Theorien zur deiktischen Referenz. | |
BibTeX:
@inproceedings{Pfeiffer:Interpretation:2007,
author = {Thies Pfeiffer and Ipke Wachsmuth},
title = {Interpretation von Objektreferenzen in multimodalen Äußerungen},
booktitle = {Kognitionsforschung 2007 - Beiträge zur 8. Jahrestagung der Gesellschaft für Kognitionswissenschaft},
publisher = {Shaker Verlag},
year = {2007},
pages = {109-110}
}
|
|
|
|
Schaffranietz, G., Weiß, P., Pfeiffer, T. & Rickheit, G. (2007). Ein Experiment zur Koordination von Objektbezeichnungen im Dialog. In Kognitionsforschung 2007 - Beiträge zur 8. Jahrestagung der Gesellschaft für Kognitionswissenschaft, 41-42, March. Aachen: Shaker Verlag. |
| Abstract: People engaged in successful dialog have to share knowledge, e.g., when naming objects, for coordinating their actions. According to Clark (1996), this shared knowledge, the common ground, is explicitly established, particularly by negotiations. Pickering and Garrod (2004) propose with their alignment approach a more automatic and resource-sensitive mechanism based on priming. Within the collaborative research center (CRC) “Alignment in Communication” a series of experimental investigations of natural face-to-face dialogs should bring about vital evidence to arbitrate between the two positions. This series should ideally be based on a common setting. In this article we review experimental settings in this research line and refine a set of requirements. We then present a flexible design called the Jigsaw Map Game and demonstrate its applicability by reporting on a first experiment on object naming. | |
BibTeX:
@inproceedings{Schaffranietz:Koordination:2007,
author = {Gesche Schaffranietz and Petra Weiß and Thies Pfeiffer and Gert Rickheit},
title = {Ein Experiment zur Koordination von Objektbezeichnungen im Dialog},
booktitle = {Kognitionsforschung 2007 - Beiträge zur 8. Jahrestagung der Gesellschaft für Kognitionswissenschaft},
publisher = {Shaker Verlag},
year = {2007},
pages = {41-42}
}
|
|
|
|
Flitter, H., Pfeiffer, T. & Rickheit, G. (2006). Psycholinguistic experiments on spatial relations using stereoscopic presentation. In Situated Communication, 127-153. Berlin: Mouton de Gruyter. |
| Abstract: This contribution presents investigations of the usage of computer gene¬rated 3D stimuli for psycholinguistic experiments. In the first part, we introduce VDesigner. VDesigner is a visual programming environment that operates in two different modes, a design mode to implement the materials and the structure of an experiment, and a runtime mode to actually run the experiment. We have extended VDesigner to support interactive experimentation in 3D. In the second part, we de-scribe a practical application of the programming environment. We have replicated a previous 2½D study of the production of spatial terms in a 3D setting, with the objective of investigating the effect of the presentation modes (2½D vs. 3D) on the choice of the referential system. In each trial, on being presented with a scene, the participants had to verbally specify the position of a target object in relation to a reference object. We recorded the answers of the participants as well as their reac-tion times. The results suggest that stereoscopic 3D presentations are a promising technology to elicit a more natural behavior of participants in computer-based experiments. | |
BibTeX:
@incollection{Flitter:Psycholinguistic:2006,
author = {H. Flitter and T. Pfeiffer and G. Rickheit},
title = {Psycholinguistic experiments on spatial relations using stereoscopic presentation},
booktitle = {Situated Communication},
publisher = {Mouton de Gruyter},
year = {2006},
pages = {127--153}
}
|
|
|
|
Kranstedt, A., Lücking, A., Pfeiffer, T., Rieser, H. & Wachsmuth, I. (2006). Deictic object reference in task-oriented dialogue. In Situated Communication, 155-207. Berlin: Mouton de Gruyter. |
| Abstract: This chapter presents an original approach towards a detailed understanding of the usage of pointing gestures accompanying referring expressions. This effort is undertaken in the context of human-machine interaction integrating empirical studies, theory of grammar and logics, and simulation techniques. In particular, we take steps to classify the role of pointing in deictic expressions and to model the focussed area of pointing gestures, the so-called pointing cone. This pointing cone serves as a central concept in a formal account of multi-modal integration at the linguistic speech-gesture interface as well as in a computational model of processing multi-modal deictic expressions. | |
BibTeX:
@incollection{Kranstedt:Deictic:2006,
author = {Alfred Kranstedt and Andy Lücking and Thies Pfeiffer and Hannes Rieser and Ipke Wachsmuth},
title = {Deictic object reference in task-oriented dialogue},
booktitle = {Situated Communication},
publisher = {Mouton de Gruyter},
year = {2006},
pages = {155-207}
}
|
|
|
|
Kranstedt, A., Lücking, A., Pfeiffer, T., Rieser, H. & Wachsmuth, I. (2006). Deixis: How to Determine Demonstrated Objects Using a Pointing Cone. In Gesture Workshop 2005, 300-311. Berlin Heidelberg: Springer-Verlag GmbH. |
| Abstract: We present a collaborative approach towards a detailed understanding of the usage of pointing gestures accompanying referring expressions. This effort is undertaken in the context of human-machine interaction integrating empirical studies, theory of grammar and logics, and simulation techniques. In particular, we attempt to measure the precision of the focussed area of a pointing gesture, the so-called pointing cone. The pointing cone serves as a central concept in a formal account of multi-modal integration at the linguistic speechgesture interface as well as in a computational model of processing multi-modal deictic expressions. | |
BibTeX:
@inproceedings{Kranstedt:Deixis::2006,
author = {Alfred Kranstedt and Andy Lücking and Thies Pfeiffer and Hannes Rieser and Ipke Wachsmuth},
title = {Deixis: How to Determine Demonstrated Objects Using a Pointing Cone},
booktitle = {Gesture Workshop 2005},
publisher = {Springer-Verlag GmbH},
year = {2006},
pages = {300--311},
doi = {http://dx.doi.org/10.1007/11678816_34}
}
|
|
|
|
Kranstedt, A., Lücking, A., Pfeiffer, T., Rieser, H. & Staudacher, M. (2006). Measuring and Reconstructing Pointing in Visual Contexts. In Proceedings of the brandial 2006 - The 10th Workshop on the Semantics and Pragmatics of Dialogue, 82-89. Potsdam: Universitätsverlag Potsdam. |
| Abstract: We describe an experiment to gather original data on geometrical aspects of pointing. In particular, we are focusing upon the concept of the pointing cone, a geometrical model of a pointing’s extension. In our setting we employed methodological and technical procedures of a new type to integrate data from annotations as well as from tracker recordings. We combined exact information on position and orientation with rater’s classifications. Our first results seem to challenge classical linguistic and philosophical theories of demonstration in that they advise to separate pointings from reference. | |
BibTeX:
@inproceedings{Kranstedt:Measuring:2006,
author = {Alfred Kranstedt and Andy Lücking and Thies Pfeiffer and Hannes Rieser and Marc Staudacher},
title = {Measuring and Reconstructing Pointing in Visual Contexts},
booktitle = {Proceedings of the brandial 2006 - The 10th Workshop on the Semantics and Pragmatics of Dialogue},
publisher = {Universitätsverlag Potsdam},
year = {2006},
pages = {82-89},
url = {http://opus.kobv.de/ubp/volltexte/2006/1048/pdf/brandial06_proceedings.pdf}
}
|
|
|
|
Pfeiffer, T., Kranstedt, A. & Lücking, A. (2006). Sprach-Gestik Experimente mit IADE, dem Interactive Augmented Data Explorer. In Dritter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR, 61-72. Aachen: Shaker. |
| Abstract: Für die empirische Erforschung situierter natürlicher menschlicher Kommunikation sind wir auf die Akquise und Auswertung umfangreicher Daten angewiesen. Die Modalitäten, über die sich Menschen ausdrücken können, sind sehr unterschiedlich. Entsprechend heterogen sind die Repräsentationen, mit denen die erhobenen Daten für die Auswertung verfügbar gemacht werden können. Für eine Untersuchung des Zeigeverhaltens bei der Referenzierung von Objekten haben wir mit IADE ein Framework für die Aufzeichnung, Analyse und Simulation von Sprach-Gestik Daten entwickelt. Durch den Einsatz von Techniken aus der interaktiven VR erlaubt IADE die synchronisierte Aufnahme von Bewegungs-, Video- und Audiodaten und unterstützt einen iterativen Auswertungsprozess der gewonnenen Daten durch komfortable integrierte Revisualisierungen und Simulationen. Damit stellt IADE einen entscheidenden Fortschritt für unsere linguistische Experimentalmethodik dar. | |
BibTeX:
@inproceedings{Pfeiffer:Sprach-Gestik:2006,
author = {Thies Pfeiffer and Alfred Kranstedt and Andy Lücking},
title = {Sprach-Gestik Experimente mit IADE, dem Interactive Augmented Data Explorer},
booktitle = {Dritter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR},
publisher = {Shaker},
year = {2006},
pages = {61-72}
}
|
|
|
|
Weiß, P., Pfeiffer, T., Eikmeyer, H.-J. & Rickheit, G. (2006). Processing Instructions. In Situated Communication, 31-76. Berlin: Mouton de Gruyter. |
| Abstract: Instructions play an important role in everyday communication, e.g. in task-oriented dialogs. Based on a (psycho-)linguistic theoretical background, which classifies instructions as requests, we conducted experiments using a cross-modal experimental design in combination with a reaction time paradigm in order to get insights in human instruction processing. We concentrated on the interpretation of basic single sentence instructions. Here, we especially examined the effects of the specificity of verbs, object names, and prepositions in interaction with factors of the visual object context regarding an adequate reference resolution. We were able to show that linguistic semantic and syntactic factors as well as visual context information context influence the interpretation of instructions. Especially the context information proves to be very important. Above and beyond the relevance for basic research, these results are also important for the design of human-computer interfaces capable of understanding natural language. Thus, following the experimental-simulative approach, we also pursued the processing of instructions from the perspective of computer science. Here, a natural language processing interface created for a virtual reality environment served as basis for the simulation of the empirical findings. The comparison of human vs. virtual system performance using a local performance measure for instruction understanding based on fuzzy constraint satisfaction led to further insights concerning the complexity of instruction processing in humans and artificial systems. Using selected examples, we were able to show that the visual context has a comparable influence on the performance of both systems, whereas this approach is limited when it comes to explaining some effects due to variations of the linguistic structure. In order to get deeper insights into the timing and interaction of the sub-processes relevant for instruction understanding and to model these effects in the computer simulation, more specific data on human performance are necessary, e.g. by using eye-tracking techniques. In the long run, such an approach will result in the development of a more natural and cognitively adequate human-computer interface. | |
BibTeX:
@incollection{Weiss:Processing:2006,
author = {P. Weiß and T. Pfeiffer and H.-J. Eikmeyer and G. Rickheit},
title = {Processing Instructions},
booktitle = {Situated Communication},
publisher = {Mouton de Gruyter},
year = {2006},
pages = {31-76}
}
|
|
|
|
Weiß, P., Pfeiffer, T. & Rickheit, G. (2006). Situated processing of elliptical instructions: An eye-tracking study, Poster at the 12th Annual Conference on Architectures and Mechanisms for Language Processing. |
| Abstract: We are interested in the situated processing of complex instructions in task-oriented dialogs between humans and machines. For the human processing of single sentence instructions (e.g. “Connect the red bolt with the cube.”) we were able to show earlier that syntax and semantics, as well as visual context information contribute to interpretation (Weiß et al., 2006). These empirical findings could also be replicated in simulations with a natural language processing interface for a virtual reality environment. In these first experiments we used a cross-modal presentation technique in combination with the reaction time paradigm. Using reaction times alone, however, we could not get deeper insights into the timing and interaction of sub-processes relevant for human instruction understanding. For the experiment presented here, in which we studied the processing of more complex, elliptical instruction sequences, we therefore tracked the eyes of the participants over the visual context (visual world paradigm; Tanenhaus et al., 1995). In our experiment, 20 subjects were presented with short instruction sequences with and without N-ellipses in the second sentence: “Jetzt die blaue Leiste (1A) neben die gelbe Schraube (1B) legen. Dann kommt die grüne [Leiste] (2A) neben die rote Schraube (2B).” Participants had to pick the object referred to at 2A (green [bar] or [bolt]) on a computer display containing four objects, e.g. green and red bolt, green and yellow bar. The objects referred to by the first sentence were not shown on the display. As a second factor we varied the structural parallelism of both sentences with regard to the sequence of the phrases (AB.AB vs. BA.AB). Besides the picking, we also recorded eye movement data. Here we were interested in the trajectory of the fixations during the processing of the second sentence. Results show that, whereas the actions of the participants indicate an interpretation following the parallel syntactic structure, the eye fixations show that recency of mentioning plays an important role in early processing: In the case of elliptical instructions participants more frequently first fixated objects with the type referred to in the recent NP, but then picked the object with the type referred to in the syntactically parallel NP. We will present our results regarding a deeper insight in the influence of linguistic and structural factors on situated language processing and discuss implications for natural language processing interfaces. |
|
BibTeX:
@misc{Weiss:Situated:2006,
author = {Petra Weiß and Thies Pfeiffer and Gert Rickheit},
title = {Situated processing of elliptical instructions: An eye-tracking study},
howpublished = {Poster at the 12th Annual Conference on Architectures and Mechanisms for Language Processing},
year = {2006}
}
|
|
|
|
Pfeiffer, T. & Flitter, H. (2005). Psycholinguistic Experiments in 3D, Poster at the Interdisciplinary College 2005. |
BibTeX:
@misc{Pfeiffer:Psycholinguistic:2005,
author = {Thies Pfeiffer and Helmut Flitter},
title = {Psycholinguistic Experiments in 3D},
howpublished = {Poster at the Interdisciplinary College 2005},
year = {2005}
}
|
|
|
|
Pfeiffer, T., Weber, M. & Jung, B. (2005). Ubiquitous Virtual Reality: Accessing Shared Virtual Environments through Videoconferencing Technology. In Theory and Practice of Computer Graphics 2005, 209-216. Aire-la-Ville, Switzerland: Eurographics Association. |
| Abstract: This paper presents an alternative to existing methods for remotely accessing Virtual Reality (VR) systems. Common solutions are based on specialised software and/or hardware capable of rendering 3D content, which not only restricts accessibility to specific platforms but also increases the barrier for non expert users. Our approach addresses new audiences by making existing Virtual Environments (VEs) ubiquitously accessible. Its appeal is that a large variety of clients, like desktop PCs and handhelds, are ready to connect to VEs out of the box. We achieve this combining established videoconferencing protocol standards with a server based interaction handling. Currently interaction is based on natural speech, typed textual input and visual feedback, but extensions to support natural gestures are possible and planned. This paper presents the conceptual framework enabling videoconferencing with collaborative VEs as well as an example application for a virtual prototyping system. | |
BibTeX:
@inproceedings{Pfeiffer:Ubiquitous:2005,
author = {Thies Pfeiffer and Matthias Weber and Bernhard Jung},
title = {Ubiquitous Virtual Reality: Accessing Shared Virtual Environments through Videoconferencing Technology},
booktitle = {Theory and Practice of Computer Graphics 2005},
publisher = {Eurographics Association},
year = {2005},
pages = {209--216}
}
|
|
|
|
Weber, M., Pfeiffer, T. & Jung, B. (2005). Pr@senZ - P@CE: Mobile Interaction with Virtual Reality. In MOBILE HCI 05 Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices and Services, 351-352, September. New York: ACM Press. |
| Abstract: Recently videoconferencing has been extended from human face-to-face communication to human machine interaction with Virtual Environments (VE)[6]. Relying on established videoconferencing (VC) protocol standards this thin client solution does not require specialised 3D soft- or hardware and scales well to multimedia enabled mobile devices. This would bring a whole range of new applications to the mobile platform. To facilitate our research in mobile interaction the Open Source project P@CE has been started to bring a fullfeatured VC client to the Pocket PC platform. | |
BibTeX:
@inproceedings{Weber:MobileVR:2005,
author = {Matthias Weber and Thies Pfeiffer and Bernhard Jung},
title = {Pr@senZ - P@CE: Mobile Interaction with Virtual Reality},
booktitle = {MOBILE HCI 05 Proceedings of the 7th International Conference on Human Computer Interaction with Mobile Devices and Services},
publisher = {ACM Press},
year = {2005},
pages = {351--352}
}
|
|
|
|
Pfeiffer, T. & Latoschik, M.E. (2004). Resolving Object References in Multimodal Dialogues for Immersive Virtual Environments. In Proceedings of the IEEE Virtual Reality 2004, 35-42. |
| Abstract: This paper describes the underlying concepts and the technical implementation of a system for resolving multimodal references in Virtual Reality (VR). In this system the temporal and semantic relations intrinsic to referential utterances are expressed as a constraint satisfaction problem, where the propositional value of each referential unit during a multimodal dialogue updates incrementally the active set of constraints. As the system is based on findings of human cognition research it also regards, e.g., constraints implicitly assumed by human communicators. The implementation takes VR related real-time and immersive conditions into account and adapts its architecture to well known scene-graph based design patterns by introducing a socalled reference resolution engine. Regarding the conceptual work as well as regarding the implementation, special care has been taken to allow further refinements and modifications to the underlying resolving processes on a high level basis. | |
BibTeX:
@inproceedings{Pfeiffer:Resolving:2004,
author = {T. Pfeiffer and M. E. Latoschik},
title = {Resolving Object References in Multimodal Dialogues for Immersive Virtual Environments},
booktitle = {Proceedings of the IEEE Virtual Reality 2004},
year = {2004},
pages = {35--42}
}
|
|
|
|
Weiß, P., Pfeiffer, T. & Allmaier, K. (2004). Blickbewegungsmessungen bei der Verarbeitung elliptischer Konstruktionsanweisungen in situierter Kommunikation, Poster at the 44. Kongress der Deutschen Gesellschaft für Psychologie: 100 Jahre Deutsche Gesellschaft für Psychologie. |
BibTeX:
@misc{Weiss:Blickbewegungsmessungen:2004,
author = {Petra Weiß and Thies Pfeiffer and Kirsten Allmaier},
title = {Blickbewegungsmessungen bei der Verarbeitung elliptischer Konstruktionsanweisungen in situierter Kommunikation},
howpublished = {Poster at the 44. Kongress der Deutschen Gesellschaft für Psychologie: 100 Jahre Deutsche Gesellschaft für Psychologie},
year = {2004}
}
|
|
|
|
Pfeiffer, T. & Voss, I. (2003). Integrating Knowledge Bases Using UML Metamodels, Poster at SCAI 2003. November. |
| Abstract: When merging different knowledge bases one has to cope with the problem of classifying and linking concepts as well as the possibly heterogeneous representations the knowledge is expressed in. We are presenting an implementation that follows the Model Driven Architecture (MDA) [Miller and Mukerji, 2003] approach defined by the Object Management Group (OMG). Metamodels defined in the Unified Modeling Language (UML) are used to implement different knowledge representation formalisms. Knowledge is expressed as a Model instantiating the Metamodel. Integrating Metamodels are defined for merging knowledge distributed over different knowledge bases. | |
BibTeX:
@misc{Pfeiffer:Integrating:2003,
author = {Thies Pfeiffer and Ian Voss},
title = {Integrating Knowledge Bases Using UML Metamodels},
howpublished = {Poster at SCAI 2003},
year = {2003}
}
|
|
|
|
Pfeiffer, T. (2003). Eine Referenzauflösung für die dynamische Anwendung in Konstruktionssituationen in der Virtuellen Realität, School: Faculty of Technology, University of Bielefeld,. |
BibTeX:
@mastersthesis{Pfeiffer:Referenzaufloesung:2003,
author = {T. Pfeiffer},
title = {Eine Referenzauflösung für die dynamische Anwendung in Konstruktionssituationen in der Virtuellen Realität},
school = {Faculty of Technology, University of Bielefeld},
year = {2003}
}
|
|
|
|
Pfeiffer, T., Voss, I. & Latoschik, M.E. (2003). Resolution of Multimodal Object References using Conceptual Short Term Memory. In Proceedings of the EuroCogSci03, 426. Mahwah, New Jersey: Lawrence Erlbaum Associates Inc. |
| Abstract: This poster presents cognitive-motivated aspects of a technical system for the resolution of references to objects within an assembly-task domain. The research is integrated in the Collaborative Research Center SFB 360 which is concerned with situated artificial communicators. One application scenario consists of a task-oriented discourse between an instructor and a constructor who collaboratively build aggregates from a wooden toy kit (”Baufix”), or from generic CAD parts. In our current setting this scenario is embedded in a virtual reality (VR) installation, where the human user, taking the role of the instructor, guides the artificial constructor (embodied by the ECA Max) through the assembly process by means of multimodal task descriptions (see Figure 1). The system handles instructions like: ”Plug the left red screw from above in the middle hole of the wing and turn it this way.” accompanied by coverbal deictic and mimetic gestures (see Latoschik, 2001). |
|
BibTeX:
@inproceedings{Pfeiffer:Resolution:2003,
author = {T. Pfeiffer and I. Voss and M. E. Latoschik},
title = {Resolution of Multimodal Object References using Conceptual Short Term Memory},
booktitle = {Proceedings of the EuroCogSci03},
publisher = {Lawrence Erlbaum Associates Inc},
year = {2003},
pages = {426}
}
|
|
|
|
Jung, B., Pfeiffer, T. & Zakotnik, J. (2002). Natural Language Based Virtual Prototyping on the Web. In Proceedings Structured Design of Virtual Environments and 3D-Components, 101-110. Aachen: Shaker. |
| Abstract: This contribution describes a WWW-based multi-user system for concurrent virtual prototyping. A 3D scene of CAD parts is presented to the users in the web browser. By instructing the system using simple natural language commands, complex aggregates can be assembled from the basic parts. The current state of the assembly is instantly published to all system users who can discuss design choices in a chat area. The implementation builds on an existing system for virtual assembly made available as a web service. The client side components are fully implemented as Java applets and require no plugin for visualization of 3D content. Http tunneled messaging between web clients and server ensures system accessibility from any modern web browser even behind firewalls. The system is first to demonstrate natural language based virtual prototyping on the web. |
|
BibTeX:
@inproceedings{Jung:Natural:2002,
author = {B. Jung and T. Pfeiffer and J. Zakotnik},
title = {Natural Language Based Virtual Prototyping on the Web},
booktitle = {Proceedings Structured Design of Virtual Environments and 3D-Components},
publisher = {Shaker},
year = {2002},
pages = {101-110}
}
|
Dr. Thies Pfeiffer