Universität Bielefeld - Technische Fakultät
Termin: Do. 16-18, T2-213
Beginn: 19.10.2000
Veranstalter: Timo Sowa, Ipke Wachsmuth
19.10. Einführung/Vorbesprechung
26.10. Laborführung; Treffen vor M4-112
2.11. Beispiele multimodaler Systeme: XTRA, AIMI, Alfresco, CUBRICON, GEORAL, SCHOLAR
Jan Kleinlützum und Henning Mersch - Handout (PDF) - Folien (PDF)
9.11. Beispiele multimodaler Systeme: XTRA, AIMI, Alfresco, CUBRICON, GEORAL, SCHOLAR
Henning Mersch und Jan Kleinlützum
16.11. Multimodale Integration am Beispiel von MATIS und VIENA
Ilker Savas - Handout (PDF) - Folien (PDF)
23.11. Evaluation multimodaler Systeme: CARE, TYCOON
Daniel Naber - Handout (PDF) - Folien (PDF)
30.11. Gerätetechnik für multimodale Systeme
Daniel Weßling - Handout (PDF)
7.12. Anmerkungen zur Anfertigung einer Ausarbeitung
14.12. fällt aus/nach Absprache (Vortrag verlegt auf 8.2.)
21.12. Multimodale Präsentationsplanung: COMET, WIP/PPP
Nadine Boley - Handout (PDF)
4.1. fällt aus
11.1. Robustheit durch Multimodalität: INTERACT
Hendrik Hasenbein - Handout (PDF) - Folien (PDF)
18.1. "3D"-Gestenerkennung und Sprache: ICONIC
Marco Balke - Handout (PDF) - Folien (PDF)
25.1. Anthropomorphe Agenten
Lutz Krause - Handout (PDF) - Folien (PDF)
1.2. Talking Heads: Sprache und Grafik synchronisiert
Jan Paller - Handout (PDF) - Folien (PDF)
8.2. Stiftbasierte Gestik und Sprache: Multimodal Maps (ursprünglich 14.12. geplant)
Arben Shoshi
15.2. Stiftbasierte Gestik und Sprache: QuickSet / Abschlußdiskussion
Timo Altenseuer
Die Kennzeichnung nach jeder Literaturreferenz gibt an, wo der Artikel/das Buch
zu finden ist. Dabei bedeutet:
[BIB] Unibibliothek
[SA] Semesterapparat Wachsmuth (FB 10)
[HA] Handapparat 311 (M4-126)
[T] bei Timo (M4-128)
Elektronisch verfügbare Dokumente sind mit einem Link
(Download) versehen.
Neal, J.G., Shapiro, S.C.: Intelligent Multi-Media Interface Technology. In Sullivan, J.W. & Tyler, S.W. (Eds.): Intelligent User Interfaces (pp. 11-43). Reading (MA): ACM Press, 1991. [SA,HA]
Wahlster, W.: User and Discourse Models for Multimodal Communication. In Sullivan, J.W. & Tyler, S.W. (Eds.): Intelligent User Interfaces (pp. 45-67). Reading (MA): ACM Press, 1991. [SA,HA]
Burger, J.D., Marshall, R.J.: The Application of Natural Language Models to Intelligent Multimedia. In Maybury, M.T. (Ed.): Intelligent Multimedia Interfaces (pp. 174-196). Menlo Park (CA): AAAI Press/The MIT Press, 1993. [SA]
Stock, O., et al.: Alfresco: Enjoying the Combination of Natural Language Processsing and Hypermedia for Information Exploration. In Maybury, M.T. (Ed.): Intelligent Multimedia Interfaces (pp. 197-224). Menlo Park (CA): AAAI Press/The MIT Press, 1993. [SA]
Siroux, J., et al.: Modeling and Processing of Oral and Tactile Activities in the GEORAL System. In Bunt, H. et al. (Eds.): Multimodal Human-Computer Communication (pp. 101-110). Berlin, Heidelberg: Springer-Verlag, 1998. [BIB,T]
Bos, E., Huls, C., Claassen, W.: EDWARD: Full Integration of Language and Action in a Multimodal Interface. International Journal of Human-Computer Studies 40, 1994, 473-495. [BIB]
Pouteau, X.: Interpretation of Gesture and Speech: A Practical Approach to Multimodal Communication. In Bunt, H., et al. (Eds.): Proceedings of the Second International Conference on Cooperative Multimodal Communication (pp. 142-154), Tilburg, Jan. 1998. [T]
Carbonell, J.R.: Mixed-Initiative Man-Computer Dialogues. BBN (Bolt, Beranek and Newman) Report No. 1971. Cambridge (MA), 1970.
Nigay, L., Coutaz, J.: A Generic Platform for Adressing the Multimodal Challenge. In Katz, I.R., et al. (Eds.): Human Factors in Computing Systems: CHI' 95 Conference Proceedings (pp. 98-105). New York: ACM Press, 1995. [T]
Lenzmann, B.: Benutzeradaptive und multimodale Interface-Agenten. Dissertationen zur Künstlichen Intelligenz. Sankt Augustin: Infix, 1998. [BIB]
Nigay, L., Coutaz, J., Salber, D.: Multimodality from the User and System Perspectives. In Stephanidis, C. (Ed.): Proceedings of the 1st ERCIM Workshop on Multimedia Multimodal Interfaces, Heraklion (Greece), October 1995. Download PDF
Multifeature Systems: The CARE Properties and Their Impact on Software Design. In Lee, J. (Ed.): Intelligence and Multimodality in Multimedia Interfaces: Research and Applications. Menlo Park: AAAI Press, 1995. (Elektronisches Buch!) Download PDF
Coutaz, J., et al.: Four easy pieces for assessing the usability of multimodal interaction: The CARE properties. In Nordby, K., et al. (Eds.): Proceedings of the Interact '95, Human-Computer Interaction Conference (pp. 115-120). London: Chapman and Hall, 1995.
Martin, J.-C., Julia, L., Cheyer, A.: A Theoretical Framework for Multimodal User Studies. In Bunt, H., et al. (Eds.): Proceedings of the Second International Conference on Cooperative Multimodal Communication (pp. 104-110), Tilburg, Jan. 1998. [T]
Martin, J.-C.: TYCOON: Theoretical Framework and Software Tools for Multimodal Interfaces. In Lee, J. (Ed.): Intelligence and Multimodality in Multimedia Interfaces: Research and Applications. Menlo Park: AAAI Press, 1995. (Elektronisches Buch!) Download PDF
Jacob, R.J.K.: Human-computer interaction input devices. ACM Comput. Surv. 28, 1 (1996), pp. 177-179. Download PDF
Jacob, R.J.K.: The future of input devices. ACM Comput. Surv. 28, 4 (1996), Article 138. HTML
Hinckley, K., Sinclair, M.: Touch-sensing input devices. Proceedings of the CHI`99 conference on Human factors in computing systems: the CHI is the limit (pp. 223-230), 1999. Download PDF
Ware, C.: Multimedia output devices and techniques. ACM Comp. Surv. 28, 1 (1996), pp. 181-183. Download PDF
Münch, S., Dillmann, R.: Haptic output in multimodal user interfaces. Proceedings of the 1997 international conference on Intelligent user interfaces (pp. 105-112), 1997. Download PDF
Burdea, G., et al.: Multimodal Virtual Reality: Input-Output Devices, System Integration, and Human Factors. International Journal of Human-Computer Interaction 8(1), 1996, 5-24. [T]
Cohen, P.R., et al.: QuickSet: Multimodal Interaction for Distributed Applications. In: Proceedings of the 5th Annual International Multimodal Conference (pp. 31-40). Reading (MA): ACM Press, 1997.
Johnston, M.: Multimodal Unification-based Grammars. In Ali, S.S., McRoy, S.W. (Eds.): Representations for Multi-modal Human-Computer Interaction. Working Notes of the AAAI 1998 Workshop, Madison, Wisconsin (pp. 31-32). [T]
Weitere Papers und Infos sind auf der QuickSet-Homepage zu finden.
Cheyer, A., Julia, L.: Multimodal Maps: An Agent-Based Approach. In Bunt, H. et al. (Eds.): Multimodal Human-Computer Communication (pp. 111-121). Berlin, Heidelberg: Springer-Verlag, 1998. [BIB,T]
Moran, D., et al.: Multimodal User Interfaces in the Open Agent Architecture. Proceedings of the Conference on Intelligent User Interfaces (IUI), 1997, Orlando. Download PS
Weitere Papers und Infos sind auf der MMaps-Homepage zu finden.
Feiner, S.K., et al.: Towards Coordinated Temporal Multimedia Presentations. In Maybury, M.T. (Ed.): Intelligent Multimedia Interfaces (pp. 139-147). Menlo Park (CA): AAAI Press/The MIT Press, 1993. [SA]
Feiner, S.K., McKeown, K.R.: Automating the Generation of Coordinated Multimedia Explanations. In Maybury, M.T., Wahlster, W. (Eds.): Readings in Intelligent User Interfaces (pp. 89-98). San Francisco: Morgan Kaufmann Publishers, 1998. [SA]
Andre, E., Rist, T..: The Design of Illustrated Documents as a Planning Task. In Maybury, M.T. (Ed.): Intelligent Multimedia Interfaces (pp. 94-116). Menlo Park (CA): AAAI Press/The MIT Press, 1993. [SA]
Wahlster, W., et al.: Plan-Based Integration of Natural Language and Graphics Generation. In Maybury, M.T., Wahlster, W. (Eds.): Readings in Intelligent User Interfaces (pp. 109-130). San Francisco: Morgan Kaufmann Publishers, 1998. [SA]
Rist, T., et al.: Adding Animated Presentation Agents to the Interface. In: Proceedings of IUI '97: Int. Conf. on Intelligent User Interfaces, pp. 79-86, Orlando, FL, 1997. Download PS (zip)
Weitere Papers und Infos zu PPP/WWW-Persona sind über die DFKI-Homepage (Suchen nach PPP) zu finden.
Waibel, A., et al.: Multimodal Interfaces. Artificial Intelligence Review 10, 1996, 299-319. Download PS (zip)
Weitere Papers und Infos sind auf der INTERACT Homepage zu finden.
Koons, D.B., et al.: Integrating Simultaneous Input from Speech, Gaze, and Hand Gestures. In Maybury, M.T. (Ed.): Intelligent Multimedia Interfaces (pp. 257-276). Menlo Park (CA): AAAI Press/The MIT Press, 1993. [SA]
Sparrell, C.J., Koons, D.B.: Interpretation of Coverbal Depictive Gestures. AAAI Spring Symposium 1994, pp. 8-12, Stanford University. [T]
Wexelblat, A.D.: An Approach to Natural Gesture in Virtual Environments. ACM Transactions on Computer-Human Interaction 2(3), 1995, 179-200. Download PDF
Kapitel 5 des ergänzenden Überblickstextes (s.o.): Speech Output with Talking Heads
Massaro, D.W., et al.: Developing and Evaluating Conversational Agents. In Cassell, J., et al. (Eds.): Embodied Conversational Agents (pp. 287-318). Cambridge (MA): The MIT Press, 2000. [T]
Informationen zu den Projekten über "multimodale Sprachsynthese" am "Center for Speech Technology" der KTH Stockholm sind hier zu finden. Insbesondere das August-Projekt ist von Interesse. Auf den Webseiten sind auch entsprechende Papers zu finden.
Cassell, J.: More than Just Another Pretty Face: Embodied Conversational Interface Agents. Communications of the ACM 43(4), 2000, 70-78. Download PDF
Cassell, J., et al.: Animated Conversation: Rule-based Generation of Facial Expression, Gesture & Spoken Intonation for Multiple Conversational Agents. In Maybury, M.T., Wahlster, W. (Eds.): Readings in Intelligent User Interfaces (pp. 582-589). San Francisco: Morgan Kaufmann Publishers, 1998. [SA]
Thorisson, K.: Communicative Humanoids - A Computational Model of Psychosocial Dialogue Skills. Ph.D. Thesis, Massachusetts Institute of Technology, 1996. [T]
Thorisson, K.: A Mind Model for Multimodal Communicative Creatures & Humanoids. International Journal of Applied Artificial Intelligence 13(4-5), 1999, 449-486. Download PDF
Ein Einblick in die laufenden und abgeschlossenen Projekte zum "Conversational Humanoid" der Gruppe "Gesture & Narrative Language" am MIT Media Lab ist hier zu finden.
Weitere Infos zu Gandalf gibt es hier.