Smartkom

by Wahlster, Wolfgang

• ISBN13: 9783540237327
• ISBN10: 3540237321
• Format: Hardcover
• Copyright: 2006-08-30
• Publisher: Springer-Verlag New York Inc

Summary

The result of four years of intensive research in a large multimodal dialogue project involving 12 partners from academia and industry, SmartKom is one of the most advanced multimodal dialogue systems worldwide and is a landmark project in the history of intelligent user interfaces. The system provides symmetric multimodality in a mixed-initiative dialogue system with an embodied conversational agent. The same software architecture and components are used in three fully operational application scenarios. The theoretical and practical foundations of SmartKom represent a new generation of multimodal dialogue systems that deal not only with simple modality integration and synchronization, but cover the full spectrum of multimodal dialogue. With contributions by leading scientists in the field, this book gives the first comprehensive overview of the results of this seminal project.

Author Biography

Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster is the Director and CEO of the German Research Center for Artificial Intelligence (DFKI GmbH) and a Professor of Computer Science at the Universit+ñt des Saarlandes, Saarbr++cken. In 2000, he was coopted as a Professor of Computational Linguistics at the same university. In addition, he is the Head of the Intelligent User Interfaces Lab at DFKI.He was the Scientific Director of the VERBMOBIL consortium on spontaneous speech translation (1993-2000) as well as the SmartKom consortium on multimodal dialog systems (1999-2003) and currently serves as the Scientific Director of the SmartWeb consortium on mobile multimodal access to semantic web services (2004-2008). He has published more than 150 technical papers and 6 books on language technology and intelligent user interfaces. His current research includes multimodal and perceptive user interfaces, user modeling, embodied conversational agents, smart navigation systems, semantic web services, and resource-adaptive cognitive technologies. He is the editor of the book "Verbmobil: Foundations of Speech-to-Speech Translation" (Springer) and the Co-Editor of the Readings in Intelligent User Interfaces.

Table of Contents

Part I Introduction
Dialogue Systems Go Multimodal: The SmartKom Experience
Wolfgang Wahlster	3	(26)
Facts and Figures About the SmartKom Project
Anselm Blocher	29	(12)
An Exemplary Interaction with SmartKom
Norbert Reithinger, Gerd Herzog	41	(14)
Part II Multimodal Input Analysis
The SmartKom Architecture: A Framework for Multimodal Dialogue Systems
Gerd Herzog, Norbert Reithinger	55	(16)
Modeling Domain Knowledge: Know-How and Know-What
Iryna Gurevych, Robert Porzel, Rainer Malaka	71	(14)
Speech Recognition
André Berton, Alfred Kaltenmeier, Udo Haiber, Olaf Schreiner	85	(24)
Class-Based Language Model Adaptation
Martin C. Emele, Zica Valsan, Yin Hay Lam, Silke Goronzy	109	(14)
The Dynamic Lexicon
Silke Goronzy, Stefan Rapp, Martin Emele	123	(16)
The Prosody Module
Viktor Zeißler, Johann Adelhardt, Anton Batliner, Carmen Frank, Elmar Nöth, Rui Ping Shi, Heinrich Niemann	139	(14)
The Sense of Vision: Gestures and Real Objects
Jens Racky, Michael Lützeler, Hans Röttger	153	(14)
The Facial Expression Module
Carmen Frank, Johann Adelhardt, Anton Batliner, Elmar Nöth, Rui Ping Shi, Viktor Zeißlen Heinrich Niemann	167	(14)
Multiple Biometrics
Stephan Grashey, Matthias Schuster	181	(14)
Natural Language Understanding
Ralf Engel	195	(14)
The Gesture Interpretation Module
Rui Ping Shi, Johann Adelhardt, Anton Batliner, Carmen Frank, Elmar Nöth, Viktor Zeßler, Heinrich Niemann	209	(14)
Part DI Multimodal Dialogue Processing
Modality Fusion
Ralf Engel, Norbert Pfleger	223	(14)
Discourse Modeling
Jan Alexandersson, Norbert Pfleger	237	(18)
Overlay: The Basic Operation for Discourse Processing
Jan Alexandersson, Tilman Becker, Norbert Pfleger	255	(14)
In Context: Integrating Domain- and Situation-Specific Knowledge
Robert Porzel, Irma Gurevych, Rainer Malaka	269	(16)
Intention Recognition
Jürgen te Vrugt, Thomas Portele	285	(16)
Plan-Based Dialogue Management for Multiple Cooperating Applications
Markus Löckelt	301	(16)
Emotion Analysis and Emotion-Handling Subdialogues
Michaei Streit, Anton Batliner, Thomas Portele	317	(16)
Problematic, Indirect, Affective, and Other Nonstandard Input Processing
Michael Streit	333	(16)
Part IV Multimodal Output Generation
Realizing Complex User Wishes with a Function Planning Module
Sunna Torge, Christian Hying	349	(14)
Intelligent Integration of External Data and Services into SmartKom
Hidir Aras, Vasu Chandrasekhara, Sven Krüger, Rainer Malaka, Robert Porzel	363	(16)
Multimodal Fission and Media Design
Peter Poller, Valentin Tschernomas	379	(22)
Natural Language Generation with Fully Specified Templates
Tilman Becker	401	(10)
Multimodal Speech Synthesis
Antje Schweitzer, Norbert Braunschweiler, Grzegorz Dogil, Tanja Klankert, Bernd Möbius, Gregor Möhler, Edmilson Morais, Bettina Säuberlich, Matthias Thomae	411	(28)
Part V Scenarios and Applications
Building Multimodal Dialogue Applications: System Integration in SmartKom
Gerd Herzog, Alassane Ndiaye	439	(14)
SmartKom-English: From Robust Recognition to Felicitous Interaction
David Gelbart, John Bryant, Andreas Stolcke, Robert Porzel, Manja Baudis and Nelson Morgan	453	(18)
SmartKom-Public
Axel Horndasch, Horst Rapp, Hans Röttger	471	(22)
SmartKom-Home: The Interface to Home Entertainment
Thomas Portele, Silke Goronzy, Martin Emele, Andreas Kellner, Sunna Torge, Jürgen te Vrugt	493	(12)
SmartKom-Mobile: Intelligent Interaction with a Mobile System
Rainer Malaka, Jochen Häußler, Hidir Aras, Matthias Merdes, Dennis Pfisterer, Matthias Jöst, and Robert Porzel	505	(18)
SmartKom-Mobile Car: User Interaction with Mobile Services in a Car Environment
André Berton, Dirk Bühler, Wolfgang Minker	523	(18)
Part VI Data Collection and Evaluation
Wizard-of-Oz Recordings
Florian Schiel, Ulli Türk	541	(30)
Annotation of Multimodal Data
Silke Steininger, Florian Schiel, Susen Rabold	571	(26)
Multimodal Emogram, Data Collection and Presentation
Johann Adelhardt, Carmen Frank, Elmar Nöth, Rui Ping Shi, Viktor Zeißler, Heinrich Niemann	597	(6)
Empirical Studies for Intuitive Interaction
Iryna Gurevych, Robert Porzel	603	(14)
Evaluation of Multimodal Dialogue Systems
Florian Schiel	617