Multimodal Corpor

by Kipp, Michael; Martin, Jean-claude; Paggio, Patrizia; Heylen, Dirk

• ISBN13: 9783642047923
• ISBN10: 3642047920
• Edition: 1st
• Format: Paperback
• Copyright: 2009-10-01
• Publisher: Springer-Verlag New York Inc

Summary

This state-of-the-art-survey documents the scientific outcome of the International Workshop on "Multimodal Corpora: From Models of Natural Interaction to Systems and Applications", held in conjunction with the 6th International Conference for Language Resources and Evaluation (LREC) on May 27, 2008, in Marrakech, Morocco. It presents an introduction to, as well as current research from fields as diverse as psychology, artificial intelligence, robotics, signal processing, computational linguistics and human-computer interaction.Empirically based research in multimodality starts with collecting video data relevant for a specific research question, this data must then be enriched by human coders, who annotate the corpus and who also explore the potential interconnections between modalities (e.g. face, gesture, speech, posture etc.). The data must then be validated for consistency using quantitative measures - systematic analysis of the observed multimodal behaviors is the central characteristic of multimodal corpora. For analysis and modeling tasks, exploratory qualitative and quantitative analysis tools are needed for browsing, viewing, extraction and modeling. All the time the intercultural dimension must be taken into account, since multimodal interaction is always embedded in a social setting.The 6 selected workshop contributions are complemented by 7 invited research articles, including contributions from major international multimodal corpus projects like AMI and SmartWeb. All papers underwent a special review process for this volume, resulting in significant revisions and extensions based on the experts' advice.

Table of Contents

Collecting and Distributing Multimodal Corpora
The SmartWeb Corpora: Multimodal Access to the Web in Natural Environments	p. 1
Promoting free Dialog Video Corpora: The IFADV Corpus Example	p. 18
Coding and Analyzing Multimodal Corpora
Creating and Exploiting Multimodal Annotated Corpora: The ToMA Project	p. 38
Multi-track Annotation of Child Language and Gestures	p. 54
Gesture, Gaze and Persuasive Strategies in Political Discourse	p. 73
Unsupervised Clustering in Multimodal Multiparty Meeting Analysis	p. 93
Validating Multimodal Corpora
Multimodal Corpora Annotation: Validation Methods to Assess Coding Scheme Reliability	p. 109
on the Contextual Analysis of Agreement Scores	p. 122
The Intercultural Dimension of Multimodal Corpora
Creating Standardized Video Recordings of Multimodal Interactions across Cultures	p. 138
Multimodal Intercultural Information and Communication Technology-A Framework for Designing and Evaluating Multimodal Intercultural Communicators	p. 160
Tools for Browsing, Coding and Exchanging Data in Multimodal Corpora
Tools and Resources for Visualising Conversational-Speech Interaction	p. 176
Accessing a Large Multimodal Corpus Using an Automatic Content Linking Device	p. 189
An Exchange Format for Multimodal Annotations	p. 207
Author Index	p. 223
Table of Contents provided by Ingram. All Rights Reserved.