Multimedia Semantics Metadata, Analysis and Interaction

by Troncy, Raphael; Huet, Benoit; Schenk, Simon

• ISBN13: 9780470747001
• ISBN10: 0470747005
• Edition: 1st
• Format: Hardcover
• Copyright: 2011-08-22
• Publisher: Wiley-Blackwell

Summary

Multimedia Semantics: Metadata, Analysis and Interaction explains, collects and reports on the latest research results that aim at narrowing the so-called multimedia ;Semantic Gap ;; the large disparity between descriptions of multimedia content that can be computed automatically and the richness and subjectivity of semantics in user queries and human interpretations of audiovisual media. Addressing the grand challenge posed by the ;Semantic Gap ; requires a multi-disciplinary approach (computer science, cognitive science, web science, etc.) and this is reflected in recent research in this area. This book is thus targeting an interdisciplinary community and in particular the Multimedia and the Semantic Web communities. A growing community of researchers is now pursuing joint approaches in various high-profile projects across the globe. However, it remains difficult for both sides of the divide to communicate and learn from each other. This book aims to provide both the fundamental knowledge and the latest state-of-the-art results from both communities with the goal of making the knowledge of one community understandable to the other.   This book is divided into four parts: Part A (Use Cases Scenarios, Canonical Processes of Semantically Annotated Media Production, Feature Extraction for Multimedia Analysis, Machine Learning Techniques for Multimedia Analysis, Semantic Web Basics, Semantic Web Languages), Part B (Multimedia Metadata Standards, COMM: A Core Ontology for Multimedia, Multimedia Ontology and Domain-Specific Ontologies), Part C (Knowledge Driven Segmentation and Classification, Reasoning for Multimedia Analysis, Large Scale Detection of Semantic Concepts in Multimedia) and Part D (Multimedia Annotation Tools, Multimedia Retrieval, Multimedia Browsing.)

Author Biography

Dr. Raphaël Troncy, Centre for Mathematics and Computer Science, Netherlands
Raphaël Troncy obtained his Master's thesis with honours in computer science at the University Joseph Fourier of Grenoble, France. He received his PhD with honours in 2004. His research interests include Semantic Web and Multimedia Technologies, Knowledge Representation, Ontology Modeling and Alignment. Raphaël Troncy is an expert in audio visual metadata and in combining existing metadata standards (such as MPEG-7) with current Semantic Web technologies.

Dr. Benoit Huet, Institut EURECOM, France
Benoit Huet received his BSc degree in computer science and engineering from the Ecole Superieure de Technologie Electrique (Groupe ESIEE, France) in 1992. In 1993, he was awarded the MSc degree in Artificial Intelligence from the University of Westminster (UK) with distinction. He received his PhD degree in Computer Science from the University of York (UK). His research interests include computer vision, content-based retrieval, multimedia data mining and indexing (still and/or moving images) and pattern recognition.

Simon Schenk, University of Koblenz-Landau, Germany
Simon Schenk is a research and teaching assistant at the Information Systems and Semantic Web Group of University of Koblenz-Landau.Simon is working towards his PhD degree under the supervision of Professor Dr. Steffen Staab. Previously, he has worked as a consultant for Capgemini. Schenk studied at NORDAKADEMIE University of Applied Sciences, Germany and Karlstads Universitet, Sweden and received his diploma in Computer Science and Business Management from NORDAKADEMIE in 2004.

Table of Contents

Foreword	p. xi
List of Figures	p. xiii
List of Tables	p. xvii
List of Contributors	p. xix
Introduction	p. 1
Use Case Scenarios	p. 7
Photo Use Case	p. 8
Motivating Examples	p. 8
Semantic Description of Photos Today	p. 9
Services We Need for Photo Collections	p. 10
Music Use Case	p. 10
Semantic Description of Music Assets	p. 11
Music Recommendation and Discovery	p. 12
Management of Personal Music Collections	p. 13
Annotation in Professional Media Production and Archiving	p. 14
Motivating Examples	p. 15
Requirements for Content Annotation	p. 17
Discussion	p. 18
Acknowledgements	p. 19
Canonical Processes of Semantically Annotated Media Production	p. 21
Canonical Processes	p. 22
Premeditate	p. 23
Create Media Asset	p. 23
Annotate	p. 23
Package	p. 24
Query	p. 24
Construct Message	p. 25
Organize	p. 25
Publish	p. 26
Distribute	p. 26
Example Systems	p. 27
CeWe Color Photo Book	p. 27
SenseCam	p. 29
Conclusion and Future Work	p. 33
Feature Extraction for Multimedia Analysis	p. 35
Low-Level Feature Extraction	p. 36
What Are Relevant Low-Level Features?	p. 36
Visual Descriptors	p. 36
Audio Descriptors	p. 45
Feature Fusion and Multi-modality	p. 54
Feature Normalization	p. 54
Homogeneous Fusion	p. 55
Cross-modal Fusion	p. 56
Conclusion	p. 58
Machine Learning Techniques for Multimedia Analysis	p. 59
Feature Selection	p. 61
Selection Criteria	p. 61
Subset Search	p. 62
Feature Ranking	p. 63
A Supervised Algorithm Example	p. 63
Classification	p. 65
Historical Classification Algorithms	p. 65
Kernel Methods	p. 67
Classifying Sequences	p. 71
Biologically Inspired Machine Learning Techniques	p. 73
Classifier Fusion	p. 75
Introduction	p. 75
Non-trainable Combiners	p. 75
Trainable Combiners	p. 76
Combination of Weak Classifiers	p. 77
Evidence Theory	p. 78
Consensual Clustering	p. 78
Classifier Fusion Properties	p. 80
Conclusion	p. 80
Semantic Web Basics	p. 81
The Semantic Web	p. 82
RDF	p. 83
RDF Graphs	p. 86
Named Graphs	p. 87
RDF Semantics	p. 88
RDF Schema	p. 90
Data Models	p. 93
Linked Data Principles	p. 94
Dereferencing Using Basic Web Look-up	p. 95
Dereferencing Using HTTP 303 Redirects	p. 95
Development Practicalities	p. 96
Data Stores	p. 97
Toolkits	p. 97
Semantic Web Languages	p. 99
The Need for Ontologies on the Semantic Web	p. 100
Representing Ontological Knowledge Using OWL	p. 100
OWL Constructs and OWL Syntax	p. 100
The Formal Semantics of OWL and its Different Layers	p. 102
Reasoning Tasks	p. 106
OWL Flavors	p. 107
Beyond OWL	p. 107
A Language to Represent Simple Conceptual Vocabularies: SKOS	p. 108
Ontologies versus Knowledge Organization Systems	p. 108
Representing Concept Schemes Using SKOS	p. 109
Characterizing Concepts beyond SKOS	p. 111
Using SKOS Concept Schemes on the Semantic Web	p. 112
Querying on the Semantic Web	p. 113
Syntax	p. 113
Semantics	p. 118
Default Negation in SPARQL	p. 123
Well-Formed Queries	p. 124
Querying for Multimedia Metadata	p. 124
Partitioning Datasets	p. 126
Related Work	p. 127
Multimedia Metadata Standards	p. 129
Selected Standards	p. 130
MPEG-7	p. 130
EBU P_Meta	p. 132
SMPTE Metadata Standards	p. 133
Dublin Core	p. 133
TV-Anytime	p. 134
METS and VRA	p. 134
MPEG-21	p. 135
XMP, IPTC in XMP	p. 135
EXIF	p. 136
DIG35	p. 137
ID3/MP3	p. 137
NewsML G2 and rNews	p. 138
W3C Ontology for Media Resources	p. 138
EBUCore	p. 139
Comparison	p. 140
Conclusion	p. 143
The Core Ontology for Multimedia	p. 145
Introduction	p. 145
A Multimedia Presentation for Granddad	p. 146
Related Work	p. 149
Requirements for Designing a Multimedia Ontology	p. 150
A Formal Representation for MPEG-7	p. 150
DOLCE as Modeling Basis	p. 151
Multimedia Patterns	p. 151
Basic Patterns	p. 155
Comparison with Requirements	p. 157
Granddad's Presentation Explained by COMM	p. 157
Lessons Learned	p. 159
Conclusion	p. 160
Knowledge-Driven Segmentation and Classification	p. 163
Related Work	p. 164
Semantic Image Segmentation	p. 165
Graph Representation of an Image	p. 165
Image Graph Initialization	p. 165
Semantic Region Growing	p. 167
Using Contextual Knowledge to Aid Visual Analysis	p. 170
Contextual Knowledge Formulation	p. 170
Contextual Relevance	p. 173
Spatial Context and Optimization	p. 177
Introduction	p. 177
Low-Level Visual Information Processing	p. 177
Initial Region-Concept Association	p. 178
Final Region-Concept Association	p. 179
Conclusions	p. 181
Reasoning for Multimedia Analysis	p. 183
Fuzzy DL Reasoning	p. 184
The Fuzzy DLf-SKLM	p. 184
The Tableaux Algorithm	p. 185
The FiRE Fuzzy Reasoning Engine	p. 187
Spatial Features for Image Region Labeling	p. 192
Fuzzy Constraint Satisfaction Problems	p. 192
Exploiting Spatial Features Using Fuzzy Constraint Reasoning	p. 193
Fuzzy Rule Based Reasoning Engine	p. 196
Reasoning over Resources Complementary to Audiovisual Streams	p. 201
Multi-Modal Analysis for Content Structuring and Event Detection	p. 205
Moving Beyond Shots for Extracting Semantics	p. 206
A Multi-Modal Approach	p. 207
Case Studies	p. 207
Case Study 1: Field Sports	p. 208
Content Structuring	p. 208
Concept Detection Leveraging Complementary Text Sources	p. 213
Case Study 2: Fictional Content	p. 214
Content Structuring	p. 215
Concept Detection Leveraging Audio Description	p. 219
Conclusions and Future Work	p. 221
Multimedia Annotation Tools	p. 223
State of the Art	p. 224
SVAT: Professional Video Annotation	p. 225
User Interface	p. 225
Semantic Annotation	p. 228
KAT: Semi-automatic, Semantic Annotation of Multimedia Content	p. 229
History	p. 231
Architecture	p. 232
Default Plugins	p. 234
Using COMM as an Underlying Model: Issues and Solutions	p. 234
Semi-automatic Annotation: An Example	p. 237
Conclusions	p. 239
Information Organization Issues in Multimedia Retrieval Using Low-Level Features	p. 241
Efficient Multimedia Indexing Structures	p. 242
An Efficient Access Structure for Multimedia Data	p. 243
Experimental Results	p. 245
Conclusion	p. 249
Feature Term Based Index	p. 249
Feature Terms	p. 250
Feature Term Distribution	p. 251
Feature Term Extraction	p. 252
Feature Dimension Selection	p. 253
Collection Representation and Retrieval System	p. 254
Experiment	p. 256
Conclusion	p. 258
Conclusion and Future Trends	p. 259
Acknowledgement	p. 259
The Role of Explicit Semantics in Search and Browsing	p. 261
Basic Search Terminology	p. 261
Analysis of Semantic Search	p. 262
Query Construction	p. 263
Search Algorithm	p. 265
Presentation of Results	p. 267
Survey Summary	p. 269
Use Case A: Keyword Search in ClioPatria	p. 270
Query Construction	p. 270
Search Algorithm	p. 270
Result Visualization and Organization	p. 273
Use Case B: Faceted Browsing in ClioPatria	p. 274
Query Construction	p. 274
Search Algorithm	p. 276
Result Visualization and Organization	p. 276
Conclusions	p. 277
Conclusion	p. 279
References	p. 281
Author Index	p. 301
Subject Index	p. 303
Table of Contents provided by Ingram. All Rights Reserved.