Perspectives on Content-Based Multimedia Systems

by Wu, Jian Kang; Kankanhalli, Mohan S.; Lim, Joo-Hwee; Hong, Dezhong

• ISBN13: 9780792379447
• ISBN10: 0792379446
• Format: Hardcover
• Copyright: 2000-08-01
• Publisher: Kluwer Academic Pub

Summary

Multimedia data comprising of images, audio and video is becoming increasingly common. The decreasing costs of consumer electronic devices such as digital cameras and digital camcorders, along with the ease of transportation facilitated by the Internet, has lead to a phenomenal rise in the amount of multimedia data generated and distributed. Given that this trend of increased use of multimedia data is likely to accelerate, there is an urgent need for providing a clear means of capturing, storing, indexing, retrieving, analyzing and summarizing such data. Content-based access to multimedia data is of primary importance since it is the natural way by which human beings interact with such information. To facilitate the content-based access of multimedia information, the first step is to derive feature measures from these data so that a feature space representation of the data content can be formed. This can subsequently allow for mapping the feature space to the symbol space (semantics) either automatically or through human intervention. Thus, signal to symbol mapping, useful for any practical system, can be successfully achieved. Perspectives on Content-Based Multimedia Systems provides a comprehensive set of techniques to tackle these important issues. This book offers detailed solutions to a wide range of practical problems in building real systems by providing specifics of three systems built by the authors. While providing a systems focus, it also equips the reader with a keen understanding of the fundamental issues, including a formalism for content-based multimedia database systems, multimedia feature extraction, object-based techniques, signature-based techniques and fuzzy retrieval techniques. The performance evaluation issues of practical systems is also explained. This book brings together essential elements of building a content-based multimedia database system in a way that makes them accessible to practitioners in computer science and electrical engineering. It can also serve as a textbook for graduate-level courses.

Table of Contents

Preface	xiii
Introduction	1	(14)
Technical Issues	2	(2)
Inter-operability	2	(1)
Automatic or semi-automatic indexing	3	(1)
Fool-proof retrieval methods	3	(1)
Content-Based Retrieval	4	(6)
Basic idea	4	(1)
Existing work	4	(6)
Challenges and solutions	10	(2)
Layout of the Book	12	(3)
References	12	(3)
Formalism of Content-based Multimedia Systems	15	(34)
The System Must be User-centered	15	(1)
Man-machine interaction must be extremely friendly	15	(1)
Reduce the work and information overload of users	15	(1)
Content-based Multimedia Information System	16	(10)
Definitions	17	(4)
Challenges in content-based retrieval	21	(2)
Representation of multimedia objects	23	(1)
Multimedia data analysis	24	(2)
Object Recall - A New Formalism for Content-Based Retrieval	26	(7)
Need for formalism of content-based retrieval	26	(2)
Contrast to pattern classification	28	(2)
Formalism for content-based navigation	30	(1)
Formalism for content-based retrieval by adaptive fuzzy concept	31	(2)
A Content-Based Similarity Retrieval Formalism	33	(2)
Feature measures and similarity functions	33	(1)
Training data set and loss function	34	(1)
Learning of Similarity Function	35	(6)
Problem statement	35	(1)
Related works	36	(1)
Learning of simple similarity functions	37	(1)
Learning of multi-level similarity functions	38	(3)
Experimental Results	41	(6)
The data set	41	(1)
Feature extraction	41	(2)
Within-sequence learning	43	(1)
Across-sequence learning	44	(3)
Conclusions	47	(2)
References	47	(2)
Color Feature Extraction	49	(20)
Color Spaces Selection	49	(3)
RGB color space	49	(1)
Munsell color system	50	(1)
CIE color systems	50	(1)
HSV color space	51	(1)
Color Measures	52	(4)
A brief review of color features	53	(2)
The distance method	55	(1)
Reference Color Table Method	56	(13)
The color clustering-based method	58	(6)
Experimental results	64	(1)
Remarks	65	(1)
References	66	(3)
Texture Feature Extraction	69	(24)
Discrete Cosine Transformed Texture	70	(4)
Discrete cosine transform	70	(1)
Texture feature based on discrete cosine transform	71	(1)
Feature vector formation	72	(1)
Test results	72	(2)
Wavelet Transformed Texture Feature	74	(8)
Wavelet Transform	74	(1)
Feature vector formation	75	(4)
Performance evaluation and test results	79	(3)
Texture Features Based on Second Moment Matrix	82	(2)
Performance evaluation and test results	83	(1)
Comparative Study	84	(9)
Comparison based on classification	84	(1)
Multiple features - combination of color and texture	84	(3)
Retrieval test	87	(3)
References	90	(3)
Video Processing	93	(28)
Review of Video Processing Techniques	94	(6)
Video features	94	(1)
Video applications	95	(1)
Research areas	96	(1)
State of the art review	97	(3)
Content-Based Reprensentative Frames Extraction and Video Summary	100	(21)
Definition	100	(1)
Related work	101	(1)
Extraction of representative frames	102	(12)
Application of representative frame extraction technique	114	(3)
References	117	(4)
Object Segmentation	121	(52)
Edge-preserved smoothing of features	123	(11)
Principle of edge-preserved smoothing	123	(4)
EPSM for 2D signal	127	(1)
Application in color feature	128	(1)
Application in texture feature	129	(5)
Segmentation algorithm: clustering and region merging	134	(22)
Clustering in the feature space	135	(3)
Cluster validation for unsupervised segmentation	138	(7)
Markov random field model and Gibbs distribution	145	(7)
Region analysis and region merging	152	(4)
Experiment results	156	(17)
Experiment setup	156	(1)
Experiment results	157	(12)
Summary	169	(1)
References	170	(3)
Human Face Detection	173	(36)
Color Segmentation of Faces	174	(14)
Chromaticity diagrams	176	(4)
Effects of the projection of 3-D color spaces on chromaticity diagrams	180	(4)
Method for face detection using chromaticity diagrams	184	(1)
Experiments	185	(3)
Shape Information As a Cue	188	(3)
Geometric characteristic of face	188	(2)
Shape descriptors	190	(1)
Experimental results	191	(1)
Face Feature Detection Using DOG Operators	191	(7)
The DOG (Difference of Gaussians) operator	191	(2)
Face feature detection by DOG operator	193	(1)
Experimental results	194	(2)
Discussion	196	(2)
Template-based Human Face Detection	198	(11)
Overview	198	(1)
Normalized ``face space''	199	(1)
Dimensionality reduction	199	(1)
Clustering and face template generation	200	(3)
Template matching	203	(3)
References	206	(3)
Visual Keywords	209	(30)
Introduction	209	(3)
Related Works	212	(2)
Methodology	214	(9)
Typification	215	(4)
Description scheme	219	(3)
Selection	222	(1)
Coding scheme	222	(1)
Image Retrieval	223	(11)
Unsupervised learning	224	(4)
Learning by instruction	228	(6)
Image Categorization	234	(2)
Conclusions and Future Directions	236	(3)
References	237	(2)
Fuzzy Retrieval	239	(34)
Problem Definition	240	(6)
Content-based fuzzy retrieval	240	(2)
Fuzzy sets over multi-dimensional universes	242	(1)
Fuzzy queries cannot be processed in the feature space	243	(1)
Fuzzy querying of multimedia information calls for new technology for its processing	244	(2)
Fuzzy Database Model	246	(5)
Extended tuple relation calculus for image retrieval	249	(2)
Fuzzy Query Processing	251	(7)
Feature space and fuzzy space	251	(1)
Fuzzy query interface	252	(1)
Context model	253	(1)
Extraction of the confidence value	254	(1)
Incomplete query condition	255	(1)
Similarity measure	256	(2)
Relevance feedback for query refinement	258	(1)
Learning Fuzzy Membership Functions	258	(6)
Need for learning fuzzy membership functions	259	(3)
A Fuzzy neuro membership function	262	(1)
Training the neural network	263	(1)
Experimental Results of Fuzzy Retrieval	264	(6)
Face database indexing and retrieval	264	(6)
Conclusion and Remarks	270	(3)
References	270	(3)
Face Retrieval	273	(38)
CAFIIR system	274	(8)
Data model	277	(2)
Indexing of facial images	279	(1)
Feature extraction	279	(1)
Content-based indexing	280	(2)
Content-Based Indexing of Multimedia Object	282	(5)
Definition	282	(1)
Horizontal links	283	(2)
A practical example	285	(1)
Iconic images construction	286	(1)
Content-based retrieval using ContIndex	286	(1)
ContIndex Creation by Self-organization Neural Networks	287	(7)
LEP neural network architecture	289	(1)
Fusion of multi-modal feature measures	290	(1)
Spatial self-organization	291	(2)
Bi-directional learning on experiences	293	(1)
Experimental Results	294	(6)
Conclusion and Remarks	300	(1)
Visual Retrieval of Facial Images	300	(3)
Facial composition	300	(1)
Browsing and similarity retrieval	301	(2)
Descriptive Queries	303	(4)
Fuzzy retrieval of facial images	303	(3)
Free text retrieval	306	(1)
Further Improvement of Queries	307	(1)
Feedback for query refinement	307	(1)
Combined query	308	(1)
Implementation and Concluding Remarks	308	(3)
References	309	(2)
System for Trademark Archival and Retrieval	311	(24)
Representation of Trademarks	312	(2)
Segmentation of Trademarks	314	(4)
Color segmentation	315	(3)
Capturing Visual Features of Trademark Images	318	(5)
Structural description	319	(1)
Feature measures	319	(3)
Match shape interpretation using fuzzy thesaurus	322	(1)
Composite Similarity Measures	323	(3)
Evaluation and Learning of Similarity Measures	326	(4)
Selection of training and test data sets	326	(1)
Learning of similarity functions	326	(2)
Evaluating the shape retrieval	328	(2)
Experimental Results	330	(2)
Conclusions	332	(3)
References	332	(3)
Digital Home Photo Album	335	(26)
Digital photo is becoming popular	335	(6)
What do the home users want?	336	(1)
User study 1	337	(2)
User study 2	339	(1)
The gap between ideal and realistic	339	(2)
Object-based Indexing and Retrieval	341	(20)
Object categories	343	(1)
Object models	344	(1)
System training	345	(3)
Image categorization	348	(4)
Image retrieval	352	(2)
Experimental results	354	(4)
References	358	(3)
Evaluation of Content-based Retrieval	361	(28)
Definition of the Problem	361	(3)
Retrieval as a function of data and query	362	(1)
Definition of benchmarking of multimedia databases	363	(1)
Benchmarking for Content-based Systems	364	(16)
Benchmarking for database retrieval methods	364	(1)
Benchmarking of information retrieval methods	365	(2)
Benchmarking links	367	(1)
Benchmarks for approximate retrieval methods	368	(12)
Complete benchmark for multimedia databases	380	(1)
Testing Multimedia Databases	380	(6)
Scalability with respect to time	381	(1)
Scalability with respect to quality	381	(2)
Examples of testing and evaluation	383	(3)
Conclusion	386	(3)
References	386	(3)
Index	389