Applications of Supervised and Unsupervised Ensemble Methods

by Okun, Oleg; Valentini, Giorgio

• ISBN13: 9783642039980
• ISBN10: 3642039987
• Format: Hardcover
• Copyright: 2009-10-01
• Publisher: Springer Verlag

Summary

This book contains the extended papers presented at the 2nd Workshop on Supervised and Unsupervised Ensemble Methods and their Applications (SUEMA) held on 21-22 July, 2008 in Patras, Greece, in conjunction with the 18th European Conference on Artificial Intelligence (ECAI'2008). This workshop was a successor of the smaller event held in 2007 in conjunctionwith 3rd Iberian Conference on Pattern Recognition and Image Analysis, Girona, Spain. The success of that event as well as the publication of workshop papers in the edited book 'œSupervised and Unsupervised Ensemble Methods and their Applications', published by Springer-Verlag in Studies in Computational Intelligence Series in volume 126, encouraged us to continue a good tradition.The purpose of this book is to support practitioners in various branches of science and techology to adopt the ensemble approach for their daily research work. We hope that fourteen chapters composing the book will be a good guide in the sea of numerous opportunities for ensemble methods.

Table of Contents

An Ensemble Pruning Primer	p. 1
Introduction	p. 1
Background	p. 2
Producing the Models	p. 2
Combining the Models	p. 3
A Taxonomy of Ensemble Pruning Methods	p. 4
Ranking-Based Methods	p. 4
Clustering-Based Methods	p. 5
Optimization-Based Methods	p. 6
Genetic Algorithms	p. 6
Semi-definite Programming	p. 6
Hill Climbing	p. 7
Other Methods	p. 10
Statistical Procedures	p. 10
Reinforcement Learning	p. 10
Boosting	p. 11
Conclusions	p. 11
References	p. 12
Evade Hard Multiple Classifier Systems	p. 15
Introduction	p. 15
Related Work	p. 17
Previous Works on Multiple Classifiers for Security Applications	p. 17
A Theoretical Framework for Adversarial Classification Problems	p. 19
Are Multiple Classifier Systems Harder to Evade?	p. 21
Adding Features to a Classification System	p. 22
Splitting Features across an Ensemble of Classifiers	p. 24
A Case Study in Spam Filtering	p. 28
Adding Features to a Spam Filter	p. 29
Splitting the Features of a Spam Filter across an Ensemble of Classifiers	p. 32
Conclusions	p. 37
References	p. 37
A Personal Antispam System Based on a Behaviour-Knowledge Space Approach	p. 39
Introduction	p. 39
Related Work	p. 40
System Architecture	p. 42
Textual Features	p. 43
Semantic Features	p. 43
Syntactic Features	p. 46
Image Features	p. 47
Visual Features	p. 47
OCR-Based Features	p. 49
Combining Text-Based and Image-Based Classifiers	p. 50
Experimental Results	p. 52
Conclusion	p. 55
References	p. 56
Weighted Decoding ECOC for Facial Action Unit Classification	p. 59
Introduction	p. 59
Ensembles and Bootstrapping	p. 61
Error-Correcting Output Coding ECOC	p. 63
Motivation	p. 64
ECOC Algorithm and OOB Estimate	p. 65
Coding Strategies and Errors	p. 66
Weighted Decoding	p. 68
Dataset and Feature Extraction	p. 69
Experiments on Cohn-Kanade Database	p. 71
Discussion	p. 74
Conclusion	p. 75
References	p. 75
Prediction of Gene Function Using Ensembles of SVMs and Heterogeneous Data Sources	p. 79
Introduction	p. 79
Methods	p. 80
Linear Weighted Combination with Linear and Logarithmic Weights	p. 81
Decision Templates	p. 82
Experimental Setup	p. 83
Heterogeneous Biomolecular Datasets	p. 83
The Functional Catalogue (FunCat)	p. 85
Base Learners Tuning and Generation of Optimized Classifiers	p. 86
Results	p. 86
Discussion	p. 88
Conclusions	p. 89
References	p. 90
Partitioner Trees for Classification: A New Ensemble Method	p. 93
Introduction	p. 93
Partitioner Trees	p. 95
Algorithm	p. 95
Discussion	p. 99
Experiments	p. 104
Experimental Setup	p. 104
Results	p. 105
Conclusion	p. 111
References	p. 112
Disturbing Neighbors Diversity for Decision Forests	p. 113
Introduction	p. 113
Method	p. 115
Results	p. 119
Lesion Study	p. 124
Conclusion	p. 131
References	p. l32
Improving Supervised Learning with Multiple Clusterings	p. 135
Introduction	p. 135
Related Works	p. 136
Description of the Method	p. 138
Improving Supervised Classification with Clustering	p. 138
The Proposed Method	p. 140
Experiments	p. 142
Artificial Benchmark Evaluation	p. 142
Real Data Evaluation	p. 144
Conclusion	p. 148
References	p. 148
The Neighbors Voting Algorithm and Its Applications	p. 151
Introduction	p. 151
The Tensor Voting Framework	p. 152
The Neighbors Voting Algorithm	p. 155
Statistical Interpretation	p. 156
Applications of the Neighbors Voting Algorithm	p. 157
Point Clustering	p. 157
Classification	p. 158
Image Inpainting with NV	p. 160
Results	p. 161
Clustering Results	p. 161
Classification Results	p. 165
Inpainting Results	p. 170
Conclusions and Future Work	p. 170
References	p. 172
Clustering Ensembles with Active Constraints	p. 175
Introduction	p. 175
Related Work	p. 176
Locally Adaptive Clustering	p. 177
Selecting Informative Constraints	p. 179
Chunklet Graph	p. 181
Chunklet Assignment	p. 182
Constrained-Weighted Bipartite Partitioning Algorithm (C-WBPA)	p. 182
Empirical Evaluation	p. 185
Analysis of the Results	p. 188
Conclusions	p. 188
References	p. 189
Verifiable Ensembles of Low-Dimensional Submodels for Multi-class Problems with Imbalanced Misclassification Costs	p. 191
Introduction	p. 191
The Binary Ensemble Framework	p. 193
Decision Tree-Like Ensemble Model	p. 194
Non-Hierarchical Ensemble Model	p. 194
An Illustrative Example	p. 195
Multi-class Extensions of Binary Classifiers	p. 197
The Multi-class Ensemble Framework	p. 199
Ensemble of Multi-class Submodels	p. 199
Hierarchical Separate-and-Conquer Ensemble	p. 200
One-versus-Rest Ensemble	p. 201
An Illustrative Example (Cont'd)	p. 202
Experiments	p. 203
Binary Classification Problems	p. 205
Multi-class Classification Problems	p. 206
Comparison of the Ensemble Methods	p. 206
Comparison of Different Feature Selection Methods	p. 208
Conclusions	p. 210
References	p. 210
Independent Data Model Selection for Ensemble Dispersion Forecasting	p. 213
Introduction	p. 213
The æMedian ModelÆ Approach	p. 215
Negentropy-Based Hierarchical Agglomeration	p. 218
Kullback-Leibler Divergence	p. 218
Negentropy Information	p. 218
Agglomerative Approach	p. 219
Experimental Results	p. 219
Conclusions	p. 229
References	p. 229
Integrating Liknon Feature Selection and Committee Training	p. 233
Introduction	p. 233
Computational Paradigm and Parameters	p. 234
Liknon-Based Feature Selection	p. 235
Banana Example: Classification in Nonlinear Class Separation	p. 238
The Benchmark of NIPS2003 Feature Selection Challenge	p. 240
Perforinance, Size and Purity of the Feature Subsets in the Benchmark	p. 240
NN3 Committee and Liknon Feature Profiles in the Benchmark	p. 242
Conclusion	p. 248
References	p. 248
Evaluating Hybrid Ensembles for Intelligent Decision Support for Intensive Care	p. 251
Introduction	p. 251
Previous Work	p. 252
INTCare System	p. 254
Description of the Agents	p. 254
Problem Description	p. 256
Data Description	p. 257
Related Work	p. 257
Experimental Setting	p. 258
Results	p. 260
Discussion	p. 262
Conclusion	p. 263
References	p. 263
Index	p. 267
Table of Contents provided by Ingram. All Rights Reserved.