Learning and Soft Computing : Support Vector Machines, Neural Networks and Fuzzy Logic Models

by Vojislav Kecman

• ISBN13: 9780262112550
• ISBN10: 0262112558
• Format: Hardcover
• Copyright: 2001-03-19
• Publisher: Bradford Books

Summary

This textbook provides a thorough introduction to the field of learning from experimental data and soft computing. Support vector machines (SVM) and neural networks (NN) are the mathematical structures, or models, that underlie learning, while fuzzy logic systems (FLS) enable us to embed structured human knowledge into workable algorithms. The book assumes that it is not only useful, but necessary, to treat SVM, NN, and FLS as parts of a connected whole. Throughout, the theory and algorithms are illustrated by practical examples, as well as by problem sets and simulated experiments. This approach enables the reader to develop SVM, NN, and FLS in addition to understanding them. The book also presents three case studies: on NN-based control, financial time series analysis, and computer graphics. A solutions manual and all of the MATLAB programs needed for the simulated experiments are available.

Table of Contents

Preface	xi
Introduction	xvii
Learning and Soft Computing: Rationale, Motivations, Needs, Basics	1	(120)
Examples of Applications in Diverse Fields	2	(7)
Basic Tools of Soft Computing: Neural Networks, Fuzzy Logic Systems, and Support Vector Machines	9	(15)
Basics of Neural Networks	13	(5)
Basics of Fuzzy Logic Modeling	18	(6)
Basic Mathematics of Soft Computing	24	(37)
Approximation of Multivariate Functions	25	(19)
Nonlinear Error Surface and Optimization	44	(17)
Learning and Statistical Approaches to Regression and Classification	61	(60)
Regression	62	(6)
Classification	68	(35)
Problems	103	(14)
Simulation Experiments	117	(4)
Support Vector Machines	121	(72)
Risk Minimization Principles and the Concept of Uniform Convergence	129	(9)
The VC Dimension	138	(7)
Structural Risk Minimization	145	(3)
Support Vector Machine Algorithms	148	(45)
Linear Maximal Margin Classifier for Linearly Separable Data	149	(13)
Linear Soft Margin Classifier for Overlapping Classes	162	(4)
The Nonlinear Classifier	166	(10)
Regression by Support Vector Machines	176	(8)
Problems	184	(5)
Simulation Experiments	189	(4)
Single-Layer Networks	193	(62)
The Perceptron	194	(19)
The Geometry of Perceptron Mapping	196	(3)
Convergence Theorem and Perceptron Learning Rule	199	(14)
The Adaptive Linear Neuron (Adaline) and the Least Mean Square Algorithm	213	(42)
Representational Capabilities of the Adaline	214	(11)
Weights Learning for a Linear Processing Unit	225	(19)
Problems	244	(9)
Simulation Experiments	253	(2)
Multilayer Perceptrons	255	(58)
The Error Backpropagation Algorithm	255	(5)
The Generalized Delta Rule	260	(6)
Heuristics or Practical Aspects of the Error Backpropagation Algorithm	266	(47)
One, Two, or More Hidden Layers?	267	(1)
Number of Neurons in a Hidden Layer, or the Bias-Variance Dilemma	268	(7)
Type of Activation Functions in a Hidden Layer and the Geometry of Approximation	275	(15)
Weights Initialization	290	(2)
Error Function for Stopping Criterion at Learning	292	(4)
Learning Rate and the Momentum Term	296	(7)
Problems	303	(6)
Simulation Experiments	309	(4)
Radial Basis Function Networks	313	(52)
Ill-Posed Problems and the Regularization Technique	314	(15)
Stabilizers and Basis Functions	329	(4)
Generalized Radial Basis Function Networks	333	(32)
Moving Centers Learning	337	(2)
Regularization with Nonradial Basis Functions	339	(4)
Orthogonal Least Squares	343	(10)
Optimal Subset Selection by Linear Programming	353	(5)
Problems	358	(3)
Simulation Experiments	361	(4)
Fuzzy Logic Systems	365	(56)
Basics of Fuzzy Logic Theory	367	(29)
Crisp (or Classic) and Fuzzy Sets	367	(4)
Basic Set Operations	371	(3)
Fuzzy Relations	374	(6)
Composition of Fuzzy Relations	380	(2)
Fuzzy Inference	382	(3)
Zadeh's Compositional Rule of Inference	385	(6)
Defuzzification	391	(5)
Mathematical Similarities between Neural Networks and Fuzzy Logic Models	396	(8)
Fuzzy Additive Models	404	(17)
Problems	410	(9)
Simulation Experiments	419	(2)
Case Studies	421	(60)
Neural Networks-Based Adaptive Control	421	(28)
General Learning Architecture, or Direct Inverse Modeling	423	(2)
Indirect Learning Architecture	425	(1)
Specialized Learning Architecture	425	(4)
Adaptive Backthrough Control	429	(20)
Financial Time Series Analysis	449	(14)
Computer Graphics	463	(18)
One-Dimensional Morphing	466	(2)
Multidimensional Morphing	468	(2)
Radial Basis Function Networks for Human Animation	470	(4)
Radial Basis Function Networks for Engineering Drawings	474	(7)
Basic Nonlinear Optimization Methods	481	(24)
Classical Methods	482	(14)
Newton-Raphson Method	485	(1)
Variable Metric or Quasi-Newton Methods	486	(1)
Davidon-Fletcher-Powell Method	487	(1)
Broyden-Fletcher-Goldfarb-Shano Method	488	(1)
Conjugate Gradient Methods	489	(3)
Fletcher-Reeves Methods	492	(1)
Polak-Ribiere Method	493	(1)
Two Specialized Algorithms for a Sum-of-Error-Squares Error Function	494	(2)
Genetic Algorithms and Evolutionary Computing	496	(9)
Basic Structure of Genetic Algorithms	497	(1)
Mechanism of Genetic Algorithms	497	(8)
Mathematical Tools of Soft Computing	505	(20)
Systems of Linear Equations	505	(5)
Vectors and Matrices	510	(6)
Linear Algebra and Analytic Geometry	516	(2)
Basics of Multivariable Analysis	518	(2)
Basics from Probability Theory	520	(5)
Selected Abbreviations	525	(2)
Notes	527	(4)
References	531	(8)
Index	539