Self-Organizing Maps

by Kohonen, Teuvo; Huang, Thomas S.; Schroeder, M. R.

• ISBN13: 9783540679219
• ISBN10: 3540679219
• Edition: 3rd
• Format: Paperback
• Copyright: 2000-11-01
• Publisher: Springer Verlag

Summary

The Self-Organizing Map (SOM), with its variants, is the most popular artificial neural network algorithm in the unsupervised learning category. Many fields of science have adopted the SOM as a standard analytical tool: in statistics,signal processing, control theory, financial analyses, experimental physics, chemistry and medicine. A new area is organization of very large document collections. The SOM is also one of the most realistic models of the biological brain functions. This new edition includes a survey of over 2000 contemporary studies to cover the newest results; the case examples were provided with detailed formulae, illustrations and tables; a new chapter on software tools for SOM was written, other chapters were extended or reorganized.

Table of Contents

Mathematical Preliminaries	1	(71)
Mathematical Concepts and Notations	2	(15)
Vector Space Concepts	2	(6)
Matrix Notations	8	(3)
Eigenvectors and Eigenvalues of Matrices	11	(2)
Further Properties of Matrices	13	(2)
On Matrix Differential Calculus	15	(2)
Distance Measures for Patterns	17	(12)
Measures of Similarity and Distance in Vector Spaces	17	(4)
Measures of Similarity and Distance Between Symbol Strings	21	(7)
Averages Over Nonvectorial Variables	28	(1)
Statistical Pattern Analysis	29	(17)
Basic Probabilistic Concepts	29	(5)
Projection Methods	34	(5)
Supervised Classification	39	(5)
Unsupervised Classification	44	(2)
The Subspace Methods of Classification	46	(13)
The Basic Subspace Method	46	(3)
Adaptation of a Model Subspace to Input Subspace	49	(4)
The Learning Subspace Method (LSM)	53	(6)
Vector Quantization	59	(5)
Definitions	59	(1)
Derivation of the VQ Algorithm	60	(2)
Point Density in VQ	62	(2)
Dynamically Expanding Context	64	(7)
Setting Up the Problem	65	(1)
Automatic Determination of Context-Independent Productions	66	(1)
Conflict Bit	67	(1)
Construction of Memory for the Context-Dependent Productions	68	(1)
The Algorithm for the Correction of New Strings	68	(1)
Estimation Procedure for Unsucessful Searches	69	(1)
Practical Experiments	69	(2)
Neural Modeling	71	(34)
Models, Paradigms, and Methods	71	(1)
A History of Some Main Ideas in Neural Modeling	72	(3)
Issues on Artificial Intelligence	75	(1)
On the Complexity of Biological Nervous Systems	76	(2)
What the Brain Circuits Are Not	78	(1)
Relation Between Biological and Artificial Neural Networks	79	(2)
What Functions of the Brain Are Usually Modeled?	81	(1)
When Do We Have to Use Neural Computing?	81	(1)
Transformation, Relaxation, and Decoder	82	(3)
Categories of ANNs	85	(2)
A Simple Nonlinear Dynamic Model of the Neuron	87	(2)
Three Phases of Development of Neural Models	89	(2)
Learning Laws	91	(5)
Hebb's Law	91	(1)
The Riccati-Type Learning Law	92	(3)
The PCA-Type Learning Law	95	(1)
Some Really Hard Problems	96	(3)
Brain Maps	99	(6)
The Basic SOM	105	(72)
A Qualitative Introduction to the SOM	106	(3)
The Original Incremental SOM Algorithm	109	(6)
The ``Dot-Product SOM''	115	(1)
Other Preliminary Demonstrations of Topology-Preserving Mappings	116	(11)
Ordering of Reference Vectors in the Input Space	116	(4)
Demonstrations of Ordering of Responses in the Output Space	120	(7)
Basic Mathematical Approaches to Self-Organization	127	(11)
One-Dimensional Case	128	(4)
Constructive Proof of Ordering of Another One-Dimensional SOM	132	(6)
The Batch Map	138	(4)
Initialization of the SOM Algorithms	142	(1)
On the ``Optimal'' Learning-Rate Factor	143	(2)
Effect of the Form of the Neighborhood Function	145	(1)
Does the SOM Algorithm Ensue from a Distortion Measure?	146	(2)
An Attempt to Optimize the SOM	148	(4)
Point Density of the Model Vectors	152	(7)
Earlier Studies	152	(1)
Numerical Check of Point Densities in a Finite One-Dimensional SOM	153	(6)
Practical Advice for the Construction of Good Maps	159	(2)
Examples of Data Analyses Implemented by the SOM	161	(4)
Attribute Maps with Full Data Matrix	161	(4)
Case Example of Attribute Maps Based on Incomplete Data Matrices (Missing Data): ``Poverty Map''	165	(1)
Using Gray Levels to Indicate Clusters in the SOM	165	(1)
Interpretation of the SOM Mapping	166	(4)
``Local Principal Components''	166	(3)
Contribution of a Variable to Cluster Structures	169	(1)
Speedup of SOM Computation	170	(7)
Shortent Winner Search	170	(2)
Increasing the Number of Units in the SOM	172	(3)
Smoothing	175	(1)
Combination of Smoothing. Lattice Growing. and SOM Algorithm	176	(1)
Physiological Interpretation of SOM	177	(14)
Conditions for Abstract Feature Maps in the Brain	177	(1)
Two Different Lateral Control Mechanisms	178	(7)
The WTA Function. Based on Lateral Activity Control	179	(5)
Lateral Control of Plasticity	184	(1)
Learning Equation	185	(1)
System Models of SOM and Their Simulations	185	(3)
Recapitulation of the Features of the Physiological SOM Model	188	(1)
Similarities Between the Brain Maps and Simulated Feature Maps	188	(3)
Magnification	189	(1)
Imperfect Maps	189	(1)
Overlapping Maps	189	(2)
Variants of SOM	191	(54)
Overview of Ideas to Modify the Basic SOM	191	(1)
Adaptive Tensorial Weights	191	(6)
Tree-Structured SOM in Searching	197	(1)
Different Definitions of the Neighborhood	198	(2)
Neighborhoods in the Signal Space	200	(1)
Dynamical Elements Added to the SOM	201	(4)
The SOM for Symbol Strings	205	(2)
Initialization of the SOM for Strings	205	(1)
The Batch Map for Strings	206	(1)
Tie-Break Rules	206	(1)
A Simple Example: The SOM of Phonemic Transcriptions	207	(1)
Operator Maps	207	(4)
Evolutionary-Learning SOM	211	(4)
Evolutionary-Learning Filters	211	(1)
Self-Organization According to a Fitness Function	212	(3)
Supervised SOM	215	(1)
The Adaptive-Subspace SOM (ASSOM)	216	(26)
The Problem of Invariant Features	216	(2)
Relation Between Invariant Features and Linear Subspaces	218	(4)
The ASSOM Algorithm	222	(4)
Derivation of the ASSOM Algorithm by Stochastic Approximation	226	(2)
ASSOM Experiments	228	(14)
Feedback-Controlled Adaptive-Subspace SOM (FASSOM)	242	(3)
Learning Vector Quantization	245	(18)
Optimal Decision	245	(1)
The LVQ1	246	(4)
The Optimized-Learning-Rate LVQ1 (OLVQ1)	250	(1)
The Batch-LVQ1	251	(1)
The Batch-LVQ1 for Symbol Strings	252	(1)
The LVQ2 (LVQ2.1)	252	(1)
The LVQ3	253	(1)
Differences Between LVQ1. LVQ2 and LVQ3	254	(1)
General Considerations	254	(2)
The Hypermap-Type LVQ	256	(5)
The ``LVQ-SOM''	261	(2)
Applications	263	(48)
Preprocessing of Optic Patterns	264	(4)
Blurring	265	(1)
Expansion in Terms of Global Features	266	(1)
Spectral Analysis	266	(1)
Expansion in Terms of Local Features (Wavelets)	267	(1)
Recapitulation of Features of Optic Patterns	267	(1)
Acoustic Preprocessing	268	(1)
Process and Machine Monitoring	269	(2)
Selection of Input Variables and Their Scaling	269	(1)
Analysis of Large Systems	270	(1)
Diagnosis of Speech Voicing	271	(1)
Transcription of Continuous Speech	271	(9)
Texture Analysis	280	(1)
Contextual Maps	281	(5)
Artifically Generated Clauses	283	(2)
Natural Text	285	(1)
Organization of Large Document Files	286	(13)
Statistical Models of Documents	286	(6)
Construction of Very Large WEBSOM Maps by the Projection Method	292	(4)
The WEBSOM of All Electronic Patent Abstracts	296	(3)
Robot-Arm Control	299	(5)
Simultaneous Learning of Input and Output Parameters	299	(4)
Another Simple Robot-Arm Control	303	(1)
Telecommunications	304	(4)
Adaptive Detector for Quantized Signals	304	(1)
Channel Equalization in the Adaptive QAM	305	(1)
Error-Tolerant Transmission of Images by a Pair of SOMs	306	(2)
The SOM as an Estimator	308	(3)
Symmetric (Autoassociative) Mapping	308	(1)
Asymmetric (Heteroassociative) Mapping	309	(2)
Software Tools for SOM	311	(18)
Necessary Requirements	311	(2)
Desirable Auxiliary Features	313	(2)
SOM Program Packages	315	(4)
SOM PAK	315	(2)
SOM Toolbox	317	(1)
Nenet (Neural Networks Tool)	318	(1)
Viscovery SOMine	318	(1)
Examples of the Use of SOM PAK	319	(8)
File Formats	319	(3)
Description of the Programs in SOM PAK	322	(4)
A Typical Training Sequence	326	(1)
Neural-Networks Software with the SOM Option	327	(2)
Hardware for SOM	329	(18)
An Analog Classifier Circuit	329	(3)
Fast Digital Classifier Circuits	332	(5)
SIMD Implementation of SOM	337	(2)
Transputer Implementation of SOM	339	(2)
Systolic-Array Implementation of SOM	341	(1)
The COKOS Chip	342	(1)
The TlnMANN Chip	342	(2)
NBISOM 25 Chip	344	(3)
An Overview of SOM Literature	347	(26)
Books and Review Articles	347	(1)
Early Works on Competitive Learning	348	(1)
Status of the Mathematical Analyses	349	(9)
Zero-Order Topology (Classical VQ) Results	349	(1)
Alternative Topological Mappings	350	(1)
Alternative Architectures	350	(1)
Functional Variants	351	(1)
Theory of the Basic SOM	352	(6)
The Learning Vector Quantization	358	(1)
Diverse Applications of SOM	358	(11)
Machine Vision and Image Analysis	358	(2)
Optical Character and Script Reading	360	(1)
Speech Analysis and Recognition	360	(1)
Acoustic and Musical Studies	361	(1)
Signal Processing and Radar Measurements	362	(1)
Telecommunications	362	(1)
Industrial and Other Real-World Measurements	362	(1)
Process Control	363	(1)
Robotics	364	(1)
Electronic-Circuit Design	364	(1)
Physics	364	(1)
Chemistry	365	(1)
Biomedical Applications Without Image Processing	365	(1)
Neurophysiological Research	366	(1)
Data Processing and Analysis	366	(1)
Linguistic and AI Problems	367	(1)
Mathematical and Other Theoretical Problems	368	(1)
Applications of LVQ	369	(1)
Survey of SOM and LVQ Implementations	370	(3)
Glossary of ``Neural'' Terms	373	(30)
References	403	(84)
Index	487