Self-Organizing Neural Networks

by Girolami, Mark

• ISBN13: 9781852330668
• ISBN10: 185233066X
• Format: Paperback
• Copyright: 1999-09-01
• Publisher: Springer Verlag

Summary

Presents the theory and applications of self- organizing neural network models which perform the Independent Component Analysis (ICA) trans- formation and Blind Source Separation (BSS). Softcover. DLC: Neural networks (Computer science).

Table of Contents

Foreword	ix
Introduction	1	(4)
Self-Organisation and Blind Signal Processing	1	(2)
Outline of Book Chapters	3	(2)
Background to Blind Source Separation	5	(30)
Problem Formulation	5	(2)
Entropy and Information	7	(11)
Entropy	7	(3)
Kullback-Leibler Entropy and Mutual Information	10	(5)
Invertible Probability Density Transformations	15	(3)
A Contrast Function for ICA	18	(2)
Cumulant Expansions of Probability Densities and Higher Order Statistics	20	(10)
Moment Generating and Cumulant Generating Functions	20	(7)
Properties of Moments and Cumulants	27	(3)
Gradient Based Function Optimisation	30	(5)
The Natural Gradient and Covariant Algorithms	31	(4)
Fourth Order Cumulant Based Blind Source Separation	35	(12)
Early Algorithms and Techniques	35	(4)
The Method of Contrast Minimisation	39	(3)
Adaptive Source Separation Methods	42	(2)
Conclusions	44	(3)
Self-Organising Neural Networks	47	(30)
Linear Self-Organising Neural Networks	47	(9)
Linear Hebbian Learning	47	(3)
Principal Component Analysis	50	(2)
Linear Anti-Hebbian Learning	52	(4)
Non-Linear Self-Organising Neural Networks	56	(19)
Non-Linear Anti-Hebbian Learning: The Herrault-Jutten Network	56	(6)
Information Theoretic Algorithms	62	(8)
Non-Linear Hebbian Learning Algorithms	70	(1)
Signal Representation Error Minimisation	71	(2)
Non-Linear Criterion Maximisation	73	(2)
Conclusions	75	(2)
The Non-Linear PCA Algorithm and Blind Source Separation	77	(42)
Introduction	77	(1)
Non-Linear PCA Algorithm and Source Separation	77	(2)
Non-Linear PCA Algorithm Cost Function	79	(7)
Non-Linear PCA Algorithm Activation Function	86	(30)
Asymptotic Stability Requirements	87	(5)
Stability Properties of the Compound Activation Function	92	(4)
Stability of Solution with Sub-Gaussian Sources	96	(2)
Simulation: Separation of Mixtures of Sub-Gaussian Sources	98	(6)
Stability of Solution with Super-Gaussian Sources	104	(4)
Simulation: Separation of Mixtures of Super-Gaussian Sources	108	(6)
Separation of Mixtures of Both Sub-and Super-Gaussian Sources	114	(2)
Conclusions	116	(3)
Non-Linear Feature Extraction and Blind Source Separation	119	(46)
Introduction	119	(1)
Structure Identification in Multivariate Data	119	(2)
Neural Network Implementation of Exploratory Projection Pursuit	121	(2)
Neural Exploratory Projection Pursuit and Blind Source Separation	123	(1)
Kurtosis Extrema	124	(3)
Finding Interesting and Independent Directions	127	(5)
Finding Multiple Interesting and Independent Directions Using Symmetric Feedback and Adaptive Whitening	132	(18)
Adaptive Spatial Whitening	133	(3)
Simulations	136	(5)
An Extended EPP Network with Non-Linear Output Connections	141	(9)
Finding Multiple Interesting and Independent Directions Using Hierarchic Feedback and Adaptive Whitening	150	(1)
Simulations	151	(1)
Adaptive BSS Using a Deflationary EPP Network	152	(7)
Conclusions	159	(6)
Information Theoretic Non-Linear Feature Extraction And Blind Source Separation	165	(36)
Introduction	165	(1)
Information Theoretic Indices for EPP	165	(2)
Maximum Negentropy Learning	167	(14)
Single Neuron Maximum Negentropy Learning	167	(4)
Multiple Output Neuron Maximum Negentropy Learning	171	(5)
Maximum Negentropy Learning and Infomax Equivalence	176	(2)
The Natural Gradient and Covariant Learning	178	(3)
General Maximum Negentropy Learning	181	(10)
Stability Analysis of Generalised Algorithm	191	(1)
Simulation Results	192	(8)
Conclusions	200	(1)
Temporal Anti-Hebbian Learning	201	(38)
Introduction	201	(1)
Blind Source Separation of Convolutive Mixtures	201	(4)
Temporal Linear Anti-Hebbian Model	205	(5)
Comparative Simulation	210	(3)
Review of Existing Work on Adaptive Separation of Convolutive Mixtures	213	(7)
Maximum Likelihood Estimation and Source Separation	220	(3)
Temporal Anti-Hebbian Learning Based on Maximum Likelihood Estimation	223	(6)
Comparative Simulations Using Varying PDF Models	229	(8)
Conclusions	237	(2)
Applications	239	(16)
Introduction	239	(1)
Industrial Applications	239	(3)
Rotating Machine Vibration Analysis	240	(1)
A Multi-Tag Frequency Identification System	241	(1)
Biomedical Applications	242	(1)
Detection of Sleep Spindles in EEG	242	(1)
ICA: A Data Mining Tool	243	(5)
Experimental Results	248	(6)
The Oil Pipeline Data	249	(1)
The Swiss Banknote Data	250	(4)
Conclusions	254	(1)
References	255	(14)
Index	269