Neural Nets and Chaotic Carriers

by Whittle, Peter

• ISBN13: 9781848165908
• ISBN10: 1848165900
• Edition: 2nd
• Format: Hardcover
• Copyright: 2010-07-03
• Publisher: Textstream

Summary

Neural Nets and Chaotic Carriers develops rational principles for the design of associative memories, with a view to applying these principles to models with irregularly oscillatory operation so evident in biological neural systems, and necessitated by the meaninglessness of absolute signal levels. Design is based on the criterion that an associative memory must be able to cope with "fading data", i.e., to form an inference from the data even as its memory of that data degrades. The resultant net shows striking biological parallels. When these principles are combined with the Freeman specification of a neural oscillator, some remarkable effects emerge. for example, the commonly-observed phenomenon of neuronal bursting appears, with gamma-range oscillation modulated by a low-frequency square-wave oscillation (the "escapement oscillation"). Bridging studies and new results of artificial and biological neural networks, the book has a strong research character. It is, on the other hand, accessible to non-specialists for its concise exposition on the basics.

Table of Contents

Preface	p. v
Opening and Themes	p. 1
Introduction and Aspirations	p. 3
Optimal Statistical Procedures	p. 9
The optimisation of actions	p. 9
Effective estimation of state	p. 12
The quadratic/Gaussian case: Estimation and certainty equivalence	p. 13
The linear model, in Bayesian and classic versions	p. 15
Linear Links and Nonlinear Knots: The Basic Neural Net	p. 17
Neural calculations: The linear gate and the McCulloch-Pitts net	p. 17
Sigmoid and threshold functions	p. 21
Iteration	p. 22
Neural systems and feedback in continuous time	p. 26
Equilibrium excitation patterns	p. 28
Some special-purpose nets	p. 28
Bifurcations and Chaos	p. 33
The Hopf bifurcation	p. 33
Chaos	p. 35
Associative and Storage Memories	p. 41
What is a Memory? The Hamming and Hopfield Nets	p. 43
Associative memories	p. 43
The Hamming net	p. 45
Autoassociation, feedback and storage	p. 47
The Hopfield net	p. 50
Alternative formulations of the Hopfield net	p. 52
Compound and 'Spurious' Traces	p. 54
Performance and trace structure	p. 54
The recognition of simple traces	p. 55
Inference for compound traces	p. 57
Network realisation of the quantised regression	p. 59
Reliability constraints for the quantised regression	p. 60
Stability constraints for the quantised regression	p. 62
The Hopfield net	p. 64
Preserving Plasticity: A Bayesian Approach	p. 69
A Bayesian view	p. 69
A robust estimation method	p. 72
Dynamic and neural versions of the algorithm	p. 74
The Key Task: the Fixing of Fading Data. Conclusions I	p. 76
Fading data, and the need for quantisation	p. 76
The probability-maximising algorithm (PMA)	p. 78
Properties of the vector activation function F(z)	p. 81
Some special cases	p. 83
The network realisation of the full PMA	p. 85
Neural implementation of the PMA	p. 89
The PMA and the exponential family	p. 92
Conclusions I	p. 93
Performance of the Probability-Maximising Algorithm	p. 96
A general formulation	p. 96
Considerations for reliable inference	p. 98
Performance of the PMA for simple stimuli	p. 100
Compound stimuli: The general pattern	p. 103
Compound stimuli in the Gaussian case	p. 107
Other Memories - Other Considerations	p. 109
The supervised learning of a linear relation	p. 109
Unsupervised learning: The criterion of economy	p. 111
Principal components	p. 112
The learning of an optimal reduction	p. 115
Dual variables, back-propagation and Hebb's rule	p. 117
Self-organising feature maps	p. 120
Other proposals	p. 124
Oscillatory Operation and the Biological Model	p. 127
Neuron Models and Neural Masses	p. 131
The biological neuron	p. 131
Neural masses: Reduced dynamics	p. 134
Neural masses: Full dynamics and the tank/sump model	p. 137
Systems of neural units (masses)	p. 142
The Freeman oscillator	p. 142
Numerical solution for the Freeman oscillator	p. 146
The evidence of EEG traces	p. 148
Freeman Oscillators - Solo and in Concert	p. 152
Systems of neural units	p. 152
The case of simple positive feedback	p. 157
Systems of coupled oscillators	p. 159
The Freeman oscillator with positive feedback	p. 160
Numerical results for the oscillator with feedback	p. 165
A block of oscillators	p. 168
A chain of oscillators	p. 169
Feedback plus standardisation: An essential mechanism	p. 170
Numerical results for the standardised oscillator	p. 173
Some notes on the literature	p. 175
Associative Memories Incorporating the Freeman Oscillator	p. 179
Safe data: Adaptation of the Bayesian inferential net	p. 179
The oscillatory version of the one-stage PMA	p. 181
The oscillatory version of the two-stage PMA	p. 184
Numerical results for a two-stage system	p. 188
Olfactory Comparisons. Conclusions II	p. 190
The anatomy of the olfactory system	p. 191
Neural components	p. 195
Block versions and analogues of the olfactory system	p. 197
Interpretation	p. 200
Conclusions II	p. 205
Transmission Delays	p. 208
The Freeman oscillator with feedback	p. 208
Interacting Freeman oscillators	p. 210
Extension of the Wigner Semi-Circle Law	p. 213
Realisation of the PMA equation	p. 216
References	p. 219
Index	p. 225
Table of Contents provided by Ingram. All Rights Reserved.