Pulsed Neural Networks

by Maass, Wolfgang; Bishop, Christopher M.

• ISBN13: 9780262632218
• ISBN10: 0262632217
• Format: Paperback
• Copyright: 2001-01-26
• Publisher: Bradford Books

Summary

In recent years, data from neurobiological experiments have made it increasingly clear that biological neural networks, which communicate through pulses, use the timing of the pulses to transmit information and perform computation. This realization has stimulated significant research on pulsed neural networks, including theoretical analyses and model development, neurobiological modeling, and hardware implementation. This book presents the complete spectrum of current research in pulsed neural networks and includes the most important work from many of the key scientists in the field. The first half of the book consists of longer tutorial articles spanning neurobiology, theory, algorithms, and hardware. The second half contains a larger number of shorter research chapters that present more advanced concepts. The contributors use consistent notation and terminology throughout the book.

Author Biography

Wolfgang Maass is Professor at the Institute for Theoretical Computer Science, Technische Universität Graz.

Christopher M. Bishop is Assistant Director at Microsoft Research, Cambridge, and Professor of Computer Science at the University of Edinburgh.

Table of Contents

Foreword

Neural Pulse Coding

Spike Timing

Population Codes

Hippocampal Place Field

Hardware Models

References

Preface

The Isaac Newton Institute

Overview of the Book

Acknowledgments

Contributors

Basic Concepts and Models

Spiking Neurons

The Problem of Neural Coding

Motivation

Rate Codes

Rate as a Spike Count (Average over Time)

Rate as a Spike Density (Average over Several Runs)

Rate as Population Activity (Average over Several Neurons)

Candidate Pulse Codes

Time-to-First-Spike

Phase

Correlations and Synchrony

Stimulus Reconstruction and Reverse Correlation

Discussion: Spikes or Rates?

Neuron Models

Simple Spiking Neuron Model

First Steps towards Coding by Spikes

Threshold-Fire Models

Spike Response Model -- Further Details

Integrate-and-Fire Model

Models of Noise

Conductance-Based Models

Hodgkin-Huxley Model

Relation to the Spike Response Model

Compartmental Models

Rate Models

Conclusions

References

Computing with Spiking Neurons

Introduction

A Formal Computational Model for a Network of Spiking Neurons

McCulloch-Pitts Neurons versus Spiking Neurons

Computing with Temporal Patterns

Conincidence Detection

RBF-Units in the Temporal Domain

Computing a Weighted Sum in Temporal Coding

Universal Approximation of Continuous Functions with Spiking Neurons Remarks:

Other Computations with Temporal Patterns in Networks of Spiking Neurons

Computing with a Space-Rate Code

Computing with Firing Rates

Computing with Firing Rates and Temporal Correlations

Networks of Spiking Neurons for Storing and Retrieving Information

Computing on Spike Trains

Conclusions

References

Pulse-Based Computation in VLSI Neural Networks

Background

Pulsed Coding: A VLSI Perspective

Pulse Amplitude Modulation

Pulse Width Modulation

Pulse Frequency Modulation

Phase or Delay Modulation

Noise, Robustness, Accuracy and Speed

A MOSFET Introduction

Subthreshold Circuits for Neural Networks

Pulse Generation in VLSI

Pulse Intercommunication

Pulsed Arithmetic in VLSI

Addition of Pulse Stream Signals

Multiplication of Pulse Stream Signals

MOS Transconductance Multiplier

MOSFET Analog Multiplier

Learning in Pulsed Systems

Summary and Issues Raised

References

Encoding Information in Neuronal Activity

Introduction

Synchronization and Oscillations

Temporal Binding

Phase Coding

Dynamic Range and Firing Rate Codes

Interspike Interval Variability

Synapses and Rate Coding

Summary and Implications

References

Implementations

Building Silicon Nervous Systems with Dendritic Tree Neuromorphs

Introduction

Why Spikes?

Dendritic Processing of Spikes

Tunability

Implementation in VLSI

Artificial Dendrites

Synapses

Dendritic Non-Linearities

Spike-Generating Soma

Excitability Control

Spike Distribution -- Virtual Wires

Neuromorphs in Action

Feedback to Threshold-Setting Synapses

Discrimination of Complex Spatio-Temporal Patterns

Processing of Temporally Encoded Information

Conclusions

Acknowledgments

References

A Pulse-Coded Communications Infrastructure for Neuromorphic Systems

Introduction

Neuromorphic Computational Nodes

Neuromorphic aVLSI Neurons

Address Event Representation (AER)

Implementations of AER

Silicon Cortex

Basic Layout

Functional Tests of Silicon Cortex

An Example Neuronal Network

An Example of Sensory Input to SCX

Future Research on AER Neuromorphic Systems

Acknowledgements

References

Analog VLSI Pulsed Networks for Perceptive Processing

Introduction

Analog Perceptive Nets Communication Requirements

Coding Information with Pulses

Multiplexing of the Signals Issued by Each Neuron

Non-Arbitered PFM Communication

Analysis of the NAPFM Communication Systems

Statistical Assumptions

Detection

Detection by Time-Windowing

Direct Interpulse Time Measurement

Performance

Detection by Time-Windowing

Direct Interpulse Time Measurement

Data Dependency of System Performance

Discussion

Detection by Time-Windowing

Detection by Direct Interpulse Time Measurement

Address Coding

Silicon Retina Equipped with the NAPFM Communication System

Circuit Description

Noise Measurement Results

Projective Field Generation

Overview

Anisotropic Current Pulse Spreading in a Nonlinear Network

Analysis of the Spatial Response of the Nonlinear Network

Analysis of the Size and Shape of the Bubbles Generable by the Nonlinear Network

Description of the Integrated Circuit for Orientation Enhancement

Overview

Circuit Description

System Measurement Results

Other Applications

Weighted Projective Field Generation

Complex Projective Field Generation

Display Interface

Conclusion

References

Preprocessing for Pulsed Neural VLSI Syste

Introduction

A Sound Segmentation System

Signal Processing in Analog VLSI

Continuous Time Active Filters

Sampled Data Active Switched Capacitor (SC) Filters

Sampled Data Active Switched Current (SI) Filters

Discussion

Palmo -- Pulse Based Signal Processing

Basic Palmo Concepts

The Palmo Signal Representation

The Analog Palmo Cell

A Palmo Signal Processing System

Sources of Harmonic Distortion in a Palmo System

A CMOS Analog Palmo Cell Implementation

The Analog Palmo Cell: Details of Circuit Operation

Interconnecting Analog Palmo Cells

Results from a Palmo VLSI Device

Digital Processing of Palmo Signals

CMOS Analog Palmo Cell: Performance

Conclusions

Further Work

Acknowledgements

References

Digital Simulation of Spiking Neural Networks

Introduction

Implementation Issues of Pulse-Coded Neural Networks

Discrete-Time Simulation

Requisite Arithmetic Precision

Basic Procedures of Network Computation

Programming Environment

Concepts of Efficient Simulation

Mapping Neural Networks on Parallel Computers

Neuron-Parallelism

Synapse-Parallelism

Pattern-Parallelism

Partitioning of the Network

Performance Study

Single PE Workstations

Neurocomputer

Parallel Computers

Results of the Performance Study

Conclusions

References

Design and Analysis of Pulsed Neural Systems

Populations of Spiking Neurons

Introduction

Model

Population Activity Equation

Integral Equation for the Dynamics

Normalization

Noise-Free Population Dynamics

Locking

Locking Condition

Graphical Interpretation

Transients

Incoherent Firing

Determination of the Activity

Stability of Asynchronous Firing

Conclusions

References

Collective Excitation Phenomena and Their Applications

Introduction

Two Variable Formulation of IAF Neurons

Synchronization of Pulse Coupled Oscillators

Clustering via Temporal Segmentation

Limits on Temporal Segmentation

Image Analysis

Image Segmentation

Edge Detection

Solitary Waves

The Importance of Noise

Conclusions

Acknowledgment

References

Computing and Learning with Dynamic Synapses

Introduction

Biological Data on Dynamic Synapses

Quantitative Models

On the Computational Role of Dynamic Synapses

Implications for Learning in Pulsed Neural Nets

Conclusions

References

Stochastic Bit-Stream Neural Networks

Introduction

Basic Neural Modelling

Feedforward Networks and Learning

Probability Level Learning

Bit-Stream Level Learning

Generalization Analysis

Recurrent Networks

Applications to Graph Colouring

Hardware Implementation

The Stochastic Neuron

Calculating Output Derivatives

Generating Stochastic Bit-Streams

Recurrent Networks

Conclusions

References

Hebbian Learning of Pulse Timing in the Barn Owl Auditory System

Introduction

Hebbian Learning

Review of Standard Formulations

Spike-Based Learning

Example

Learning Window

Barn Owl Auditory System

The Localization Task

Auditory Localization Pathway

Phase Locking

Neuron Model

Phase Locking -- Schematic

Simulation Results

Delay Tuning by Hebbian Learning

Motivation

Selection of Delays

Conclusions

References

Table of Contents provided by Publisher. All Rights Reserved.