Event-based Neuromorphic Systems

Neuromorphic electronic engineering takes its inspiration from the functioning of nervous systems to build more power efficient electronic sensors and processors. Event-based neuromorphic systems are inspired by the brain's efficient data-driven communication design, which is key to its quick responses and remarkable capabilities.  This cross-disciplinary text establishes how circuit building blocks are combined in architectures to construct complete systems. These include vision and auditory sensors as well as neuronal processing and learning circuits that implement models of nervous systems.

Techniques for building multi-chip scalable systems are considered throughout the book, including methods for dealing with transistor mismatch, extensive discussions of communication and interfacing, and making systems that operate in the real world. The book also provides historical context that helps relate the architectures and circuits to each other and that guides readers to the extensive literature. Chapters are written by founding experts and have been extensively edited for overall coherence.

This pioneering text is an indispensable resource for practicing neuromorphic electronic engineers, advanced electrical engineering and computer science students and researchers interested in neuromorphic systems.

Key features:

• Summarises the latest design approaches, applications, and future challenges in the field of neuromorphic engineering.
• Presents examples of practical applications of neuromorphic design principles.
• Covers address-event communication, retinas, cochleas, locomotion, learning theory, neurons, synapses, floating gate circuits, hardware and software infrastructure, algorithms, and future challenges.

Shih-Chii Liu is a group leader at the Institute of Neuroinformatics, University of Zurich and ETH Zurich.  She received her Ph.D. in the Computation and Neural Systems program at Caltech. She has been an instructor and topic organizer at the NSF Telluride Neuromorphic Cognition Engineering Workshop in Telluride, Colorado since 1998. She has also co-authored a book on analog VLSI circuits (published by MIT Press), is an IEEE Senior member and has held offices in a number of scientific and IEEE engineering international conferences. Dr Liu has been working on event-based vision and auditory sensors, multi-neuron networks, and asynchronous circuits for more than 20 years.

Tobi Delbruck has been Professor of Physics and Electrical Engineering at the Institute of Neuroinformatics since 1998. He leads the Sensors group which focuses on neuromorphic sensors and processing. He received his Ph.D. in the Computation and Neural Systems program at Caltech.  He worked on electronic imaging at Arithmos, Synaptics, National Semiconductor, and Foveon. He co-organized the Telluride Neuromorphic Cognition Engineering summer workshop and the live demonstration sessions at ISCAS and NIPS, and is former chair of the IEEE CAS Sensory Systems Technical Committee. He has been awarded 9 IEEE awards and is an IEEE Fellow.

Giacomo Indiveri is a Professor at the University of Zurich’s Faculty of Science. He obtained his M.Sc. degree in Electrical Engineering and his Ph.D. degree in Computer Science from the University of Genoa, Italy. He is an ERC fellow and an IEEE Senior member. His research interests lie in the study of real and artificial neural processing systems, and in the hardware implementation of neuromorphic cognitive systems, using full custom analog and digital VLSI technology.

Adrian M. Whatley gained a degree in Chemistry at the University of Bristol in England in 1986. After working for 10 years in the British computer industry, he took up his current software engineering position at the Institute of Neuroinformatics where he works primarily on asynchronous Address-Event communication systems.

Rodney Douglas is a co-founder of the Institute of Neuroinformatics. His central research interest over the past 25 years has been the nature of computation by the circuits of the neocortex and their implementation both in software simulation, in custom electronic hardware. The experimental aspect of his work has inspired a number of cortical models of processing that use recurrently connected neuronal architectures.  He is currently exploring principles of self-assembly in simple organisms and circuits which he considers crucial for building truly autonomous neuromorphic cognitive systems.

Foreword xvii

Acknowledgments xix

List of Abbreviations and Acronyms xxi

1 Introduction 1

1.1 Origins and Historical Context 3

1.2 Building Useful Neuromorphic Systems 5

References 5

Part I UNDERSTANDING NEUROMORPHIC SYSTEMS 7

2 Communication 9

2.1 Introduction 9

2.2 Address-Event Representation 12

2.2.1 AER Encoders 13

2.2.2 Arbitration Mechanisms 13

2.2.3 Encoding Mechanisms 17

2.2.4 Multiple AER Endpoints 19

2.2.5 Address Mapping 19

2.2.6 Routing 19

2.3 Considerations for AER Link Design 20

2.3.1 Trade-off: Dynamic or Static Allocation 21

2.3.2 Trade-off: Arbitered Access or Collisions? 23

2.3.3 Trade-off: Queueing versus Dropping Spikes 24

2.3.4 Predicting Throughput Requirements 25

2.3.5 Design Trade-offs 27

2.4 The Evolution of AER Links 28

2.4.1 Single Sender, Single Receiver 28

2.4.2 Multiple Senders, Multiple Receivers 30

2.4.3 Parallel Signal Protocol 31

2.4.4 Word-Serial Addressing 32

2.4.5 Serial Differential Signaling 33

2.5 Discussion 34

References 35

3 Silicon Retinas 37

3.1 Introduction 37

3.2 Biological Retinas 38

3.3 Silicon Retinas with Serial Analog Output 39

3.4 Asynchronous Event-Based Pixel Output Versus Synchronous Frames 40

3.5 AER Retinas 40

3.5.1 Dynamic Vision Sensor 41

3.5.2 Asynchronous Time-Based Image Sensor 46

3.5.3 Asynchronous Parvo–Magno Retina Model 46

3.5.4 Event-Based Intensity-Coding Imagers (Octopus and TTFS) 48

3.5.5 Spatial Contrast and Orientation Vision Sensor (VISe) 50

3.6 Silicon Retina Pixels 54

3.6.1 DVS Pixel 54

3.6.2 ATIS Pixel 56

3.6.3 VISe Pixel 58

3.6.4 Octopus Pixel 59

3.7 New Specifications for Silicon Retinas 60

3.7.1 DVS Response Uniformity 60

3.7.2 DVS Background Activity 62

3.7.3 DVS Dynamic Range 62

3.7.4 DVS Latency and Jitter 63

3.8 Discussion 64

References 67

4 Silicon Cochleas 71

4.1 Introduction 72

4.2 Cochlea Architectures 75

4.2.1 Cascaded 1D 76

4.2.2 Basic 1D Silicon Cochlea 77

4.2.3 2D Architecture 78

4.2.4 The Resistive (Conductive) Network 79

4.2.5 The BM Resonators 80

4.2.6 The 2D Silicon Cochlea Model 80

4.2.7 Adding the Active Nonlinear Behavior of the OHCs 82

4.3 Spike-Based Cochleas 83

4.3.1 Q-control of AEREAR2 Filters 85

4.3.2 Applications: Spike-Based Auditory Processing 86

4.4 Tree Diagram 87

4.5 Discussion 87

References 89

5 Locomotion Motor Control 91

5.1 Introduction 92

5.1.1 Determining Functional Biological Elements 92

5.1.2 Rhythmic Motor Patterns 93

5.2 Modeling Neural Circuits in Locomotor Control 95

5.2.1 Describing Locomotor Behavior 96

5.2.2 Fictive Analysis 97

5.2.3 Connection Models 99

5.2.4 Basic CPG Construction 100

5.2.5 Neuromorphic Architectures 102

5.3 Neuromorphic CPGs at Work 108

5.3.1 A Neuroprosthesis: Control of Locomotion in Vivo 109

5.3.2 Walking Robots 111

5.3.3 Modeling Intersegmental Coordination 112

5.4 Discussion 113

References 115

6 Learning in Neuromorphic Systems 119

6.1 Introduction: Synaptic Connections, Memory, and Learning 120

6.2 Retaining Memories in Neuromorphic Hardware 121

6.2.1 The Problem of Memory Maintenance: Intuition 121

6.2.2 The Problem of Memory Maintenance: Quantitative Analysis 122

6.2.3 Solving the Problem of Memory Maintenance 124

6.3 Storing Memories in Neuromorphic Hardware 128

6.3.1 Synaptic Models for Learning 128

6.3.2 Implementing a Synaptic Model in Neuromorphic Hardware 132

6.4 Toward Associative Memories in Neuromorphic Hardware 136

6.4.1 Memory Retrieval in Attractor Neural Networks 137

6.4.2 Issues 142

6.5 Attractor States in a Neuromorphic Chip 143

6.5.1 Memory Retrieval 143

6.5.2 Learning Visual Stimuli in Real Time 145

6.6 Discussion 148

References 149

Part II BUILDING NEUROMORPHIC SYSTEMS 153

7 Silicon Neurons 155

7.1 Introduction 156

7.2 Silicon Neuron Circuit Blocks 158

7.2.1 Conductance Dynamics 158

7.2.2 Spike-Event Generation 159

7.2.3 Spiking Thresholds and Refractory Periods 161

7.2.4 Spike-Frequency Adaptation and Adaptive Thresholds 162

7.2.5 Axons and Dendritic Trees 164

7.2.6 Additional Useful Building Blocks 165

7.3 Silicon Neuron Implementations 166

7.3.1 Subthreshold Biophysically Realistic Models 166

7.3.2 Compact I&F Circuits for Event-Based Systems 169

7.3.3 Generalized I&F Neuron Circuits 170

7.3.4 Above Threshold, Accelerated-Time, and Switched-Capacitor Designs 174

7.4 Discussion 176

References 180

8 Silicon Synapses 185

8.1 Introduction 186

8.2 Silicon Synapse Implementations 188

8.2.1 Non Conductance-Based Circuits 188

8.2.2 Conductance-Based Circuits 198

8.2.3 NMDA Synapse 200

8.3 Dynamic Plastic Synapses 201

8.3.1 Short-Term Plasticity 201

8.3.2 Long-Term Plasticity 203

8.4 Discussion 213

References 215

9 Silicon Cochlea Building Blocks 219

9.1 Introduction 219

9.2 Voltage-Domain Second-Order Filter 220

9.2.1 Transconductance Amplifier 220

9.2.2 Second-Order Low-Pass Filter 222

9.2.3 Stability of the Filter 223

9.2.4 Stabilised Second-Order Low-Pass Filter 225

9.2.5 Differentiation 225

9.3 Current-Domain Second-Order Filter 227

9.3.1 The Translinear Loop 227

9.3.2 Second-Order Tau Cell Log-Domain Filter 229

9.4 Exponential Bias Generation 230

9.5 The Inner Hair Cell Model 233

9.6 Discussion 234

References 234

10 Programmable and Configurable Analog Neuromorphic ICs 237

10.1 Introduction 238

10.2 Floating-Gate Circuit Basics 238

10.3 Floating-Gate Circuits Enabling Capacitive Circuits 238

10.4 Modifying Floating-Gate Charge 242

10.4.1 Electron Tunneling 242

10.4.2 pFET Hot-Electron Injection 242

10.5 Accurate Programming of Programmable Analog Devices 244

10.6 Scaling of Programmable Analog Approaches 246

10.7 Low-Power Analog Signal Processing 247

10.8 Low-Power Comparisons to Digital Approaches: Analog Computing in Memory 249

10.9 Analog Programming at Digital Complexity: Large-Scale Field Programmable Analog Arrays 251

10.10 Applications of Complex Analog Signal Processing 253

10.10.1 Analog Transform Imagers 253

10.10.2 Adaptive Filters and Classifiers 253

10.11 Discussion 256

References 257

11 Bias Generator Circuits 261

11.1 Introduction 261

11.2 Bias Generator Circuits 263

11.2.1 Bootstrapped Current Mirror Master Bias Current Reference 263

11.2.2 Master Bias Power Supply Rejection Ratio (PSRR) 265

11.2.3 Stability of the Master Bias 265

11.2.4 Master Bias Startup and Power Control 266

11.2.5 Current Splitters: Obtaining a Digitally Controlled Fraction of the Master Current 267

11.2.6 Achieving Fine Monotonic Resolution of Bias Currents 271

11.2.7 Using Coarse–Fine Range Selection 273

11.2.8 Shifted-Source Biasing for Small Currents 274

11.2.9 Buffering and Bypass Decoupling of Individual Biases 275

11.2.10 A General Purpose Bias Buffer Circuit 278

11.2.11 Protecting Bias Splitter Currents from Parasitic Photocurrents 279

11.3 Overall Bias Generator Architecture Including External Controller 279

11.4 Typical Characteristics 280

11.5 Design Kits 281

11.6 Discussion 282

References 282

12 On-Chip AER Communication Circuits 285

12.1 Introduction 286

12.1.1 Communication Cycle 286

12.1.2 Speedup in Communication 287

12.2 AER Transmitter Blocks 289

12.2.1 AER Circuits within a Pixel 289

12.2.2 Arbiter 290

12.2.3 Other AER Blocks 295

12.2.4 Combined Operation 297

12.3 AER Receiver Blocks 298

12.3.1 Chip-Level Handshaking Block 298

12.3.2 Decoder 299

12.3.3 Handshaking Circuits in Receiver Pixel 300

12.3.4 Pulse Extender Circuits 301

12.3.5 Receiver Array Peripheral Handshaking Circuits 301

12.4 Discussion 302

References 303

13 Hardware Infrastructure 305

13.1 Introduction 306

13.1.1 Monitoring AER Events 307

13.1.2 Sequencing AER Events 311

13.1.3 Mapping AER Events 313

13.2 Hardware Infrastructure Boards for Small Systems 316

13.2.1 Silicon Cortex 316

13.2.2 Centralized Communication 317

13.2.3 Composable Architecture Solution 318

13.2.4 Daisy-Chain Architecture 324

13.2.5 Interfacing Boards using Serial AER 324

13.2.6 Reconfigurable Mesh-Grid Architecture 328

13.3 Medium-Scale Multichip Systems 329

13.3.1 Octopus Retina + IFAT 329

13.3.2 Multichip Orientation System 332

13.3.3 CAVIAR 335

13.4 FPGAs 340

13.5 Discussion 342

References 345

14 Software Infrastructure 349

14.1 Introduction 349

14.1.1 Importance of Cross-Community Commonality 350

14.2 Chip and System Description Software 350

14.2.1 Extensible Markup Language 351

14.2.2 NeuroML 351

14.3 Configuration Software 352

14.4 Address Event Stream Handling Software 352

14.4.1 Field-Programmable Gate Arrays 353

14.4.2 Structure of AE Stream Handling Software 353

14.4.3 Bandwidth and Latency 353

14.4.4 Optimization 354

14.4.5 Application Programming Interface 355

14.4.6 Network Transport of AE Streams 355

14.5 Mapping Software 356

14.6 Software Examples 357

14.6.1 ChipDatabase – A System for Tuning Neuromorphic aVLSI Chips 357

14.6.2 Spike Toolbox 359

14.6.3 jAER 359

14.6.4 Python and PyNN 360

14.7 Discussion 363

References 363

15 Algorithmic Processing of Event Streams 365

15.1 Introduction 365

15.2 Requirements for Software Infrastructure 367

15.2.1 Processing Latency 369

15.3 Embedded Implementations 369

15.4 Examples of Algorithms 370

15.4.1 Noise Reduction Filters 370

15.4.2 Time-Stamp Maps and Subsampling by Bit-Shifting Addresses 372

15.4.3 Event Labelers as Low-Level Feature Detectors 372

15.4.4 Visual Trackers 374

15.4.5 Event-Based Audio Processing 378

15.5 Discussion 379

References 379

16 Towards Large-Scale Neuromorphic Systems 381

16.1 Introduction 381

16.2 Large-Scale System Examples 382

16.2.1 Spiking Neural Network Architecture 382

16.2.2 Hierarchical AER 384

16.2.3 Neurogrid 386

16.2.4 High Input Count Analog Neural Network System 388

16.3 Discussion 390

References 391

17 The Brain as Potential Technology 393

17.1 Introduction 393

17.2 The Nature of Neuronal Computation: Principles of Brain Technology 395

17.3 Approaches to Understanding Brains 396

17.4 Some Principles of Brain Construction and Function 398

17.5 An Example Model of Neural Circuit Processing 400

17.6 Toward Neuromorphic Cognition 402

References 404

Index 407

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.