Advanced Dynamic-System Simulation Model Replication and Monte Carlo Studies

by Korn, Granino A.

• ISBN13: 9781118397350
• ISBN10: 1118397355
• Edition: 2nd
• Format: Hardcover
• Copyright: 2013-04-01
• Publisher: Wiley

Summary

This book introduces Dynamic-system Simulation with a main emphasis on OPEN DESIRE and DESIRE software. The book includes eight comprehensive chapters amounting to approximately 250 pages, as well as includes three appendices housing information on Radial-basis-function, Fuzzy-basis-function Networks, and CLEARN Algorithm. In addition, a CD will be packaged with each book, containing complete binary OPEN DESIRE modeling/simulation program packages for personal-computer LINUX and MS Windows, DESIRE examples, source code and a comprehensive, indexed reference manual. The second edition offers a complete update of all material, boasting two completely new chapters on fast simulation of neural networks.

Author Biography

GRANINO A. KORN, PhD, is Professor of Electrical and Computer Engineering at the University of Arizona and a partner with G.A. and T.M. Korn Industrial Consultants, a company that designs systems for interactive simulation of dynamic systems and neural networks. He is the author of fifteen books, a Fellow of the IEEE, and the recipient of several awards for his work on computer simulation.

Table of Contents

PREFACE xiii

CHAPTER 1 DYNAMIC-SYSTEM MODELS AND SIMULATION 1

SIMULATION IS EXPERIMENTATION WITH MODELS 1

1-1 Simulation and Computer Programs 1

1-2 Dynamic-System Models 2

(a) Difference-Equation Models 2

(b) Differential-Equation Models 2

(c) Discussion 3

1-3 Experiment Protocols Define Simulation Studies 3

1-4 Simulation Software 4

1-5 Fast Simulation Program for Interactive Modeling 5

ANATOMY OF A SIMULATION RUN 8

1-6 Dynamic-System Time Histories Are Sampled Periodically 8

1-7 Numerical Integration 10

(a) Euler Integration 10

(b) Improved Integration Rules 10

1-8 Sampling Times and Integration Steps 11

1-9 Sorting Defined-Variable Assignments 12

SIMPLE APPLICATION PROGRAMS 12

1-10 Oscillators and Computer Displays 12

(a) Linear Oscillator 12

(b) Nonlinear Oscillator: Duffing’s Differential Equation 14

1-11 Space-Vehicle Orbit Simulation with Variable-Step Integration 15

1-12 Population-Dynamics Model 17

1-13 Splicing Multiple Simulation Runs: Billiard-Ball Simulation 17

INRODUCTION TO CONTROL-SYSTEM SIMULATION 21

1-14 Electrical Servomechanism with Motor-Field Delay and Saturation 21

1-15 Control-System Frequency Response 23

1-16 Simulation of a Simple Guided Missile 24

(a) Guided Torpedo 24

(b) Complete Torpedo-Simulation Program 26

STOP AND LOOK 28

1-17 Simulation in the Real World: A Word of Caution 28

References 29

CHAPTER 2 MODELS WITH DIFFERENCE EQUATIONS, LIMITERS, AND

SWITCHES 31

SAMPLED-DATA SYSTEMS AND DIFFERENCE EQUATIONS 31

2-1 Sampled-Data Difference-Equation Systems 31

(a) Introduction 31

(b) Difference Equations 31

(c) A Minefield of Possible Errors 32

2-2 Solving Systems of First-Order Difference Equations 32

(a) General Difference-Equation Model 32

(b) Simple Recurrence Relations 33

2-3 Models Combining Differential Equations and Sampled-Data Operations 35

2-4 Simple Example 35

2-5 Initializing and Resetting Sampled-Data Variables 35

TWO MIXED CONTINUOUS/SAMPLED-DATA SYSTEMS 37

2-6 Guided Torpedo with Digital Control 37

2-7 Simulation of a Plant with a Digital PID Controller 37

DYNAMIC-SYSTEM MODELS WITH LIMITERS

AND SWITCHES 40

2-8 Limiters, Switches, and Comparators 40

(a) Limiter Functions 40

(b) Switching Functions and Comparators 42

2-9 Integration of Switch and Limiter Outputs, Event Prediction, and Display

Problems 43

2-10 Using Sampled-Data Assignments 44

2-11 Using the step Operator and Heuristic Integration-Step Control 44

2-12 Example: Simulation of a Bang-Bang Servomechanism 45

2-13 Limiters, Absolute Values, and Maximum/Minimum Selection 46

2-14 Output-Limited Integration 47

2-15 Modeling Signal Quantization 48

EFFICIENT DEVICE MODELS USING RECURSIVE ASSIGNMENTS 48

2-16 Recursive Switching and Limiter Operations 48

2-17 Track/Hold Simulation 49

2-18 Maximum-Value and Minimum-Value Holding 50

2-19 Simple Backlash and Hysteresis Models 51

2-20 Comparator with Hysteresis (Schmitt Trigger) 52

2-21 Signal Generators and Signal Modulation 53

References 55

CHAPTER 3 FAST VECTOR–MATRIX OPERATIONS AND SUBMODELS 57

ARRAYS, VECTORS, AND MATRICES 57

3-1 Arrays and Subscripted Variables 57

(a) Improved Modeling 57

(b) Array Declarations, Vectors, and Matrices 57

(c) State-Variable Declarations 58

3-2 Vector and Matrices in Experiment Protocols 58

3-3 Time-History Arrays 58

VECTORS AND MODEL REPLICATION 59

3-4 Vector Operations in DYNAMIC Program Segments: The Vectorizing Compiler 59

(a) Vector Assignments and Vector Expressions 59

(b) Vector Differential Equations 60

(c) Vector Sampled–Data Assignments and Difference Equations 60

3-5 Matrix–Vector Products in Vector Expressions 61

(a) Definition 61

(b) Simple Example: Resonating Oscillators 61

3-6 Index-Shift Operation 63

(a) Definition 63

(b) Preview of Significant Applications 63

3-7 Sorting Vector and Subscripted-Variable Assignments 64

3-8 Replication of Dynamic-System Models 64

MORE VECTOR OPERATIONS 65

3-9 Sums, DOT Products, and Vector Norms 65

(a) Sums and DOT Products 65

(b) Euclidean, Taxicab, and Hamming Norms 65

3-10 Maximum/Minimum Selection and Masking 66

(a) Maximum/Minimum Selection 66

(b) Masking Vector Expressions 66

VECTOR EQUIVALENCE DECLARATIONS SIMPLIFY MODELS 67

3-11 Subvectors 67

3-12 Matrix–Vector Equivalence 67

MATRIX OPERATIONS IN DYNAMIC-SYSTEM MODELS 67

3-13 Simple Matrix Assignments 67

3-14 Two-Dimensional Model Replication 68

(a) Matrix Expressions and DOT Products 68

(b) Matrix Differential Equations 68

(c) Matrix Difference Equations 69

VECTORS IN PHYSICS AND CONTROL-SYSTEM PROBLEMS 69

3-15 Vectors in Physics Problems 69

3-16 Vector Model of a Nuclear Reactor 69

3-17 Linear Transformations and Rotation Matrices 70

3-18 State-Equation Models of Linear Control Systems 72

USER-DEFINED FUNCTIONS AND SUBMODELS 72

3-19 Introduction 72

3-20 User-Defined Functions 72

3-21 Submodel Declaration and Invocation 73

3-22 Dealing with Sampled-Data Assignments, Limiters, and Switches 75

References 75

CHAPTER 4 EFFICIENT PARAMETER-INFLUENCE STUDIES AND

STATISTICS COMPUTATION 77

MODEL REPLICATION SIMPLIFIES PARAMETER-INFLUENCE STUDIES 77

4-1 Exploring the Effects of Parameter Changes 77

4-2 Repeated Simulation Runs Versus Model Replication 78

(a) Simple Repeated-Run Study 78

(b) Model Replication (Vectorization) 78

4-3 Programming Parameter-Influence Studies 80

(a) Measures of System Performance 80

(b) Program Design 81

(c) Two-Dimensional Model Replication 81

(d) Cross-Plotting Results 82

(e) Maximum/Minimum Selection 83

(f) Iterative Parameter Optimization 83

STATISTICS 84

4-4 Random Data and Statistics 84

4-5 Sample Averages and Statistical Relative Frequencies 85

COMPUTING STATISTICS BY VECTOR AVERAGING 85

4-6 Fast Computation of Sample Averages 85

4-7 Fast Probability Estimation 86

4-8 Fast Probability-Density Estimation 86

(a) Simple Probability-Density Estimate 86

(b) Triangle and Parzen Windows 87

(c) Computation and Display of Parzen-Window Estimates 88

4-9 Sample-Range Estimation 90

REPLICATED AVERAGES GENERATE SAMPLING DISTRIBUTIONS 91

4-10 Computing Statistics by Time Averaging 91

4-11 Sample Replication and Sampling-Distribution Statistics 91

(a) Introduction 91

(b) Demonstrations of Empirical Laws of Large Numbers 93

(c) Counterexample: Fat-Tailed Distribution 95

RANDOM-PROCESS SIMULATION 95

4-12 Random Processes and Monte Carlo Simulation 95

4-13 Modeling Random Parameters and Random Initial Values 97

4-14 Sampled-Data Random Processes 97

4-15 “Continuous” Random Processes 98

(a) Modeling Continuous Noise 98

(b) Continuous Time Averaging 99

(c) Correlation Functions and Spectral Densities 100

4-16 Problems with Simulated Noise 100

SIMPLE MONTE CARLO EXPERIMENTS 100

4-17 Introduction 100

4-18 Gambling Returns 100

4-19 Vectorized Monte Carlo Study of a Continuous Random Walk 102

References 106

CHAPTER 5 MONTE CARLO SIMULATION OF REAL DYNAMIC SYSTEMS 109

INTRODUCTION 109

5-1 Survey 109

REPEATED-RUN MONTE CARLO SIMULATION 109

5-2 End-of-Run Statistics for Repeated Simulation Runs 109

5-3 Example: Effects of Gun-Elevation Errors on a 1776 Cannnonball Trajectory 110

5-4 Sequential Monte Carlo Simulation 113

VECTORIZED MONTE CARLO SIMULATION 113

5-5 Vectorized Monte Carlo Simulation of the 1776

Cannon Shot 113

5-6 Combined Vectorized and Repeated-Run Monte Carlo Simulation 115

5-7 Interactive Monte Carlo Simulation: Computing Runtime Histories of

Statistics with DYNAMIC-Segment DOT Operations 115

5-8 Example: Torpedo Trajectory Dispersion 117

SIMULATION OF NOISY CONTROL SYSTEMS 119

5-9 Monte Carlo Simulation of a Nonlinear Servomechanism: A Noise-Input Test 119

5-10 Monte Carlo Study of Control-System Errors Caused by Noise 121

ADDITIONAL TOPICS 123

5-11 Monte Carlo Optimization 123

5-12 Convenient Heuristic Method for Testing Pseudorandom Noise 123

5-13 Alternative to Monte Carlo Simulation 123

(a) Introduction 123

(b) Dynamic Systems with Random Perturbations 123

(c) Mean-Square Errors in Linearized Systems 124

References 125

CHAPTER 6 VECTOR MODELS OF NEURAL NETWORKS 127

ARTIFICIAL NEURAL NETWORKS 127

6-1 Introduction 127

6-2 Artificial Neural Networks 127

6-3 Static Neural Networks: Training, Validation, and Applications 128

6-4 Dynamic Neural Networks 129

SIMPLE VECTOR ASSIGNMENTS MODEL NEURON LAYERS 130

6-5 Neuron-Layer Declarations and Neuron Operations 130

6-6 Neuron-Layer Concatenation Simplifies Bias Inputs 130

6-7 Normalizing and Contrast-Enhancing Layers 131

(a) Pattern Normalization 131

(b) Contrast Enhancement: Softmax and Thresholding 131

6-8 Multilayer Networks 132

6-9 Exercising a Neural-Network Model 132

(a) Computing Successive Neuron-Layer Outputs 132

(b) Input from Pattern-Row Matrices 133

(c) Input from Text Files and Spreadsheets 133

SUPERVISED TRAINING FOR REGRESSION 134

6-10 Mean-Square Regression 134

(a) Problem Statement 134

(b) Linear Mean-Square Regression and the Delta Rule 135

(c) Nonlinear Neuron Layers and Activation-Function Derivatives 136

(d) Error-Measure Display 136

6-11 Backpropagation Networks 137

(a) The Generalized Delta Rule 137

(b) Momentum Learning 139

(c) Simple Example 139

(d) The Classical XOR Problem and Other Examples 140

MORE NEURAL-NETWORK MODELS 140

6-12 Functional-Link Networks 140

6-13 Radial-Basis-Function Networks 142

(a) Basis-Function Expansion and Linear Optimization 142

(b) Radial Basis Functions 143

6-14 Neural-Network Submodels 145

PATTERN CLASSIFICATION 146

6-15 Introduction 146

6-16 Classifier Input from Files 147

6-17 Classifier Networks 147

(a) Simple Linear Classifiers 147

(b) Softmax Classifiers 148

(c) Backpropagation Classifiers 148

(d) Functional-Link Classifiers 149

(e) Other Classsifiers 149

6-18 Examples 149

(a) Classification Using an Empirical Database: Fisher’s Iris Problem 149

(b) Image-Pattern Recognition and Associative Memory 151

PATTERN SIMPLIFICATION 155

6-19 Pattern Centering 155

6-20 Feature Reduction 156

(a) Bottleneck Layers and Encoders 156

(b) Principal Components 156

NETWORK-TRAINING PROBLEMS 157

6-21 Learning-Rate Adjustment 157

6-22 Overfitting and Generalization 157

(a) Introduction 157

(b) Adding Noise 158

(c) Early Stopping 158

(d) Regularization 159

6-23 Beyond Simple Gradient Descent 159

UNSUPERVISED COMPETITIVE-LAYER CLASSIFIERS 159

6-24 Template-Pattern Matching and the CLEARN Operation 159

(a) Template Patterns and Template Matrix 159

(b) Matching Known Template Patterns 160

(c) Template-Pattern Training 160

(d) Correlation Training 162

6-25 Learning with Conscience 163

6-26 Competitive-Learning Experiments 164

(a) Pattern Classification 164

(b) Vector Quantization 164

6-27 Simplified Adaptive-Resonance Emulation 165

SUPERVISED COMPETITIVE LEARNING 167

6-28 The LVQ Algorithm for Two-Way Classification 167

6-29 Counterpropagation Networks 167

EXAMPLES OF CLEARN CLASSIFIERS 168

6-30 Recognition of Known Patterns 168

(a) Image Recognition 168

(b) Fast Solution of the Spiral Benchmark Problem 169

6-31 Learning Unknown Patterns 173

References 174

CHAPTER 7 DYNAMIC NEURAL NETWORKS 177

INTRODUCTION 177

7-1 Dynamic Versus Static Neural Networks 177

7-2 Applications of Dynamic Neural Networks 177

7-3 Simulations Combining Neural Networks and Differential-Equation Models 178

NEURAL NETWORKS WITH DELAY-LINE INPUT 178

7-4 Introduction 178

7-5 The Delay-Line Model 180

7-6 Delay-Line-Input Networks 180

(a) Linear Combiners 180

(b) One-Layer Nonlinear Network 181

(c) Functional-Link Network 181

(d) Backpropagation Network with Delay-Line Input 182

7-7 Using Gamma Delay Lines 182

STATIC NEURAL NETWORKS USED AS DYNAMIC NETWORKS 183

7-8 Introduction 183

7-9 Simple Backpropagation Networks 184

RECURRENT NEURAL NETWORKS 185

7-10 Layer-Feedback Networks 185

7-11 Simplified Recurrent-Network Models Combine Context and Input Layers 185

(a) Conventional Model of a Jordan Network 185

(b) Simplified Jordan-Network Model 186

(c) Simplified Models for Other Feedback Networks 187

7-12 Neural Networks with Feedback Delay Lines 187

(a) Delay-Line Feedback 187

(b) Neural Networks with Both Input and Feedback Delay Lines 188

7-13 Teacher Forcing 189

PREDICTOR NETWORKS 189

7-14 Off-Line Predictor Training 189

(a) Off-Line Prediction Using Stored Time Series 189

(b) Off-Line Training System for Online Predictors 189

(c) Example: Simple Linear Predictor 190

7-15 Online Trainng for True Online Prediction 192

7-16 Chaotic Time Series for Prediction Experiments 192

7-17 Gallery of Predictor Networks 193

OTHER APPLICATIONS OF DYNAMIC NETWORKS 199

7-18 Temporal-Pattern Recognition: Regression and Classification 199

7-19 Model Matching 201

(a) Introduction 201

(b) Example: Program for Matching Narendra’s Plant Model 201

MISCELLANEOUS TOPICS 204

7-20 Biological-Network Software 204

References 204

CHAPTER 8 MORE APPLICATIONS OF VECTOR MODELS 207

VECTORIZED SIMULATION WITH LOGARITHMIC PLOTS 207

8-1 The EUROSIM No. 1 Benchmark Problem 207

8-2 Vectorized Simulation with Logarithmic Plots 207

MODELING FUZZY-LOGIC FUNCTION GENERATORS 209

8-3 Rule Tables Specify Heuristic Functions 209

8-4 Fuzzy-Set Logic 210

(a) Fuzzy Sets and Membership Functions 210

(b) Fuzzy Intersections and Unions 210

(c) Joint Membership Functions 213

(d) Normalized Fuzzy-Set Partitions 213

8-5 Fuzzy-Set Rule Tables and Function Generators 214

8-6 Simplified Function Generation with Fuzzy Basis Functions 214

8-7 Vector Models of Fuzzy-Set Partitions 215

(a) Gaussian Bumps: Effects of Normalization 215

(b) Triangle Functions 215

(c) Smooth Fuzzy-Basis Functions 216

8-8 Vector Models for Multidimensional Fuzzy-Set Partitions 216

8-9 Example: Fuzzy-Logic Control of a Servomechanism 217

(a) Problem Statement 217

(b) Experiment Protocol and Rule Table 217

(c) DYNAMIC Program Segment and Results 220

PARTIAL DIFFERENTIAL EQUATIONS 221

8-10 Method of Lines 221

8-11 Vectorized Method of Lines 221

(a) Introduction 221

(b) Using Differentiation Operators 221

(c) Numerical Problems 224

8-12 Heat-Conduction Equation in Cylindrical Coordinates 225

8-13 Generalizations 225

8-14 Simple Heat-Exchanger Model 227

FOURIER ANALYSIS AND LINEAR-SYSTEM DYNAMICS 229

8-15 Introduction 229

8-16 Function-Table Lookup and Interpolation 230

8-17 Fast-Fourier-Transform Operations 230

8-18 Impulse and Freqency Response of a Linear Servomechanism 231

8-19 Compact Vector Models of Linear Dynamic Systems 232

(a) Using the Index-Shift Operation with Analog Integration 232

(b) Linear Sampled-Data Systems 235

(c) Example: Digital Comb Filter 236

REPLICATION OF AGROECOLOGICAL MODELS ON MAP GRIDS 237

8-20 Geographical Information System 237

8-21 Modeling the Evolution of Landscape Features 239

8-22 Matrix Operations on a Map Grid 239

References 242

APPENDIX: ADDITIONAL REFERENCE MATERIAL 245

A-1 Example of a Radial-Basis-Function Network 245

A-2 Fuzzy-Basis-Function Network 245

References 248

USING THE BOOK CD 251

INDEX 253