Robust Adaptive Dynamic Programming

by Jiang, Yu; Jiang, Zhong-ping

• ISBN13: 9781119132646
• ISBN10: 1119132649
• Edition: 1st
• Format: Hardcover
• Copyright: 2017-05-08
• Publisher: Wiley-IEEE Press

Summary

A comprehensive look at state-of-the-art ADP theory and real-world applications

This book fills a gap in the literature by providing a theoretical framework for integrating techniques from adaptive dynamic programming (ADP) and modern nonlinear control to address data-driven optimal control design challenges arising from both parametric and dynamic uncertainties.

Traditional model-based approaches leave much to be desired when addressing the challenges posed by the ever-increasing complexity of real-world engineering systems. An alternative which has received much interest in recent years are biologically-inspired approaches, primarily RADP. Despite their growing popularity worldwide, until now books on ADP have focused nearly exclusively on analysis and design, with scant consideration given to how it can be applied to address robustness issues, a new challenge arising from dynamic uncertainties encountered in common engineering problems. 

Robust Adaptive Dynamic Programming zeros in on the practical concerns of engineers. The authors develop RADP theory from linear systems to partially-linear, large-scale, and completely nonlinear systems. They provide in-depth coverage of state-of-the-art applications in power systems, supplemented with numerous real-world examples implemented in MATLAB. They also explore fascinating reverse engineering topics, such how ADP theory can be applied to the study of the human brain and cognition. In addition, the book:  

• Covers the latest developments in RADP theory and applications for solving a range of systems’ complexity problems
• Explores multiple real-world implementations in power systems with illustrative examples backed up by reusable MATLAB code and Simulink block sets
• Provides an overview of nonlinear control, machine learning, and dynamic control
• Features discussions of novel applications for RADP theory, including an entire chapter on how it can be used as a computational mechanism of human movement control

Robust Adaptive Dynamic Programming is both a valuable working resource and an intriguing exploration of contemporary ADP theory and applications for practicing engineers and advanced students in systems theory, control engineering, computer science, and applied mathematics.

Author Biography

Yu Jiang, PhD, is a Software Developer of Simulink Control Design at MathWorks. In the past five years, he has published nearly 30 papers and one book chapter on the subject of ADP theory and its applications. He was the recipient of the Shimemura Young Author Prize (with Prof. Z.P.Jiang) at the 9th Asian Control Conference in Istanbul, Turkey, 2013.

Zhong-Ping Jiang, PhD, is a Professor of Electrical and Computer Engineering at New York University with a doctorate in automatic control and mathematics. He has authored three books and over 400 journal and conference papers on nonlinear systems and control, dynamic networks, and more.

Table of Contents

ABOUT THE AUTHORS xi

PREFACE AND ACKNOWLEDGMENTS xiii

ACRONYMS xvii

GLOSSARY xix

1 INTRODUCTION 1

1.1 From RL to RADP 1

1.2 Summary of Each Chapter 5

References 6

2 ADAPTIVE DYNAMIC PROGRAMMING FOR UNCERTAIN LINEAR SYSTEMS 11

2.1 Problem Formulation and Preliminaries 11

2.2 Online Policy Iteration 14

2.3 Learning Algorithms 16

2.4 Applications 24

2.5 Notes 29

References 30

3 SEMI-GLOBAL ADAPTIVE DYNAMIC PROGRAMMING 35

3.1 Problem Formulation and Preliminaries 35

3.2 Semi-Global Online Policy Iteration 38

3.3 Application 43

3.4 Notes 46

References 46

4 GLOBAL ADAPTIVE DYNAMIC PROGRAMMING FOR NONLINEAR POLYNOMIAL SYSTEMS 49

4.1 Problem Formulation and Preliminaries 49

4.2 Relaxed HJB Equation and Suboptimal Control 52

4.3 SOS-Based Policy Iteration for Polynomial Systems 55

4.4 Global ADP for Uncertain Polynomial Systems 59

4.5 Extension for Nonlinear Non-Polynomial Systems 64

4.6 Applications 70

4.7 Notes 81

References 81

5 ROBUST ADAPTIVE DYNAMIC PROGRAMMING 85

5.1 RADP for Partially Linear Composite Systems 86

5.2 RADP for Nonlinear Systems 97

5.3 Applications 103

5.4 Notes 109

References 110

6 ROBUST ADAPTIVE DYNAMIC PROGRAMMING FOR LARGE-SCALE SYSTEMS 113

6.1 Stability and Optimality for Large-Scale Systems 113

6.2 RADP for Large-Scale Systems 122

6.3 Extension for Systems with Unmatched Dynamic Uncertainties 124

6.4 Application to a Ten-Machine Power System 128

6.5 Notes 132

References 133

7 ROBUST ADAPTIVE DYNAMIC PROGRAMMING AS A THEORY OF SENSORIMOTOR CONTROL 137

7.1 ADP for Continuous-Time Stochastic Systems 138

7.2 RADP for Continuous-Time Stochastic Systems 143

7.3 Numerical Results: ADP-Based Sensorimotor Control 153

7.4 Numerical Results: RADP-Based Sensorimotor Control 165

7.5 Discussion 167

7.6 Notes 172

References 173

A BASIC CONCEPTS IN NONLINEAR SYSTEMS 177

A.1 Lyapunov Stability 177

A.2 ISS and the Small-Gain Theorem 178

B SEMIDEFINITE PROGRAMMING AND SUM-OF-SQUARES PROGRAMMING 181

B.1 SDP and SOSP 181

C PROOFS 183

C.1 Proof of Theorem 3.1.4 183

C.2 Proof of Theorem 3.2.3 186

References 188

INDEX 191