Dynamic programming is a powerful method for solving optimization problems, but has a number of drawbacks that limit its use to solving problems of very low dimension. To overcome these limitations, author Rein Luus suggested using it in an iterative fashion. Although this method required vast computer resources, modifications to his original scheme have made the computational procedure feasible.With iteration, dynamic programming becomes an effective optimization procedure for very high-dimensional optimal control problems and has demonstrated applicability to singular control problems. Recently, iterative dynamic programming (IDP) has been refined to handle inequality state constraints and noncontinuous functions.Iterative Dynamic Programming offers a comprehensive presentation of this powerful tool. It brings together the results of work carried out by the author and others - previously available only in scattered journal articles - along with the insight that led to its development. The author provides the necessary background, examines the effects of the parameters involved, and clearly illustrates IDP's advantages.

Rein Luus is a Professor in the Department of Chemical Engineering at the University of Toronto, Canada.

Fundamental concepts

1

(35)

Introduction

1

(1)

Fundamental definitions and notation

2

(5)

Operator

2

(1)

Vectors and matrices

3

(2)

Differentiation of a vector

5

(1)

Taylor series expansion

5

(1)

Norm of a vector

6

(1)

Sign definite

6

(1)

Stationary and maxima (minima) points

7

(1)

Steady-state system model

7

(1)

Continuous-time system model

7

(1)

Discrete-time system model

8

(2)

The performance index

10

(1)

Interpretation of results

11

(1)

Examples of systems for optimal control

11

(6)

Linear gas absorber

11

(2)

Nonlinear continuous stirred tank reactor

13

(2)

Photochemical reaction in CSTR

15

(1)

Production of secreted protein in a fed-batch reactor

16

(1)

Solving algebraic equations

17

(15)

Separation of the equations into two groups

18

(1)

Numerical examples

19

(12)

Application to multicomponent distillation

31

(1)

Solving ordinary differential equations

32

(1)

References

32

(3)

Steady-state optimization

35

(32)

Introduction

35

(1)

Linear programming

35

(9)

Example -- diet problem with 5 foods

38

(4)

Interpretation of shadow prices

42

(2)

LJ optimization procedure

44

(20)

Determination of region size

46

(2)

Simple example -- 5 food diet problem

48

(1)

Model reduction example

48

(6)

Parameter estimation

54

(4)

Handling equality constraints

58

(6)

References

64

(3)

Dynamic programming

67

(14)

Introduction

67

(1)

Examples

67

(13)

A simple optimal path problem

68

(1)

Job allocation problem

69

(3)

The stone problem

72

(1)

Simple optimal control problem

73

(2)

Linear optimal control problem

75

(1)

Cross-current extraction system

76

(4)

Limitations of dynamic programming

80

(1)

References

80

(1)

Iterative dynamic programming

81

(10)

Introduction

81

(1)

Construction of time stages

82

(1)

Construction of grid for x

82

(1)

Allowable values for control

82

(1)

First iteration

82

(2)

Stage P

83

(1)

Stage P -- 1

83

(1)

Continuation in backward direction

83

(1)

Iterations with systematic reduction in region size

84

(1)

Example

85

(1)

Use of accessible states as grid points

85

(1)

Algorithm for IDP

86

(3)

Early applications of IDP

89

(1)

References

90

(1)

Allowable values for control

91

(8)

Introduction

91

(1)

Comparison of uniform distribution to random choice

92

(6)

Uniform distribution

93

(1)

Random choice

94

(4)

References

98

(1)

Evaluation of parameters in IDP

99

(20)

Introduction

99

(1)

Number of grid points

100

(6)

Bifunctional catalyst blend optimization problem

100

(4)

Photochemical CSTR

104

(2)

Multi-pass approach

106

(3)

Nonlinear two-stage CSTR system

107

(2)

Further example

109

(8)

Effect of region restoration factor η

111

(1)

Effect of the region contraction factor γ

112

(1)

Effect of the number of time stages

112

(5)

References

117

(2)

Piecewise linear control

119

(20)

Introduction

119

(1)

Problem formulation

119

(1)

Algorithm for IDP for piecewise linear control

120

(2)

Numerical examples

122

(16)

Nonlinear CSTR

122

(2)

Nondifferentiable system

124

(2)

Linear system with quadratic performance index

126

(10)

Gas absorber with a large number of plates

136

(2)

References

138

(1)

Time-delay systems

139

(10)

Introduction

139

(1)

Problem formulation

140

(1)

Examples

140

(8)

Example 1

140

(2)

Example 2

142

(1)

Example 3 -- Nonlinear two-stage CSTR system

143

(5)

References

148

(1)

Variable stage lengths

149

(28)

Introduction

149

(6)

Variable stage-lengths when final time is free

155

(2)

IDP algorithm

156

(1)

Problems where final time is not specified

157

(8)

Oil shale pyrolysis problem

158

(3)

Modified Denbigh reaction scheme

161

(4)

Systems with specified final time

165

(10)

Fed-batch reactor

168

(7)

References

175

(2)

Singular control problems

177

(22)

Introduction

177

(1)

Four simple-looking examples

178

(13)

Example 1

178

(3)

Example 2

181

(6)

Example 3

187

(1)

Example 4

188

(3)

Yeo's singular control problem

191

(2)

Nonlinear two-stage CSTR problem

193

(4)

References

197

(2)

State constraints

199

(38)

Introduction

199

(1)

Final state constraints

199

(25)

Problem formulation

199

(1)

Quadratic penalty function with shifting terms

200

(18)

Absolute value penalty function

218

(5)

Remarks on the choice of penalty functions

223

(1)

State inequality constraints

224

(10)

Problem formulation

224

(1)

State constraint variables

225

(9)

References

234

(3)

Time optimal control

237

(18)

Introduction

237

(1)

Time optimal control problem

237

(1)

Direct approach to time optimal control

238

(1)

Examples

239

(12)

Example 1: Bridge crane system

239

(2)

Example 2: Two-link robotic arm

241

(2)

Example 3: Drug displacement problem

243

(3)

Example 4: Two-stage CSTR system

246

(3)

Example 5

249

(2)

High dimensional systems

251

(2)

References

253

(2)

Nonseparable problems

255

(10)

Introduction

255

(1)

Problem formulation

255

(1)

Examples

256

(8)

Example 1 -- Luus-Tassone problem

256

(8)

Example 2 -- Li-Haimes problem

264

(1)

References

264

(1)

Sensitivity considerations

265

(8)

Introduction

265

(2)

Example: Lee-Ramirez bioreactor

267

(5)

Solution by IDP

268

(4)

References

272

(1)

Toward practical optimal control

273

(10)

Introduction

273

(1)

Optimal control of oil shale pyrolysis

274

(4)

Future directions

278

(4)

References

282

(1)

A Nonlinear algebraic equation solver

283

(4)

Program listing

283

(2)

Output of the program

285

(2)

B Listing of linear programming program

287

(4)

Main program for the diet problem

287

(1)

Input subroutine

288

(1)

Subroutine for maximization

289

(1)

Output subroutine

290

(1)

C LJ optimization programs

291

(6)

Five food diet problem

291

(2)

Model reduction problem

293

(2)

Geometric problem

295

(2)

D Iterative dynamic programming programs

297

(18)

CSTR with piecewise constant control

297

(6)

IDP program for piecewise linear control

303

(5)

IDP program for variable stage lengths

308

(7)

E Listing of DVERK

315

(6)

DVERK

315

(6)

Index

321

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.