Optimal Control Theory

by Sethi, Suresh P.; Thompson, Gerald L.

• ISBN13: 9780792386087
• ISBN10: 0792386086
• Edition: 2nd
• Format: Hardcover
• Copyright: 2000-07-01
• Publisher: Kluwer Academic Pub

Summary

Sethi and Thompson have provided management science and economics communities with a thoroughly revised edition of their classic text on Optimal Control Theory. Central to the book is its extraordinarily wide range of optimal control theory applications. Chapter 5 covers finance; Chapter 6 considers production and inventory problems; Chapter 7 covers marketing problems; Chapter 9 treats machine maintenance and replacement; Chapter 10 deals with problems of optimal consumption of natural resources (renewable or exhaustible); and Chapter 11 discusses a number of applications of control theory to economics. The book has been successfully used as a professional reference tool and as a graduate course book. Its usefulness lies in its emphasis on building applied models of realistic problems faced in a variety of business management situations. The new edition has been completely refined with careful attention to the text and graphic material presentation. In Chapter 3, models have been added that use mixed (control and state) constraints, current value formulations, and terminal conditions. Chapter 4 now covers more advanced material on pure state constraints as they relate to mixed constraints. Each of these chapters contains new results that were not available when the first edition was published. Another important change is the expansion of the material on stochastic optimal control theory, which has become the new Chapter 13. This new chapter provides a brief introduction to stochastic optimal control problems, and it contains formulations of simple stochastic models in production, marketing and finance, and their solutions. Optimal control methods are used to determine optimal ways to control a dynamic system. The theoretical work in this field serves as a foundation for the book, which the authors have applied to business management problems developed from their research and classroom instruction.

Table of Contents

Preface to First Edition	xiii
Preface to Second Edition	xv
What is Optimal Control Theory	1	(22)
Basic Concepts and Definitions	2	(2)
Formulation of Simple Control Models	4	(3)
History of Optimal Control Theory	7	(3)
Notation and Concepts Used	10	(13)
The Maximum Principle: Continuous Time	23	(34)
Statement of the Problem	23	(4)
The Mathematical Model	24	(1)
Constraints	24	(1)
The Objective Function	25	(1)
The Optimal Control Problem	25	(2)
Dynamic Programming and the Maximum Principle	27	(9)
The Hamilton-Jacobi-Bellman Equation	27	(4)
Derivation of the Adjoint Equation	31	(2)
The Maximum Principle	33	(1)
Economic Interpretations of the Maximum Principle	34	(2)
Elementary Examples	36	(8)
Sufficiency Conditions	44	(4)
Solving a TPBVP by Using Spreadsheet Software	48	(9)
The Maximum Principle: Mixed Inequality Constraints	57	(40)
A Maximum Principle for Problems with Mixed Inequality Constraints	58	(6)
Sufficiency Conditions	64	(1)
Current-Value Formulation	65	(4)
Terminal Conditions	69	(11)
Examples Illustrating Terminal Conditions	74	(6)
Infinite Horizon and Stationarity	80	(3)
Model Types	83	(14)
The Maximum Principle: General Inequality Constraints	97	(22)
Pure State Variable Inequality Constraints: Indirect Method	98	(6)
Jump Conditions	103	(1)
A Maximum Principle: Indirect Method	104	(7)
Current-Value Maximum Principle: Indirect Method	111	(2)
Sufficiency Conditions	113	(6)
Applications to Finance	119	(34)
The Simple Cash Balance Problem	120	(9)
The Model	120	(1)
Solution by the Maximum Principle	121	(3)
An Extension Disallowing Overdraft and Short-Selling	124	(5)
Optimal Financing Model	129	(24)
The Model	129	(2)
Application of the Maximum Principle	131	(2)
Synthesis of Optimal Control Paths	133	(11)
Solution for the Infinite Horizon Problem	144	(9)
Applications to Production and Inventory	153	(32)
A Production-Inventory System	154	(10)
The Production-Inventory Model	154	(2)
Solution by the Maximum Principle	156	(3)
The Infinite Horizon Solution	159	(1)
A Complete Analysis of the Constant Positive S Case with Infinite Horizon	160	(2)
Special Cases of Time Varying Demands	162	(2)
Continuous Wheat Trading Model	164	(9)
The Model	165	(1)
Solution by the Maximum Principle	166	(1)
Complete Solution of a Special Case	167	(3)
The Wheat Trading Model with No Short-Selling	170	(3)
Decision Horizons and Forecast Horizons	173	(12)
Horizons for the Wheat Trading Model	174	(1)
Horizons for the Wheat Trading Model with Warehousing Constraint	175	(10)
Applications to Marketing	185	(32)
The Nerlove-Arrow Advertising Model	186	(8)
The Model	186	(2)
Solution by the Maximum Principle	188	(3)
A Nonlinear Extension	191	(3)
The Vidale-Wolfe Advertising Model	194	(23)
Optimal Control Formulation for the Vidale-Wolfe Model	195	(1)
Solution Using Green's Theorem when Q is Large	196	(9)
Solution When Q Is Small	205	(1)
Solution When T Is Infinite	206	(11)
The Maximum Principle: Discrete Time	217	(24)
Nonlinear Programming Problems	217	(11)
Lagrange Multipliers	218	(2)
Inequality Constraints	220	(7)
Theorems from Nonlinear Programming	227	(1)
A Discrete Maximum Principle	228	(6)
A Discrete-Time Optimal Control Problem	228	(1)
A Discrete Maximum Principle	229	(2)
Examples	231	(3)
A General Discrete Maximum Principle	234	(7)
Maintenance and Replacement	241	(26)
A Simple Maintenance and Replacement Model	242	(6)
The Model	242	(1)
Solution by the Maximum Principle	243	(2)
A Numerical Example	245	(2)
An Extension	247	(1)
Maintenance and Replacement for a Machine Subject to Failure	248	(6)
The Model	249	(2)
Optimal Policy	251	(2)
Determination of the Sale Date	253	(1)
Chain of Machines	254	(13)
The Model	254	(2)
Solution by the Discrete Maximum Principle	256	(1)
Special Case of Bang-Bang Control	257	(1)
Incorporation into the Wagner-Whitin Framework for a Complete Solution	258	(1)
A Numerical Example	259	(8)
Applications to Natural Resources	267	(22)
The Sole Owner Fishery Resource Model	268	(5)
The Dynamics of Fishery Models	268	(1)
The Sole Owner Model	269	(1)
Solution by Green's Theorem	270	(3)
An Optimal Forest Thinning Model	273	(6)
The Forestry Model	273	(1)
Determination of Optimal Thinning	274	(2)
A Chain of Forests Model	276	(3)
An Exhaustible Resource Model	279	(10)
Formulation of the Model	279	(3)
Solution by the Maximum Principle	282	(7)
Economic Applications	289	(18)
Models of Optimal Economic Growth	289	(6)
An Optimal Capital Accumulation Model	290	(1)
Solution by the Maximum Principle	290	(1)
A One-Sector Model with a Growing Labor Force	291	(1)
Solution by the Maximum Principle	292	(3)
A Model of Optimal Epidemic Control	295	(4)
Formulation of the Model	295	(1)
Solution by Green's Theorem	296	(3)
A Pollution Control Model	299	(4)
Model Formulation	299	(1)
Solution by the Maximum Principle	300	(1)
Phase Diagram Analysis	301	(2)
Miscellaneous Applications	303	(4)
Differential Games, Distributed Systems, and Impulse Control	307	(32)
Differential Games	308	(7)
Two Person Zero-Sum Differential Games	308	(2)
Nonzero-Sum Differential Games	310	(2)
An Application to the Common-Property Fishery Resources	312	(3)
Distributed Parameter Systems	315	(7)
The Distributed Parameter Maximum Principle	317	(1)
The Cattle Ranching Problem	318	(4)
Interpretation of the Adjoint Function	322	(1)
Impulse Control	322	(17)
The Oil Driller's Problem	324	(1)
The Maximum Principle for Impulse Optimal Control	325	(2)
Solution of the Oil Driller's Problem	327	(4)
Machine Maintenance and Replacement	331	(1)
Application of the Impulse Maximum Principle	332	(7)
Stochastic Optimal Control	339	(24)
The Kalman Filter	340	(5)
Stochastic Optimal Control	345	(2)
A Stochastic Production Planning Model	347	(5)
Solution for the Production Planning Problem	350	(2)
A Stochastic Advertising Problem	352	(3)
An Optimal Consumption-Investment Problem	355	(5)
Concluding Remarks	360	(3)
A Solutions of Linear Differential Equations	363	(16)
A.1 Linear Differential Equations with Constant Coefficients	363	(1)
A.2 Homogeneous Equations of Order One	364	(1)
A.3 Homogeneous Equations of Order Two	364	(1)
A.4 Homogeneous Equations of Order n	365	(1)
A.5 Particular Solutions of Linear D.E. with Constant Coefficients	366	(2)
A.6 Integrating Factor	368	(1)
A.7 Reduction of Higher-Order Linear Equations to Systems of First-Order Linear Equations	369	(3)
A.8 Solution of Linear Two-Point Boundary Value Problems	372	(1)
A.9 Homogeneous Partial Differential Equations	372	(2)
A.10 Inhomogeneous Partial Differential Equations	374	(1)
A.11 Solutions of Finite Difference Equations	375	(4)
A.11.1 Changing Polynomials in Powers of k into Factorial Powers of k	376	(1)
A.11.2 Changing Factorial Powers of k into Ordinary Powers of k	377	(2)
B Calculus of Variations and Optimal Control Theory	379	(14)
B.1 The Simplest Variational Problem	379	(1)
B.2 The Euler Equation	380	(3)
B.3 The Shortest Distance Between Two Points on the Plane	383	(1)
B.4 The Brachistochrone Problem	384	(2)
B.5 The Weierstrass-Erdmann Corner Conditions	386	(2)
B.6 Legendre's Conditions: The Second Variation	388	(1)
B.7 Necessary Condition for a Strong Maximum	389	(1)
B.8 Relation to the Optimal Control Theory	390	(3)
C An Alternative Derivation of the Maximum Principle	393	(8)
C.1 Needle-Shaped Variation	394	(2)
C.2 Derivation of the Adjoint Equation and the Maximum Principle	396	(5)
D Special Topics in Optimal Control	401	(8)
D.1 Linear-Quadratic Problems	401	(4)
D.1.1 Certainty Equivalence or Separation Principle	403	(2)
D.2 Second Order Variations	405	(2)
D.3 Singular Control	407	(2)
E Answers to Selected Exercises	409	(8)
Bibliography	417	(66)
Index	483	(18)
List of Figures	501	(4)
List of Tables	505