Book contents
- Frontmatter
- Contents
- Foreword by Eugene Silberberg
- Preface
- 1 Essential Elements of Continuous Time Dynamic Optimization
- 2 Necessary Conditions for a Simplified Control Problem
- 3 Concavity and Sufficiency in Optimal Control Problems
- 4 The Maximum Principle and Economic Interpretations
- 5 Linear Optimal Control Problems
- 6 Necessary and Sufficient Conditions for a General Class of Control Problems
- 7 Necessary and Sufficient Conditions for Isoperimetric Problems
- 8 Economic Characterization of Reciprocal Isoperimetric Problems
- 9 The Dynamic Envelope Theorem and Economic Interpretations
- 10 The Dynamic Envelope Theorem and Transversality Conditions
- 11 Comparative Dynamics via Envelope Methods
- 12 Discounting, Current Values, and Time Consistency
- 13 Local Stability and Phase Portraits of Autonomous Differential Equations
- 14 Necessary and Sufficient Conditions for Infinite Horizon Control Problems
- 15 The Neoclassical Optimal Economic Growth Model
- 16 A Dynamic Limit Pricing Model of the Firm
- 17 The Adjustment Cost Model of the Firm
- 18 Qualitative Properties of Infinite Horizon Optimal Control Problems with One State Variable and One Control Variable
- 19 Dynamic Programming and the Hamilton-Jacobi-Bellman Equation
- 20 Intertemporal Duality in the Adjustment Cost Model of the Firm
- Index
- References
19 - Dynamic Programming and the Hamilton-Jacobi-Bellman Equation
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- Foreword by Eugene Silberberg
- Preface
- 1 Essential Elements of Continuous Time Dynamic Optimization
- 2 Necessary Conditions for a Simplified Control Problem
- 3 Concavity and Sufficiency in Optimal Control Problems
- 4 The Maximum Principle and Economic Interpretations
- 5 Linear Optimal Control Problems
- 6 Necessary and Sufficient Conditions for a General Class of Control Problems
- 7 Necessary and Sufficient Conditions for Isoperimetric Problems
- 8 Economic Characterization of Reciprocal Isoperimetric Problems
- 9 The Dynamic Envelope Theorem and Economic Interpretations
- 10 The Dynamic Envelope Theorem and Transversality Conditions
- 11 Comparative Dynamics via Envelope Methods
- 12 Discounting, Current Values, and Time Consistency
- 13 Local Stability and Phase Portraits of Autonomous Differential Equations
- 14 Necessary and Sufficient Conditions for Infinite Horizon Control Problems
- 15 The Neoclassical Optimal Economic Growth Model
- 16 A Dynamic Limit Pricing Model of the Firm
- 17 The Adjustment Cost Model of the Firm
- 18 Qualitative Properties of Infinite Horizon Optimal Control Problems with One State Variable and One Control Variable
- 19 Dynamic Programming and the Hamilton-Jacobi-Bellman Equation
- 20 Intertemporal Duality in the Adjustment Cost Model of the Firm
- Index
- References
Summary
In this chapter, we turn our attention away from the derivation of necessary and sufficient conditions that can be used to find the optimal time paths of the state, costate, and control variables, and focus on the optimal value function more closely. In particular, we will derive the fundamental first-order partial differential equation obeyed by the optimal value function, known as the Hamilton-Jacobi-Bellman equation. This shift in our attention, moreover, will lead us to a different form for the optimal value of the control vector, namely, the feedback or closed-loop form of the control. This form of the optimal control typically gives the optimal value of the control vector as a function of the current date, the current state, and the parameters of the control problem. In contrast, the form of the optimal control vector derived via the necessary conditions of optimal control theory is termed open-loop, and in general gives the optimal value of the control vector as a function of the independent variable time, the parameters, and the initial and/or terminal values of the planning horizon and the state vector. Essentially, the feedback form of the optimal control is a decision rule, for it gives the optimal value of the control for any current period and any admissible state in the current period that may arise. In contrast, the open-loop form of the optimal control is a curve, for it gives the optimal values of the control as the independent variable time varies over the planning horizon.
- Type
- Chapter
- Information
- Foundations of Dynamic Economic AnalysisOptimal Control Theory and Applications, pp. 511 - 536Publisher: Cambridge University PressPrint publication year: 2005
References
- 1
- Cited by