Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-01-13T03:03:35.175Z Has data issue: false hasContentIssue false
  • English
  • Français

Some control problems with random intervention times

Part of: Markov processes Stochastic systems and control

Published online by Cambridge University Press:  01 July 2016

Hui Wang
Hui Wang*
Affiliation:
Brown University
*
∗ Postal address: Division of Applied Mathematics, Brown University, Box F, Providence, RI 02912, USA. Email address: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider the problem of optimally tracking a Brownian motion by a sequence of impulse controls, in such a way as to minimize the total expected cost that consists of a quadratic deviation cost and a proportional control cost. The main feature of our model is that the control can only be exerted at the arrival times of an exogenous uncontrolled Poisson process (signal). In other words, the set of possible intervention times are discrete, random and determined by the signal process (not by the decision maker). We discuss both the discounted problem and the ergodic problem, where explicit solutions can be found. We also derive the asymptotic behavior of the optimal control policies and the value functions as the intensity of the Poisson process goes to infinity, or roughly speaking, as the set of admissible controls goes from the discrete-time impulse control to the continuous-time bounded variation control.

Keywords

impulse controlvariational inequalitygeneralized Ito formula

MSC classification

Primary: 93E20: Optimal stochastic control
Secondary: 60J65: Brownian motion 60J75: Jump processes
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 33 , Issue 2 , June 2001 , pp. 404 - 422
Copyright
Copyright © Applied Probability Trust 2001 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Alvarez, L. and Shepp, L. (1997). Optimal harvesting of stochastically fluctuating populations. J. Math. Biol. 37, 155–177.Google Scholar
[2] Bather, J. A. and Chernoff, H. (1966). Sequential decisions in the control of a spaceship. In Proc. 5th Berkeley Symp. Math. Statist. Prob., Vol. 3, eds Le Cam, L. M. and Neyman, J. University of California Press, Berkeley, pp. 181–207.Google Scholar
[3] Benes, V. E., Shepp, L. A. and Witsenhausen, H. S. (1980). Some solvable stochastic control problems. Stochastics 4, 38–83.CrossRefGoogle Scholar
[4] Bensoussan, A. and Lions, J. L. (1984). Impulse Control and Quasi-Variational Inequalities. Gauthier-Villars, Paris.Google Scholar
[5] Brémaud, P., (1981). Point Processes and Queues: Martingale Dynamics. Springer, New York.Google Scholar
[6] Dixit, A. K. and Pindyck, R. S. (1994). Investment under Uncertainty. Princeton University Press.Google Scholar
[7] Fleming, W. H. (1999). Controlled Markov processes and mathematical finance. In Nonlinear Analysis, Differential Equations and Control, eds Clarke, F. H., Stern, R. J. and Sabidussi, G. Kluwer, Dordrecht, pp. 407–446.CrossRefGoogle Scholar
[8] Fleming, W. H. and Soner, H. M. (1993). Controlled Markov Processes and Viscosity Solutions. Springer, New York.Google Scholar
[9] Hakansson, N. H. (1970). Optimal investment and consumption strategies under risk for a class of utility functions. Econometrica 38, 587–607.Google Scholar
[10] Harrison, J. M. and Taksar, M. I. (1983). Instantaneous control of Brownian motion. Math. Operat. Res. 8, 439–453.Google Scholar
[11] Harrison, J. M., Sellke, T. M. and Taylor, A. J. (1983). Impulse control of Brownian motion. Math. Operat. Res. 8, 454–466.Google Scholar
[12] Karatzas, I. (1983). A class of singular stochastic control problems. Adv. Appl. Prob. 15, 225–254.Google Scholar
[13] Kushner, H. J. and Dupuis, P. G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, New York.Google Scholar
[14] Lungu, E. M. and Øksendal, B. (1997). Optimal harvesting from a population in a stochastic crowded environment. Math. Biosci. 145, 47–75.Google Scholar
[15] Menaldi, J. L. and Robin, M. (1983). On some cheap control problems for diffusion processes. Trans. Amer. Math. Soc. 278, 771–802.Google Scholar
[16] Øksendal, B., (1999). Stochastic control problems where small intervention costs have dramatic effects. Appl. Math. Optimization 40, 355–375.Google Scholar
[17] Protter, Ph. (1990). Stochastic Integration and Differential Equations. Springer, New York.Google Scholar
[18] Rogers, L. C. G. (2001). The relaxed investor and parameter uncertainty. Finance Stoch. 5, 131–154.Google Scholar
[19] Rogers, L. C. G. and Williams, D. (1987). Diffusions, Markov Processes, and Martingales, Vol. 2. John Wiley, Chichester.Google Scholar
[20] Rogers, L. C. G. and Zane, O. (2001). A simple model of liquidity effects. Submitted.Google Scholar
[21] Sherve, S. E., Lehoczky, J. P. and Gavers, D. P. (1984). Optimal consumption for general diffusions with absorbing and reflecting barriers. SIAM. J. Control Optimization 22, 55–75.Google Scholar
[22] Widder, D. V. (1941). The Laplace Transform. Princeton University Press.Google Scholar