Hostname: page-component-586b7cd67f-t7czq Total loading time: 0 Render date: 2024-11-27T23:12:53.787Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal control of a class of piecewise deterministic processes

Published online by Cambridge University Press:  30 July 2013

M. ANNUNZIATO  and
A. BORZÌ
M. ANNUNZIATO
Affiliation:
Dipartimento di Matematica, Università degli Studi di Salerno, Via Giovanni Paolo II, 132 - 84084 Fisciano (SA), Italy email: [email protected]
A. BORZÌ
Affiliation:
Institut für Mathematik, Universität Würzburg, Emil-Fischer-Strasse 30, 97074 Würzburg, Germany email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

A new control strategy for a class of piecewise deterministic processes (PDP) is presented. In this class, PDP stochastic processes consist of ordinary differential equations that are subject to random switches corresponding to a discrete Markov process. The proposed strategy aims at controlling the probability density function (PDF) of the PDP. The optimal control formulation is based on the hyperbolic Fokker–Planck system that governs the time evolution of the PDF of the PDP and on tracking objectives of terminal configuration with a target PDF. The corresponding optimization problems are formulated as a sequence of open-loop hyperbolic optimality systems following a model predictive control framework. These systems are discretized by first-order schemes that guarantee positivity and conservativeness of the numerical PDF solution. The effectiveness of the proposed computational control framework is validated considering PDP with dichotomic noise.

Keywords

Optimal control problems involving partial differential equationsHybrid systemsFokker–Planck equationsStability and convergence of numerical methodsMarkov renewal processes
Type
Papers
Information
European Journal of Applied Mathematics , Volume 25 , Issue 1 , February 2014 , pp. 1 - 25
Copyright
Copyright © Cambridge University Press 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1]Amudevar, A. (2001) A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes. SIAM J. Control Optim. 40, 525–539.CrossRefGoogle Scholar
[2]Annunziato, M. (2002) Non-Gaussian equilibrium distributions arising from the Langevin equation. Phys. Rev. E 65, 21113 (1–6).Google Scholar
[3]Annunziato, M. (2007) A finite difference method for piecewise deterministic processes with memory. Math. Mod. Anal. 12, 157–178.CrossRefGoogle Scholar
[4]Annunziato, M. (2008) Analysis of upwind method for piecewise deterministic Markov processes. Comp. Meth. Appl. Math. 8, 3–20.Google Scholar
[5]Annunziato, M. (2012) On the action of a semi-Markov process on a system of ordinary differential equations. Math. Mod. Anal. 17, 650–672.Google Scholar
[6]Annunziato, M. & Borzì, A. (2010) Optimal control of probability density functions of stochastic processes. Math. Mod. Anal. 15, 393–407.CrossRefGoogle Scholar
[7]Annunziato, M. & Borzì, A. (2013) A Fokker–Planck control framework for multidimensional stochastic processes. J. Comput. Appl. Math. 237, 487–507.Google Scholar
[8]Annunziato, M., Grigolini, P. & West, B. J. (2001) Canonical and noncanonical equilibrium distribution. Phys. Rev. E 66, 011107 (1–13).Google Scholar
[9]Bäurle, N. & Rieder, U. (2009) MDP algorithms for portfolio optimization problems in pure jump markets. Finance Stoch. 13, 591–611.CrossRefGoogle Scholar
[10]Bect, J. (2010) A unifying formulation of the Fokker–Planck–Kolmogorov equation for general stochastic hybrid systems. Nonlinear Anal.: Hybrid Syst. 4, 357–370.Google Scholar
[11]Bertsekas, D. P. (2005) Dynamic Programming and Optimal Control, Athena Scientific, Belmont, MA.Google Scholar
[12]Borzì, A. & Schulz, V. (2012) Computational optimization of systems governed by partial differential equations. SIAM Book Series on Computational Science and Engineering 08, SIAM, Philadelphia, PA.Google Scholar
[13]Cassandras, C. G. & Lygeros, J. (2007) Stochastic Hybrid Systems, CRC Press Taylor & Francis Group, Boca Raton, FL.Google Scholar
[14]Chiquet, J., Limnios, N. & Eid, M. (2009) Piecewise deterministic Markov processes applied to fatigue crack growth modelling. J. Stat. Plan. Inference (Special Issue), 139, 1657–1667.Google Scholar
[15]Choo, K. G., Teo, K. L. & Wu, Z. S. (1981/1982) On an optimal control problem involving first order hyperbolic systems with boundary controls. Numer. Funct. Anal. Optim., 4, 171–190.Google Scholar
[16]Cocozza-Thivent, C., Eymard, R. & Mercier, S. (2006) A finite volume scheme for dynamic reliability models. IMA J. Numer. Anal. 26, 446–471.CrossRefGoogle Scholar
[17]Cocozza-Thivent, C., Eymard, R., Mercier, S. & Roussignol, M. (2006) Characterization of the marginal distributions of Markov processes used in dynamic reliability. J. Appl. Math. Stoch. Anal. article no. 92156, 1–18.Google Scholar
[18]Costa, O. L. V. & Dufour, F. (2003) On the Poisson equation for piecewise-deterministic Markov processes. SIAM J. Control Optim. 42, 985–1001.CrossRefGoogle Scholar
[19]Costa, O. L. V. & Dufour, F. (2010) Average continuous control of piecewise deterministic Markov processes. SIAM J. Control Optim. 48, 4262–4291.CrossRefGoogle Scholar
[20]Cox, D. R. & Miller, H. D. (2001) The Theory of Stochastic Processes, Chapman & Hall CRC, Boca Raton, FL.Google Scholar
[21]Davis, M. H. A. (1984) Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models. J. R. Stat. Soc. B, 46, 353–388.Google Scholar
[22]Dempster, M. A. H. & Ye, J. J. (1995) Impulse control of piecewise deterministic Markov processes. Ann. Appl. Probab. 5, 399–423.Google Scholar
[23]Ebeling, W., Gudowska-Nowak, E. & Sokolov, I. M. (2008) On stochastic dynamics in physics—Remarks on history and terminology. Acta Phys. Pol., 39, 1003–1018.Google Scholar
[24]Evans, L. C. (2002) Partial Differential Equations, Graduate Studies in Mathematics, Vol. 19, American Mathematical Society, Providence, RI.Google Scholar
[25]Eymard, R., Mercier, S. & Prignet, A. (2008) An implicit finite volume scheme for a scalar hyperbolic problem with measure data related to piecewise deterministic Markov processes. J. Comput. Appl. Math. 222, 293–323.CrossRefGoogle Scholar
[26]Faggionato, A., Gabrielli, D. & Ribezzi Crivellari, M. (2009) Non-equilibrium thermodynamics of piecewise deterministic Markov processes J. Stat. Phys. 137, 259–304.CrossRefGoogle Scholar
[27]Filliger, R. & Hongler, M. O. (2004) Supersymmetry in random two-velocity processes. Physica. A 332, 141–150.CrossRefGoogle Scholar
[28]Gilbert, J. C. & Nocedal, J. (1992) Global convergence properties of conjugate gradient methods for optimization. SIAM J. Opt. 2, 21–42.Google Scholar
[29]Grüne, L. & Pannek, J. (2011) Nonlinear Model Predictive Control, Theory and Algorithms, Communications and Control Engineering, Springer Verlag, London/Dordrecht/Heidelberg/New York.Google Scholar
[30]Gustafsson, M. & Holmgren, S. (2010) An implementation framework for solving high-dimensional PDEs on massively parallel computers. In: Kreiss, G.et al. (editors), Numerical Mathematics and Advanced Applications 2009, Springer-Verlag, Berlin–Heidelberg, pp. 417–424.Google Scholar
[31]Hillen, T. & Othmer, H. G. (2000) The diffusion limit of transport equations derived from velocity-jump processes. SIAM J. Appl. Math. 61, 751–775.Google Scholar
[32]Horsthemke, H. (1999) Spatial instabilities in reaction random walks with direction-independent kinetics. Phys. Rev E 60, 2651–2663.Google Scholar
[33]Ito, K. & Kunisch, K. (1990) Receding horizon optimal control for infinite dimensional systems. ESAIM: Control Optim. Calculus Var. 35, 814–824.Google Scholar
[34]Kittel, C. (2004) Elementary Statistical Physics, Dover Publications, New York/London.Google Scholar
[35]Lax, P. D. (2006) Hyperbolic Partial Differential Equations—Courant Lecture Notes in Mathematics, Courant Institute of Mathematical Sciences, American Mathematical Society, Providence, RI.Google Scholar
[36]Lions, J. L., (1971) Optimal Control of Systems Governed by Partial Differential Equations, Springer, Berlin.Google Scholar
[37]Magni, L., Raimondo, D. M. & Allgöwer, F. (2009) Nonlinear Model Predictive Control, Springer, Berlin.Google Scholar
[38]Maurer, H. & Zowe, J. (1979) First- and second-order necessary and sufficient optimality conditions for infinite-dimensional programming problems. Math. Program. 16, 98–110.Google Scholar
[39]Mayne, D. Q. & Michalska, H. (1990) Receding horizon control for nonlinear systems. IEEE Trans. Aut. Control 35, 814–824.Google Scholar
[40]Mil'shtein, G. N. & Repin, Y. M. (1972) Action of a Markov process on a system of differential equations. Differ. Equ. (translated from Russian), 5, 1010–1019.Google Scholar
[41]Moresino, F., Pourtallier, O. & Tidball, M. (1999) Using Viscosity Solution for Approximation in Piecewise Deterministic Control Systems, Unité de recherche INRIA Sophia Antipolis: Rapport de recherche n. 3687.Google Scholar
[42]Morita, A. (1990) Free Brownian motion of a particle driven by a dichotomous random force. Phys. Rev A 41, 754–760.Google Scholar
[43]Morton, K. W. & Mayers, D. F. (2005) Numerical Solution of Partial Differential Equations: An Introduction, Cambridge University Press, Cambridge, UK.Google Scholar
[44]Ou, Y. & Schuster, E. (2010) On the stability of receding horizon control of bilinear parabolic PDE systems. In: Proceedings of the 2010 IEEE Conference on Decision and Control, 15–17 December, 2010, Atlanta, Georgia.Google Scholar
[45]Pawula, R. F. (1977) The probability density and level-crossings of first-order non-linear systems driven by the random telegraph signal. Int. J. Control 25, 283–292.Google Scholar
[46]Pawula, R. F. & Rice, O. (1986) On filtered binary processes. IEEE Trans. Inf. Th. IT-32, 63–72.Google Scholar
[47]Primak, S., Kontorovich, V. & Lyandres, V. (2004) Stochastic Methods and Their Applications to Communications, John Wiley & Sons, Chichester.Google Scholar
[48]Risken, R. (1996) The Fokker–Planck Equation: Methods of Solution and Applications, Springer, Berlin.Google Scholar
[49]Shanno, D. F. (1978) Conjugate gradient methods with inexact searches. Math. Oper. Res. 3, 244–256.Google Scholar
[50]Zubair, H.Bin, , Oosterlee, C. C. & Wienands, R. (2007) Multigrid for high-dimensional elliptic partial differential equations on non-equidistant grid. SIAM J. Sci. Comput. 29, 1613–1636.Google Scholar