[1] Agmon, S., Douglis, A., and Nirenberg, L. 1959. Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions. I. Comm. Pure Appl. Math., 12, 623–727.
[2] Allinger, D. F. and Mitter, S. K. 1980. New results on the innovations problem for nonlinear filtering. Stochastics, 4(4), 339–348.
[3] Anderson, E. J. and Nash, P. 1987. Linear Programming in Infinite-Dimensional Spaces. Wiley-Interscience Series in Discrete Mathematics and Optimization. Chichester: John Wiley & Sons.
[4] Arapostathis, A. and Borkar, V. S. 2010. Uniform recurrence properties of controlled diffusions and applications to optimal control. SIAM J. Control Optim., 48(7), 152–160.
[5] Arapostathis, A. and Ghosh, M. K. 2004 (Dec). Ergodic control of jump diffusions in ℝd under a near-monotone cost assumption. Pages 4140–4145 of:43rd IEEE Conference on Decision and Control, vol. 4.
[6] Arapostathis, A., Borkar, V. S., Fernández-Gaucherand, E., Ghosh, M. K., and Marcus, S. I. 1993. Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim., 31(2), 282–344.
[7] Arapostathis, A., Ghosh, M. K., and Marcus, S. I. 1999. Harnack's inequality for cooperative, weakly coupled elliptic systems. Comm. Partial Differential Equations, 24, 1555–1571.
[8] Arisawa, M., and Lions, P.-L. 1998. On ergodic stochastic control. Comm. Partial Differential Equations, 23(11–12), 333–358.
[9] Arrow, K. J., Barankin, E. W., and Blackwell, D. 1953. Admissible points of convex sets. Pages 87–91 of: Contributions to the Theory of Games, vol. 2. Annals of Mathematics Studies, no. 28. Princeton, NJ: Princeton University Press.
[10] Bachelier, L. 2006. Louis Bachelier's Theory of Speculation: The Origins of Modern Finance. Princeton, NJ: Princeton University Press. Translated and with a commentary by Mark Davis and Alison Etheridge.
[11] Basak, G. K., Borkar, V. S., and Ghosh, M. K. 1997. Ergodic control of degenerate diffusions. Stochastic Anal. Appl., 15(1), 1–17.
[12] Bass, R. F. 1998. Diffusions and Elliptic Operators. Probability and its Applications. New York: Springer-Verlag.
[13] Beneš, V. E. 1970. Existence of optimal strategies based on specified information, for a class of stochastic decision problems. SIAM J. Control, 8, 179–188.
[14] Bensoussan, A. 1982. Stochastic Control by Functional Analysis Methods. Studies in Mathematics and its Applications, vol. 11. Amsterdam: North-Holland Publishing Co.
[15] Bensoussan, A. and Borkar, V. 1984. Ergodic control problem for one-dimensional diffusions with near-monotone cost. Systems Control Lett., 5(2), 127–133.
[16] Bensoussan, A. and Borkar, V. 1986. Corrections to: “Ergodic control problem for onedimensional diffusions with near-monotone cost” [Systems Control Lett. 5(1984), no. 2, 127–133]. Systems Control Lett., 7(3), 233–235.
[17] Bensoussan, A. and Frehse, J. 1992. On Bellman equations of ergodic control in Rn. J. Reine Angew. Math., 429, 125–160.
[18] Bensoussan, A. and Frehse, J. 2002. Ergodic control Bellman equation with Neumann boundary conditions. Pages 59–71 of: Stochastic Theory and Control (Lawrence, KS, 2001). Lecture Notes in Control and Inform. Sci., vol. 280. Berlin: Springer.
[19] Bertoin, J. 1996. Lévy Processes. Cambridge Tracts in Mathematics, vol. 121. Cambridge: Cambridge University Press.
[20] Bertsekas, D. P. and Shreve, S. E. 1978. Stochastic Optimal Control: The Discrete Time Case. New York: Academic Press.
[21] Bhatt, A. G. and Borkar, V. S. 1996. Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab., 24(3), 1531–1562.
[22] Bhatt, A. G. and Borkar, V. S. 2005. Existence of optimal Markov solutions for ergodic control of Markov processes. Sankhyā, 67(1), 1–18.
[23] Bhatt, A. G. and Karandikar, R. L. 1993. Invariant measures and evolution equations for Markov processes characterized via martingale problems. Ann. Probab., 21(4), 2246–2268.
[24] Billingsley, P. 1968. Convergence of Probability Measures. New York: John Wiley & Sons.
[25] Billingsley, P. 1995. Probability and Measure. Third edition. Wiley Series in Probability and Mathematical Statistics. New York: John Wiley & Sons.
[26] Bogachev, V. I., Krylov, N. V., and Röckner, M. 2001. On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions. Comm. Partial Differential Equations, 26(11–12), 2037–2080.
[27] Bogachev, V. I., Rökner, M., and Stannat, V. 2002. Uniqueness of solutions of elliptic equations and uniqueness of invariant measures of diffusions. Mat. Sb., 193(7), 3–36.
[28] Borkar, V. S. 1989a. Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series, vol. 203. Harlow: Longman Scientific & Technical.
[29] Borkar, V. S. 1989b. A topology for Markov controls. Appl. Math. Optim., 20(1), 55–62.
[30] Borkar, V. S. 1991. On extremal solutions to stochastic control problems. Appl. Math. Optim., 24(3), 317–330.
[31] Borkar, V. S. 1993. Controlled diffusions with constraints. II. J. Math. Anal. Appl., 176(2), 310–321.
[32] Borkar, V. S. 1995. Probability Theory: An Advanced Course. New York: Springer-Verlag.
[33] Borkar, V. S. 2003. Dynamic programming for ergodic control with partial observations. Stochastic Process. Appl., 103(2), 293–310.
[34] Borkar, V. S. and Budhiraja, A. 2004a. Ergodic control for constrained diffusions: characterization using HJB equations. SIAM J. Control Optim., 43(4), 1467–1492.
[35] Borkar, V. S. and Budhiraja, A. 2004b. A further remark on dynamic programming for partially observed Markov processes. Stochastic Process. Appl., 112(1), 79–93.
[36] Borkar, V. S. and Gaitsgory, V. 2007. Singular perturbations in ergodic control of diffusions. SIAM J. Control Optim., 46(5), 1562–1577.
[37] Borkar, V. S. and Ghosh, M. K. 1988. Ergodic control of multidimensional diffusions. I. The existence results. SIAM J. Control Optim., 26(1), 112–126.
[38] Borkar, V. S. and Ghosh, M. K. 1990a. Controlled diffusions with constraints. J. Math. Anal. Appl., 152(1), 88–108.
[39] Borkar, V. S. and Ghosh, M. K. 1990b. Ergodic control of multidimensional diffusions. II. Adaptive control. Appl. Math. Optim., 21(2), 191–220.
[40] Borkar, V. S. and Ghosh, M. K. 2003. Ergodic control of partially degenerate diffusions in a compact domain. Stochastics, 75(4), 221–231.
[41] Borkar, V. S. and Mitter, S. K. 2003. A note on stochastic dissipativeness. Pages 41– 49 of: Directions in mathematical systems theory and optimization. Lecture Notes in Control and Inform. Sci., vol. 286. Berlin: Springer.
[42] Busca, J. and Sirakov, B. 2004. Harnack type estimates for nonlinear elliptic systems and applications. Ann. Inst. H. Poincaré Anal. Non Linéaire, 21(5), 543–590.
[43] Chen, Y.-Z. and Wu, L.-C. 1998. Second Order Elliptic Equations and Elliptic Systems. Translations of Mathematical Monographs, vol. 174. Providence, RI: American Mathematical Society. Translated from the 1991 Chinese original by Bei Hu.
[44] Choquet, G. 1969. Lectures on Analysis. Vol. II: Representation Theory. Edited by Marsden, J., Lance, T. and Gelbart, S.. New York: W. A. Benjamin.
[45] Chung, K. L. 1982. Lectures from Markov Processes to BrownianMotion. Grundlehren der Mathematischen Wissenschaften, vol. 249. New York: Springer-Verlag.
[46] Crandall, M. G., Kocan, M., and Święch, A. 2000. Lp-theory for fully nonlinear uniformly parabolic equations. Comm. Partial Differential Equations, 25(11–12), 1997–2053.
[47] Dellacherie, C. and Meyer, P. 1978. Probabilities and Potential A. North-Holland Mathematics Studies, vol. 29. Amsterdam: North-Holland.
[48] Dubins, L. 1962. On extreme points of convex sets. J. Math. Anal. Appl., 5, 237–244.
[49] Dubins, L. and Freedman, D. 1964. Measurable sets of measures. Pacific J. Math., 14, 1211–1222.
[50] Dudley, R. M. 2002. Real Analysis and Probability. Cambridge Studies in Advanced Mathematics, vol. 74. Cambridge: Cambridge University Press.
[51] Dunford, N. and Schwartz, J. T. 1988. Linear Operators. Part I. Wiley Classics Library. New York: John Wiley & Sons.
[52] Dupuis, P. and Ishii, H. 1991. On Lipschitz continuity of the solution mapping to the Skorohod problem with applications. Stochastics, 35, 31–62.
[53] Dynkin, E. B. and Yushkevich, A. A. 1979. Controlled Markov Processes. Grundlehren der Mathematischen Wissenschaften, vol. 235. Berlin: Springer-Verlag.
[54] El Karoui, N., Nguyen, D. H., and Jeanblanc-Picqué, M. 1987. Compactification methods in the control of degenerate diffusions: existence of an optimal control. Stochastics, 20(3), 169–219.
[55] Ethier, S. N. and Kurtz, T. G. 1986. Markov Processes. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: John Wiley & Sons.
[56] Fabes, E. B. and Kenig, C. E. 1981. Examples of singular parabolic measures and singular transition probability densities. Duke Math. J., 48(4), 845–856.
[57] Feller, W. 1959. Non-Markovian processes with the semigroup property. Ann. Math. Statist., 30, 1252–1253.
[58] Fleming, W. H. and Rishel, R. W. 1975. Deterministic and Stochastic Optimal Control. Berlin: Springer-Verlag.
[59] Fleming, W. H. 1980. Measure-valued processes in the control of partially-observable stochastic systems. Appl. Math. Optim., 6(3), 271–285.
[60] Fleming, W. H., and Pardoux, E. 1982. Optimal control for partially observed diffusions. SIAM J. Control Optim., 20(2), 261–285.
[61] Freidlin, M. I. 1963. Diffusion processes with reflection and a directional derivative problem on a manifold with boundary. Theory Probab. Appl., 8(1), 75–83.
[62] Friedman, A. 1964. Partial Differential Equations of Parabolic Type. Englewood Cliffs, NJ: Prentice-Hall.
[63] Friedman, A. 2006. Stochastic Differential Equations and Applications. Mineola, NY: Dover Publications.
[64] Ghosh, M. K., Arapostathis, A., and Marcus, S. I. 1993. Optimal control of switching diffusions with application to flexible manufacturing systems. SIAM J. Control Optim., 31(5), 1183–1204.
[65] Ghosh, M. K., Arapostathis, A., and Marcus, S. I. 1997. Ergodic control of switching diffusions. SIAM J. Control Optim., 35(6), 1952–1988.
[66] Gikhman, I. I. and Skorokhod, A. V. 1969. Introduction to the Theory of Random Processes. Translated from the Russian by Scripta Technica, Inc. Philadelphia, PA: W. B. Saunders.
[67] Gilbarg, D. and Trudinger, N. S. 1983. Elliptic Partial Differential Equations of Second Order. Second edition. Grundlehren der Mathematischen Wissenschaften, vol. 224. Berlin: Springer-Verlag.
[68] Gyöngy, I. and Krylov, N. 1996. Existence of strong solutions for Itô's stochastic equations via approximations. Probab. Theory Related Fields, 105(2), 143–158.
[69] Has′minskiĭ, R. Z. 1960. Ergodic properties of recurrent diffusion processes and stabilization of the solution of the Cauchy problem for parabolic equations. Theory Probab. Appl., 5(2), 179–196.
[70] Has'minskiĭ, R. Z. 1980. Stochastic Stability of Differential Equations. The Netherlands: Sijthoff & Noordhoff.
[71] Haussmann, U. G. 1985. L'équation de Zakai et le problème séparè du contrôle optimal stochastique. Pages 37–62 of: Séminaire de Probabilités, XIX, 1983/84. Lecture Notes in Math., vol. 1123. Berlin: Springer.
[72] Haussmann, U. G. 1986. Existence of optimal Markovian controls for degenerate diffusions. Pages 171–186 of: Stochastic Differential Systems (Bad Honnef, 1985). Lecture Notes in Control and Information Science, vol. 78. Berlin: Springer.
[73] Hernández-Lerma, O. 1989. Adaptive Markov Control Processes. Applied Mathematical Sciences, vol. 79. New York: Springer-Verlag.
[74] Ikeda, N. and Watanabe, S. 1989. Stochastic Differential Equations and Diffusion Processes. Second edition. North-Holland Mathematical Library, vol. 24. Amsterdam: North-Holland Publishing.
[75] Kallenberg, L. C. M. 1983. Linear Programming and Finite Markovian Control Problems. Mathematical Centre Tracts, vol. 148. Amsterdam: Mathematisch Centrum.
[76] Karatzas, I. and Shreve, S. E. 1991. Brownian Motion and Stochastic Calculus. Second edition. Graduate Texts in Mathematics, vol. 113. New York: Springer-Verlag.
[77] Kogan, Ya. A. 1969. The optimal control of a non-stopping diffusion process with reflection. Theory Probab. Appl., 14(3), 496–502.
[78] Krylov, N. V. 1980. Controlled Diffusion Processes. Applications of Mathematics, vol. 14. New York: Springer-Verlag. Translated from the Russian by A. B. Aries.
[79] Krylov, N. V. 1995. Introduction to the Theory of Diffusion Processes. Translations of Mathematical Monographs, vol. 142. Providence, RI: American Mathematical Society.
[80] Kunita, H. 1981. Some extensions of Itô's formula. Pages 118–141 of: Seminar on Probability, XV (Univ. Strasbourg, Strasbourg, 1979/1980) (French). Lecture Notes in Math., vol. 850. Berlin: Springer.
[81] Kurtz, T. G. and Stockbridge, R. H. 1998. Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim., 36(2), 609–653.
[82] Kurtz, T. G. and Stockbridge, R. H. 2001. Stationary solutions and forward equations for controlled and singular martingale problems. Electron. J. Probab., 6, no. 17, 52 pp. (electronic).
[83] Ladyženskaja, O. A., Solonnikov, V. A., and Ural′ceva, N. N. 1967. Linear and Quasilinear Equations of Parabolic Type. Translated from the Russian by Smith, S.. Translations of Mathematical Monographs, Vol. 23. Providence, RI: American Mathematical Society.
[84] Lions, P.-L. 1983a. Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. I. The dynamic programming principle and applications. Comm. Partial Differential Equations, 8(10), 1101–1174.
[85] Lions, P.-L. 1983b. Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. II. Viscosity solutions and uniqueness. Comm. Partial Differential Equations, 8(11), 1229–1276.
[86] Lions, P. L. and Sznitman, A. S. 1984. Stochastic differential equations with reflecting boundary conditions. Comm. Pure Appl. Math., 37, 511–537.
[87] Liptser, R. S. and Shiryayev, A. N. 1977. Statistics of Random Processes. I. Applications of Mathematics, Vol. 5. New York: Springer-Verlag. Translated by A. B. Aries.
[88] Luenberger, D. G. 1967. Optimization by Vector Space Methods. New York: John Wiley & Sons.
[89] Menaldi, J.-L. and Robin, M. 1997. Ergodic control of reflected diffusions with jumps. Appl. Math. Optim., 35(2), 117–137.
[90] Menaldi, J.-L. and Robin, M. 1999. On optimal ergodic control of diffusions with jumps. Pages 439–456 of: Stochastic Analysis, Control, Optimization and Applications. Systems Control Found. Appl. Boston, MA: Birkhäuser Boston.
[91] Meyn, S. and Tweedie, R. L. 2009. Markov Chains and Stochastic Stability. Second edition. Cambridge: Cambridge University Press.
[92] Nadirashvili, N. 1997. Nonuniqueness in the martingale problem and the Dirichlet problem for uniformly elliptic operators. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 24(3), 537–549.
[93] Neveu, J. 1965. Mathematical Foundations of the Calculus of Probability. San Francisco, CA: Holden-Day.
[94] Parthasarathy, K. R. 1967. Probability Measures on Metric Spaces. Probability and Mathematical Statistics, No. 3. New York: Academic Press.
[95] Phelps, R. 1966. Lectures on Choquet's Theorem. New York: Van Nostrand.
[96] Portenko, N. I. 1990. Generalized Diffusion Processes. Translations of Mathematical Monographs, vol. 83. Providence, RI: American Mathematical Society. Translated from the Russian by H. H. McFaden.
[97] Puterman, M. I. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken, NJ: John Wiley & Sons.
[98] Rachev, S. T. 1991. Probability Metrics and the Stability of Stochastic Models. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Chichester: John Wiley & Sons.
[99] Rishel, R. 1970. Necessary and sufficient dynamic programming conditions for continuous time stochastic control problem. SIAM J. Control, 8, 559–571.
[100] Robin, M. 1983. Long-term average cost control problems for continuous time Markov processes: a survey. Acta Appl. Math., 1(3), 281–299.
[101] Rockafellar, R. T. 1946. Convex Analysis. Princeton Mathematical Series, vol. 28. Princeton, NJ: Princeton University Press.
[102] Rogers, L. C. G., and Williams, D. 2000a. Diffusions, Markov Processes, and Martingales. Vol. 1. Cambridge Mathematical Library. Cambridge: Cambridge University Press.
[103] Rogers, L. C. G. and Williams, D. 2000b. Diffusions, Markov Processes, and Martingales. Vol. 2. Cambridge Mathematical Library. Cambridge: Cambridge University Press.
[104] Ross, S. M. 1970. Average cost semi-Markov decision processes. J. Appl. Probability, 7, 649–656.
[105] Rudin, W. 1973. Functional Analysis. New York: McGraw-Hill.
[106] Rugh, W. J. 1996. Linear System Theory. Second edition. Prentice Hall Information and System Sciences Series. Englewood Cliffs, NJ: Prentice Hall.
[107] Safonov, M. V. 1999. Nonuniqueness for second-order elliptic equations with measurable coefficients. SIAM J. Math. Anal., 30(4), 879–895 (electronic).
[108] Skorohod, A. V. 1989. Asymptotic Methods in the Theory of Stochastic Differential Equations. Translations of Mathematical Monographs, vol. 78. Providence, RI: American Mathematical Society.
[109] Stannat, W. 1999. (Nonsymmetric) Dirichlet operators on L1: existence, uniqueness and associated Markov processes. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 28(1), 99–140.
[110] Stockbridge, R. H. 1989. Time-average control of martingale problems: the Hamilton–Jacobi–Bellman equation. Stochastics Stochastics Rep., 27(4), 249–260.
[111] Stockbridge, R. H. 1990a. Time-average control of martingale problems: a linear programming formulation. Ann. Probab., 18(1), 206–217.
[112] Stockbridge, R. H. 1990b. Time-average control of martingale problems: existence of a stationary solution. Ann. Probab., 18(1), 190–205.
[113] Striebel, C. 1984. Martingale methods for the optimal control of continuous time stochastic systems. Stoch. Process. Appl., 18, 324–347.
[114] Stroock, D. W. and Varadhan, S. R. S. 1971. Diffusion processes with boundary conditions. Comm. Pure Appl. Math., 24, 147–225.
[115] Stroock, D. W. and Varadhan, S. R. S. 1979. Multidimensional Diffusion Processes. Grundlehren der Mathematischen Wissenschaften, vol. 233. Berlin: Springer-Verlag.
[116] Veretennikov, A. Yu. 1980. Strong solutions and explicit formulas for solutions of stochastic integral equations. Mat. Sb. (N.S.), 111(153)(3), 434–452, 480.
[117] Veretennikov, A. Yu. 1987. On strong solutions of stochastic Itô equations with jumps. Theory Probab. Appl., 32(1), 148–152.
[118] Wagner, D. H. 1977. Survey of measurable selection theorems. SIAM J. Control Optim., 15, 859–903.
[119] Walters, P. 1982. An Introduction to Ergodic Theory. Graduate Texts in Mathematics, vol. 79. New York: Springer-Verlag.
[120] Willems, J. C. 1972. Dissipative dynamical systems. I. General theory. Arch. Rational Mech. Anal., 45, 321–351.
[121] Wong, E. 1971. Representation of martingales, quadratic variation and applications. SIAM J. Control, 9, 621–633.
[122] Wong, E. and Hajek, B. 1985. Stochastic Processes in Engineering Systems. Springer Texts in Electrical Engineering. New York: Springer-Verlag.
[123] Wu, W., Arapostathis, A., and Shakkottai, S. 2006. Optimal power allocation for a timevarying wireless channel under heavy-traffic approximation. IEEE Trans. Automat. Control, 51(4), 580–594.
[124] Xiong, J. 2008. An Introduction to Stochastic Filtering Theory. Oxford Graduate Texts in Mathematics, vol. 18. Oxford: Oxford University Press.
[125] Yosida, K. 1980. Functional Analysis. Sixth edition. Grundlehren der Mathematischen Wissenschaften, vol. 123. Berlin: Springer-Verlag.
[126] Young, L. C. 1969. Lectures on the Calculus of Variations and Optimal Control Theory. Foreword by Wendell Fleming, H.. Philadelphia: W. B. Saunders.
[127] Zvonkin, A. K. 1974. A transformation of the phase space of a diffusion process that will remove the drift. Mat. Sb. (N.S.), 93(135), 129–149, 152.
