Hostname: page-component-745bb68f8f-kw2vx Total loading time: 0 Render date: 2025-01-23T19:43:46.389Z Has data issue: false hasContentIssue false
  • English
  • Français

The optimal value of markov stopping problems with one-step look ahead policy

Published online by Cambridge University Press:  14 July 2016

Masami Yasuda
Masami Yasuda*
Affiliation:
Chiba University
*
∗ Postal address: College of General Education, Chiba University, Chiba, 260 Japan.
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper treats stopping problems on Markov chains in which the OLA (one-step look ahead) policy is optimal. Its associated optimal value can be explicitly expressed by a potential for a charge function of the difference between the immediate reward and the one-step-after reward. As an application to the best choice problem, we shall obtain the value of three problems: the classical secretary problem, a problem with a refusal probability and a problem with a random number of objects.

Keywords

OPTIMAL STOPPING PROBLEMMARKOV POTENTIAL THEORY
Type
Research Papers
Information
Journal of Applied Probability , Volume 25 , Issue 3 , September 1988 , pp. 544 - 552
Copyright
Copyright © Applied Probability Trust 1988 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Bojdecki, T. (1978) On optimal stopping of a sequence of independent random variables—Probability maximizing approach. Stoch. Proc. Appl. 6, 153–163.Google Scholar
[2] Chow, Y. S., Robbins, H. and Siegmund, D. (1971) Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, Boston.Google Scholar
[3] Cowan, R. and Zabczyk, J. (1978) An optimal selection problem associated with the Poisson process. Theory Prob. Appl. 23, 584–592.Google Scholar
[4] Darling, D. A. (1985) Contribution to the optimal stopping problem, Z. Wahrscheinlichkeitsth. 70, 525–533.Google Scholar
[5] Dubins, L. E. and Savage, L. J. (1965) How to Gamble if You Must: Inequalities for Stochastic Processes. McGraw-Hill, New York.Google Scholar
[7] Dynkin, E. B. and Yushkevitch, A. A. (1969) Theorems and Problems on Markov Processes. Plenum Press, New York.Google Scholar
[8] Freeman, P. R. (1983) The secretary problem and its extensions: a review. Int. Statist. Rev. 51, 189–206.Google Scholar
[9] Gianini, J. P. and Samuels, S. M. (1976) The infinite secretary problem. Ann. Prob. 13, 418–432.Google Scholar
[10] Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematisch Centrum, Amsterdam.Google Scholar
[11] Kemeny, J. G., Snell, J. L. and Knapp, A. W. (1976) Denumerable Markov Chains, 2nd edn. Springer-Verlag, Berlin.Google Scholar
[12] Mucci, A. G. (1973) Differential equations and optimal choice problem. Ann. Statist. 1, 104–113.Google Scholar
[13] Presman, E. L. and Sonin, I. M. (1972) The best choice problem for a random number of objects. Theory Prob. Appl. 17, 657–668.Google Scholar
[14] Rasmussen, W. T. and Robbins, H. (1975) The candidate problem with unknown population size. J. Appl. Prob. 12, 692–701.Google Scholar
[15] Ross, S. M. (1970) Applied Probability Models with Optimization Applications. Holden Day, San Francisco.Google Scholar
[16] Smith, M. H. (1975) A secretary problem with uncertain employment. J. Appl. Prob. 12, 620–624.Google Scholar
[17] Yasuda, M. (1984) Asymptotic results for the best-choice problem with a random number of objects. J. Appl. Prob. 21, 521–536.Google Scholar