Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-24T03:49:07.311Z Has data issue: false hasContentIssue false
  • English
  • Français

A first-passage problem for a two-dimensional controlled random walk

Published online by Cambridge University Press:  14 July 2016

S. Lalley
S. Lalley*
Affiliation:
Columbia University
*
∗Postal address: Department of Statistics, Columbia University, New York, NY 10027, USA.
Get access Rights & Permissions [Opens in a new window]

Abstract

The process of interest is a controlled random walk in two dimensions: whenever the walker is above the main diagonal, the next increment to his position is chosen from a distribution FA; whenever the walker is below the diagonal, the next increment comes from another distribution FB. The two distributions have mean vectors which tend to push the walker back toward the diagonal. We analyze the problem of first passage to the first quadrant, obtaining explicit representations for the limiting first-entry distribution and expected first-passage time.

Keywords

MARKOV RENEWAL THEORYWIENER–HOPF FACTORIZATION
Type
Research Paper
Information
Journal of Applied Probability , Volume 23 , Issue 3 , September 1986 , pp. 670 - 678
Copyright
Copyright © Applied Probability Trust 1986 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Athreya, K. B., Mcdonald, D. and Ney, P. (1978) Limit theorems for semi-Markov processes and renewal theory for Markov chains. Ann. Prob. 6, 788–797.Google Scholar
Chernoff, H. (1959) Sequential design for experiments. Ann. Math. Statist. 30, 755–770.Google Scholar
Feller, W. (1966) An Introduction to Probability Theory and its Applications, Vol. II. Wiley, New York.Google Scholar
Keener, R. (1984) Second order efficiency in the sequential design of experiments. Ann. Statist. 12, 510–532.CrossRefGoogle Scholar
Kesten, H. (1974) Renewal theory for functionals of Markov chains with general state space. Ann. Prob. 2, 355–386.Google Scholar
Lalley, S. and Lorden, G. (1986) A control problem arising in the sequential design of experiments. Ann. Prob. 14.Google Scholar
Orey, S. (1961) Change of time scale for Markov processes. Trans. Amer. Math. Soc. 99, 384–390.Google Scholar
Revuz, D. (1975) Markov Chains. North-Holland, Amsterdam.Google Scholar
Spitzer, F. (1976) Principles of Random Walk, 2nd edn. Springer-Verlag, Berlin.Google Scholar