In this paper it is shown that for a large class of replacement problems the class of stationary replacement strategies is complete, i.e. in order to minimize the average long run cost per unit time it suffices to consider replacement rules which are equal for each new unit irrespectively of what has been observed from earlier units. The main result is based on a version of the law of large numbers for martingale differences proved in the appendix.