Several notions of stochastic convexity and concavity and their properties are studied in this paper. Efficient sample path approaches are developed in order to verify the occurrence of these notions in various applications. Numerous examples are given. The use of these notions in several areas of probability and statistics is demonstrated. In queueing theory, the convexity (as a function of c) of the steady-state waiting time in a GI/D/c queue, and (as a function of the arrival or service rates) in a GI/G/1 queue, is established. Also the convexity of the queue length in the M/M/c case (as a function of the arrival rate) is shown, thus strengthening previous results while simplifying their derivation. In reliability theory, the convexity of the payoff in the success rate of an imperfect repair is obtained and used to find an optimal repair probability. Also the convexity of the damage as a function of time in a cumulative damage shock model is shown. In branching processes, the convexity of the population size as a function of a parameter of the offspring distribution is proved. In non-parametric statistics, the stochastic concavity (or convexity) of the empirical distribution function is established. And, for applications in the theory of probability inequalities, we identify several families of distributions which are convexly parametrized.