Necessary and sufficient conditions are presented for a function involving the divided difference of the psi function to be completely monotonic and for a function involving the ratio of two gamma functions to be logarithmically completely monotonic. From these, some double inequalities are derived for bounding polygamma functions, divided differences of polygamma functions, and the ratio of two gamma functions.