For smooth random dynamical systems we consider the quenched linear and higher-order response of equivariant physical measures to perturbations of the random dynamics. We show that the spectral perturbation theory of Gouëzel, Keller and Liverani [28, 33], which has been applied to deterministic systems with great success, may be adapted to study random systems that possess good mixing properties. As a consequence, we obtain general linear and higher-order response results, as well as the differentiability of the variance in quenched central limit theorems (CLTs), for random dynamical systems (RDSs) that we then apply to random Anosov diffeomorphisms and random U(1) extensions of expanding maps. We emphasize that our results apply to random dynamical systems over a general ergodic base map, and are obtained without resorting to infinite-dimensional multiplicative ergodic theory.