We have carried out systematic state-of-the-art calculations using density-functional theory, norm-conserving pseudopotentials, and large supercells in order to investigate the diffusion mechanisms of B, P, As, and Sb in Si under both equilibrium and non-equilibrium concentrations of intrinsic point defects. In addition, we have developed a theory for the non-equilibrium concentrations of the relevant diffusing species from which expressions for the activation energies may be derived. In equilibrium, we find that vacancies and self-interstitials mediate the diffusion of B, P, and As with comparable activation energies, but we show from our non-equilibrium diffusion calculations that these impurities have a dominant interstitial component. Sb diffusion, on the other hand, is mediated primarily by vacancies. We also find that the direct exchange mechanism plays only a minor role in all cases.