Physically effective fiber is needed by dairy cattle to prevent ruminal acidosis. This study aimed to examine the effects of different sources of physically effective fiber on the populations of fibrolytic bacteria and methanogens. Five ruminally cannulated Holstein cows were each fed five diets differing in physically effective fiber sources over 15 weeks (21 days/period) in a Latin Square design: (1) 44.1% corn silage, (2) 34.0% corn silage plus 11.5% alfalfa hay, (3) 34.0% corn silage plus 5.1% wheat straw, (4) 36.1% corn silage plus 10.1% wheat straw, and (5) 34.0% corn silage plus 5.5% corn stover. The impact of the physically effective fiber sources on total bacteria and archaea were examined using denaturing gradient gel electrophoresis. Specific real-time PCR assays were used to quantify total bacteria, total archaea, the genus Butyrivibrio, Fibrobacter succinogenes, Ruminococcus albus, Ruminococcus flavefaciens and three uncultured rumen bacteria that were identified from adhering ruminal fractions in a previous study. No significant differences were observed among the different sources of physical effective fiber with respect to the microbial populations quantified. Any of the physically effective fiber sources may be fed to dairy cattle without negative impact on the ruminal microbial community.