Microbiome research in the last two decades has delivered as a key finding that the human intestine hosts a unique and complex ecosystem with many variables affecting the composition of the microbiota and in turn its function in metabolism and immune defence. Hundreds of external (environmental) factors have meanwhile been identified as significantly associated with bacterial biomass and diversity and, amongst these, diet is considered as a key determinant of microbial populations. However, dietary intervention studies, including those with fermentable substrates that have bulk effects on bowel functions, have revealed only very minor effects on overall microbiome composition and usually show only a very few species changing in population size. What that means in the context of hundreds of different species coexisting in competition or mutualism in the human colon is far from understood. This review addresses some of the current limits in research on diet effects by taking anatomical and physiological features of the intestine into consideration. It also provides some recommendations on future human studies needed to assess how the diet influences the microbiome and associated effects on metabolic health.