The gut microbiota adapts to age-related changes in host physiology but is also affected by environmental stimuli, like diet. As a source of both pre- and probiotics, dairy and fermented foods modulate the gut microbiota composition, which makes them interesting food groups to use for the investigation of interactions between diet and ageing. Here we present the effects of excluding dairy products and limiting fermented food consumption for 19 days on gut microbiota composition and circulating metabolites of 28 healthy, young (YA) and older (OA) adult men. The intervention affected gut microbial composition in both groups, with significant increases in Akkermansia muciniphila and decreases in bacteria of the Clostridiales order. Lower fasting levels of glucose and insulin, as well as dairy-associated metabolites like lactose and pentadecanoic acid, were observed after the intervention, with no effect of age. The intervention also decreased HDL and LDL cholesterol levels. Dairy fat intake was positively associated with the HDL cholesterol changes but not with the LDL/HDL ratio. In conclusion, restricting the intake of dairy and fermented foods in men modified their gut microbiota and blood metabolites, while the impact of the dietary restrictions on these outcomes was more marked than the effect of age.