The early diagnostic value of glucose hypometabolism and atrophy as potential neuroimaging biomarkers of mild cognitive impairment (MCI) and Alzheimer's disease (AD) have been extensively explored using [18F]fluorodeoxyglucose positron emission tomography (FDG-PET) and structural magnetic resonance imaging (MRI). The vast majority of previous imaging studies neglected the effects of single factors, such as age, symptom severity or time to conversion in MCI thus limiting generalisability of results across studies. Here, we investigated the impact of these factors on metabolic and structural differences. FDG-PET and MRI data from AD patients (n = 80), MCI converters (n = 65) and MCI non-converters (n = 64) were compared to data of healthy subjects (n = 79). All patient groups were split into subgroups by age, time to conversion (for MCI), or symptom severity and compared to the control group. AD patients showed a strongly age-dependent pattern, with younger patients showing significantly more extensive reductions in gray matter volume and glucose utilisation. In the MCI converter group, the amount of glucose utilisation reduction was linked to the time to conversion but not to atrophy. Our findings indicate that FDG-PET might be more closely linked to future cognitive decline whilst MRI being more closely related to the current cognitive state reflects potentially irreversible damage.