Older people are particularly exposed to stressful events, known to activate the hypothalamus-pituitary-adrenal axis resulting in increased cortisol levels. High cortisol has been associated with deleterious effects on cognition. We hypothesized that stressful life events could increase cortisol secretion leading to cognitive impairment. A cross-sectional analysis was conducted using data from Colaus/PsyColaus, a longitudinal population-based study among Lausanne residents. Salivary cortisol samples were obtained from 796 nondemented subjects aged at least 65. A neuropsychological battery was used to assess cognitive performance and determine the Clinical Dementia Rating Sum of Boxes (CDRSOB). Lifetime life events and their subjective impact were assessed using a validated questionnaire. The total impact of life events was associated neither with cortisol area under the curve (AUC) nor with CDRSOB nor with any cognitive domain performance. The CDRSOB was associated with the cortisol AUC, controlling for age, sex, body mass index, education and depressive symptoms (p = 0.003; B = 0.686 [0.240; 1.333]; r = 0.114). This association between CDRSOB and the cortisol AUC remained significant after controlling for life events total impact (p = 0.040; B = 0.591 [0.027; 1.155]; r = 0.106). These findings do not support the hypothesis that stressful life events increase cortisol secretion leading to cognitive impairment. The association of higher cortisol levels with poorer cognition might be not a mere reflection of stressful events but rather explained by other factors, yet to be elucidated.