Research on the relationship between obstructive sleep apnea and cognitive functioning has yielded conflicting results, particularly in the older population, and moderators of this association have rarely been studied. Here we investigated the cross-sectional association between obstructive sleep apnea and cognitive functioning as well as the moderating effect of age, sex, apolipoprotein E4, and obesity on this association among community-dwelling older people. We analysed data from 496 participants (71.4 ± 4.4 years; 45.6% men) of the HypnoLaus study who underwent polysomnography and a battery of neuropsychological tests. The sample was categorised as no-to-mild obstructive sleep apnea (apnea-hypopnea index 0-14.9/h; reference), moderate obstructive sleep apnea (apnea-hypopnea index 15.0-29.9/h), or severe obstructive sleep apnea (apnea-hypopnea index ≥30/h). Regression and moderation analyses were performed with adjustment for confounders. Apolipoprotein E4 and obesity moderated the association between severe obstructive sleep apnea and processing speed, whereas no moderating effects were found for age and sex. In apolipoprotein E4 carriers only, severe obstructive sleep apnea was associated with lower performance in Stroop condition 1 (B = 3.13, p = 0.024). In obese participants only, severe obstructive sleep apnea was associated with lower performance in Stroop condition 1 (B = 3.02, p = 0.025) and Stroop condition 2 (B = 3.30, p = 0.034). Severe obstructive sleep apnea was also associated with lower executive function in the whole sample according to Stroop condition 3 (B = 3.44, p = 0.020) and Stroop interference score (B = 0.24, p = 0.006). Our findings support associations of severe obstructive sleep apnea (but not moderate obstructive sleep apnea) with lower performance in processing speed and executive function in the older general population. Apolipoprotein E4 and obesity appear to be vulnerability factors that strengthen the association between severe obstructive sleep apnea and lower performance in processing speed.