A plethora of data suggests a role for estrogen in cognitive function and genetic variants in the estrogen receptors ESR1 and ESR2 have been implicated in a range of hormone-sensitive diseases. It remains unknown however, whether ESR polymorphisms are associated with the risk of decline in specific domains of cognitive function. Data came from 3799 non-demented, community-dwelling elderly women recruited in France to the 3C Study. A short cognitive test battery was administered at baseline and 2, 4 and 7 years follow-up to assess global function, verbal fluency, visual memory, psychomotor speed and executive function. Detailed socio-demographic, behavioral, physical and mental health information was also gathered and genotyping of five common ESR1 and ESR2 polymorphisms was also performed. In multivariable-adjusted Cox analysis, ESR1 rs2234693 and rs9340799 were not significantly associated with the risk of decline on any of the cognitive tasks. However, significant associations with ESR2 polymorphisms were identified. The A allele of rs1256049 was associated with an increased risk of substantial decline in visual memory (HR:1.64, 95% CI: 1.23-2.18, p=0.0007), psychomotor speed (HR:1.43, 95% CI: 1.12-1.83, p=0.004), and on the incidence of Mild Cognitive Impairment (HR:1.31, 95% CI: 1.05-1.64, p=0.02). There was also a weaker association between the A allele of rs4986938 and a decreased risk of decline in psychomotor speed. Our large multicentre prospective study provides preliminary evidence that ESR2 genetic variants may be associated with specific cognitive domains and suggests that further examination of the role of this gene in cognitive function is warranted.