The difference between chronological age and the apparent age of the brain estimated from brain imaging data-the brain age gap (BAG)-is widely considered a general indicator of brain health. Converging evidence supports that BAG is sensitive to an array of genetic and nongenetic traits and diseases, yet few studies have examined the genetic architecture and its corresponding causal relationships with common brain disorders. Here, we estimate BAG using state-of-the-art neural networks trained on brain scans from 53,542 individuals (age range 3-95 years). A genome-wide association analysis across 28,104 individuals (40-84 years) from the UK Biobank revealed eight independent genomic regions significantly associated with BAG (p < 5 × 10 ^-8 ) implicating neurological, metabolic, and immunological pathways - among which seven are novel. No significant genetic correlations or causal relationships with BAG were found for Parkinson's disease, major depressive disorder, or schizophrenia, but two-sample Mendelian randomization indicated a causal influence of AD (p = 7.9 × 10 ^-4 ) and bipolar disorder (p = 1.35 × 10 ^-2 ) on BAG. These results emphasize the polygenic architecture of brain age and provide insights into the causal relationship between selected neurological and neuropsychiatric disorders and BAG.