We used voxel-based morphometry (VBM) to examine human brain asymmetry and the effects of sex and handedness on brain structure in 465 normal adults. We observed significant asymmetry of cerebral grey and white matter in the occipital, frontal, and temporal lobes (petalia), including Heschl's gyrus, planum temporale (PT) and the hippocampal formation. Males demonstrated increased leftward asymmetry within Heschl's gyrus and PT compared to females. There was no significant interaction between asymmetry and handedness and no main effect of handedness. There was a significant main effect of sex on brain morphology, even after accounting for the larger global volumes of grey and white matter in males. Females had increased grey matter volume adjacent to the depths of both central sulci and the left superior temporal sulcus, in right Heschl's gyrus and PT, in right inferior frontal and frontomarginal gyri and in the cingulate gyrus. Females had significantly increased grey matter concentration extensively and relatively symmetrically in the cortical mantle, parahippocampal gyri, and in the banks of the cingulate and calcarine sulci. Males had increased grey matter volume bilaterally in the mesial temporal lobes, entorhinal and perirhinal cortex, and in the anterior lobes of the cerebellum, but no regions of increased grey matter concentration.