Y chromosome variation is determined by several confounding factors including mutation rate, effective population size, demography, and selection. Disentangling these factors is essential to better understand the evolutionary properties of the Y chromosome. We analyzed genetic variation on the Y chromosome, X chromosome, and mtDNA of the greater white-toothed shrew, a species with low variance in male reproductive success and limited sex-biased dispersal, which enables us to control to some extent for life-history effects. We also compared ancestral (Moroccan) to derived (European) populations to investigate the role of demographic history in determining Y variation. Recent colonization of Europe by a small number of founders (combined with low mutation rates) is largely responsible for low diversity observed on the European Y and X chromosomes compared to mtDNA. After accounting for mutation rate, copy number, and demography, the Y chromosome still displays a deficit in variation relative to the X in both populations. This is possibly influenced by directional selection, but the slightly higher variance in male reproductive success is also likely to play a role, even though the difference is small compared to that in highly polygynous species. This study illustrates that demography and life-history effects should be scrutinized before inferring strong selective pressure as a reason for low diversity on the Y chromosome.