Caste differentiation and division of labour are the hallmarks of insect societies and at the root of their ecological success. Kin selection predicts that caste determination should result from environmentally induced differences in gene expression, a prediction largely supported by empirical data. However, two exceptional cases of genetically determined caste differentiation have recently been found in harvester ants. Here we show that genetic caste determination evolved in these populations after complex hybridization events. We identified four distinct genetic lineages, each consisting of unique blends of the genomes of the parental species, presumably Pogonomyrmex barbatus and P. rugosus. Crosses between lineages H1 and H2 and between J1 and J2 give rise to workers, whereas queens develop from within-lineage matings. Although historical gene flow is evident, genetic exchange among lineages and between lineages and the parental species no longer occurs. This unusual system of caste determination seems to be evolutionarily stable.