Trioecy is an uncommon sexual system in which males, females, and hermaphrodites co-occur as three clearly different gender classes. The evolutionary stability of trioecy is unclear, but would depend on factors such as hermaphroditic sex allocation and rates of outcrossing vs. selfing. Here, trioecious populations of Mercurialis annua are described for the first time. We examined the frequencies of females, males and hermaphrodites across ten natural populations and evaluated the association between the frequency of females and plant densities. Previous studies have shown that selfing rates in this species are density-dependent and are reduced in the presence of males, which produce substantially more pollen than hermaphrodites. Accordingly, we examined the evolutionary stability of trioecy using an experiment in which we (a) indirectly manipulated selfing rates by altering plant densities and the frequency of males in a fully factorial manner across 20 experimental plots and (b) examined the effect of these manipulations on the frequency of the three sex phenotypes in the next generation of plants. In the parental generation, we measured the seed and pollen allocations of hermaphrodites and compared them with allocations by unisexual plants. In natural populations, females occurred at higher frequencies in denser patches, a finding consistent with our expectations. Under our experimental conditions, however, no combination of plant densities and male frequencies was associated with increased frequencies of females. Our results suggest that the factors that regulate female frequencies in trioecious populations of M. annua are independent of those regulating male frequencies (density), and that the stable co-existence of all three sex phenotypes within populations is unlikely.