Sexual selection may contribute to the evolution of plant sexual dimorphism by favoring architectural traits in males that improve pollen dispersal to mates. In both sexes, larger individuals may be favored by allowing the allocation of more resources to gamete production (a "budget" effect of size). In wind-pollinated plants, large size may also benefit males by allowing the liberation of pollen from a greater height, fostering its dispersal (a "direct" effect of size). To assess these effects and their implications for trait selection, we measured selection on plant morphology in both males and females of the wind-pollinated dioecious herb Mercurialis annua in two separate experimental common gardens at contrasting density. In both gardens, selection strongly favored males that disperse their pollen further. Selection for pollen production was observed in the high-density garden only, and was weak. In addition, male morphologies associated with increased mean pollen dispersal differed between the two gardens, as elongated branches were favored in the high-density garden, whereas shorter plants with longer inflorescence stalks were favored in the low-density garden. Larger females were selected in both gardens. Our results point to the importance of both a direct effect of selection on male traits that affect pollen dispersal, and, to a lesser extent, a budget effect of selection on pollen production.