In sexually polymorphic species, the morphs are maintained by frequency-dependent selection through disassortative mating. In heterodichogamous populations in which disassortative mating occurs between the protandrous and protogynous morphs, a decrease in female fitness in one morph is hypothesized to drive sexual specialization in the other morph, resulting in dimorphic populations. We test these ideas in a population of the heterodichogamous species, Acer opalus. We assessed both prospective gender of individuals in terms of their allocations and actual parentage using microsatellites; we found that most matings in A. opalus occur disassortatively. We demonstrate that the protogynous morph is maintained by frequency-dependent selection, but that maintenance of males versus protandrous individuals depends on their relative siring success, which changes yearly. Seeds produced later in the reproductive season were smaller than those produced earlier; this should compromise reproduction through ovules in protandrous individuals, rendering them male biased in gender. Time-dependent gender and paternity analyses indicate that the sexual morphs are specialized in their earlier sexual functions, mediated by the seasonal decrease in seed size. Our results confirm that mating patterns are context-dependent and change seasonally, suggesting that sexual specialization can be driven by seasonal effects on fitness gained through one of the two sexual functions.