Sexual dimorphism in plants may emerge as a result of sex-specific selection on traits enhancing access to nutritive resources and/or to sexual partners. Here we investigated sex-specific differences in selection of sexually dimorphic traits and in the spatial distribution of effective fecundity (our fitness proxy) in a highly dimorphic dioecious wind-pollinated shrub, Leucadendron rubrum. In particular, we tested for the effect of density on male and female effective fecundity. We used spatial and genotypic data of parent and offspring cohorts to jointly estimate individual male and female effective fecundity on the one hand and pollen and seed dispersal kernels on the other hand. This methodology was adapted to the case of dioecious species. Explicitly modelling dispersal avoids the confounding effects of heterogeneous spatial distribution of mates and sampled seedlings on the estimation of effective fecundity. We also estimated selection gradients on plant traits while modelling sex-specific spatial autocorrelation in fecundity. Males exhibited spatial autocorrelation in effective fecundity at a smaller scale than females. A higher local density of plants was associated with lower effective fecundity in males but was not related to female effective fecundity. These results suggest sex-specific sensitivities to environmental heterogeneity in L. rubrum. Despite these sexual differences, we found directional selection for wider canopies and smaller leaves in both sexes, and no sexually antagonistic selection on strongly dimorphic traits in L. rubrum. Many empirical studies in animals similarly failed to detect sexually antagonistic selection in species expressing strong sexual dimorphism, and we discuss reasons explaining this common pattern.