The interplay between selection and aspects of the genetic architecture of traits (such as linkage, dominance, and epistasis) can either drive or constrain speciation [1-3]. Despite accumulating evidence that speciation can progress to "intermediate" stages-with populations evolving only partial reproductive isolation-studies describing selective mechanisms that impose constraints on speciation are more rare than those describing drivers. The stick insect Timema cristinae provides an example of a system in which partial reproductive isolation has evolved between populations adapted to different host plant environments, in part due to divergent selection acting on a pattern polymorphism [4, 5]. Here, we demonstrate how selection on a green/melanistic color polymorphism counteracts speciation in this system. Specifically, divergent selection between hosts does not occur on color phenotypes because melanistic T. cristinae are cryptic on the stems of both host species, are resistant to a fungal pathogen, and have a mating advantage. Using genetic crosses and genome-wide association mapping, we quantify the genetic architecture of both the pattern and color polymorphism, illustrating their simple genetic control. We use these empirical results to develop an individual-based model that shows how the melanistic phenotype acts as a "genetic bridge" that increases gene flow between populations living on different hosts. Our results demonstrate how variation in the nature of selection acting on traits, and aspects of trait genetic architecture, can impose constraints on both local adaptation and speciation.