Animals and plants are associated with symbiotic microbes whose roles range from mutualism to commensalism to parasitism. These roles may not only be taxon-specific but also dependent on environmental conditions and host factors. To experimentally test these possibilities, we drew a random sample of adult whitefish from a natural population, bred them in vitro in a full-factorial design in order to separate additive genetic from maternal environmental effects on offspring, and tested the performance of the resulting embryos under different environmental conditions. Enhancing the growth of symbiotic microbes with supplemental nutrients released cryptic additive genetic variance for viability in the fish host. These effects vanished with the concurrent addition of the water mould Saprolegnia ferax. Our findings demonstrate that the heritability of host fitness is environment-specific and critically depends on the interaction between symbiotic microbes.