Understanding drivers of biodiversity patterns is of prime importance in this era of severe environmental crisis. More diverse plant communities have been postulated to represent a larger functional trait-space, more likely to sustain a diverse assembly of herbivore species. Here, we expand this hypothesis to integrate environmental, functional and phylogenetic variation of plant communities as factors explaining the diversity of lepidopteran assemblages along elevation gradients in the Swiss Western Alps. According to expectations, we found that the association between butterflies and their host plants is highly phylogenetically structured. Multiple regression analyses showed the combined effect of climate, functional traits and phylogenetic diversity in structuring butterfly communities. Furthermore, we provide the first evidence that plant phylogenetic beta diversity is the major driver explaining butterfly phylogenetic beta diversity. Along ecological gradients, the bottom up control of herbivore diversity is thus driven by phylogenetically structured turnover of plant traits as well as environmental variables.