In shade-intolerant plants such as Arabidopsis, a reduction in the red/far-red (R/FR) ratio, indicative of competition from other plants, triggers a suite of responses known as the shade avoidance syndrome (SAS). The phytochrome photoreceptors measure the R/FR ratio and control the SAS. The phytochrome-interacting factors 4 and 5 (PIF4 and PIF5) are stabilized in the shade and are required for a full SAS, whereas the related bHLH factor HFR1 (long hypocotyl in FR light) is transcriptionally induced by shade and inhibits this response. Here we show that HFR1 interacts with PIF4 and PIF5 and limits their capacity to induce the expression of shade marker genes and to promote elongation growth. HFR1 directly inhibits these PIFs by forming non-DNA-binding heterodimers with PIF4 and PIF5. Our data indicate that PIF4 and PIF5 promote SAS by directly binding to G-boxes present in the promoter of shade marker genes, but their action is limited later in the shade when HFR1 accumulates and forms non-DNA-binding heterodimers. This negative feedback loop is important to limit the response of plants to shade.