Invasive aspergillosis (IA) is a life-threatening infection due to Aspergillus fumigatus and other Aspergillus spp. Drugs targeting the fungal cell membrane (triazoles, amphotericin B) or cell wall (echinocandins) are currently the sole therapeutic options against IA. Their limited efficacy and the emergence of resistance warrant the identification of new antifungal targets. Histone deacetylases (HDACs) are enzymes responsible of the deacetylation of lysine residues of core histones, thus controlling chromatin remodeling and transcriptional activation. HDACs also control the acetylation and activation status of multiple non-histone proteins, including the heat shock protein 90 (Hsp90), an essential molecular chaperone for fungal virulence and antifungal resistance. This review provides an overview of the different HDACs in Aspergillus spp. as well as their respective contribution to total HDAC activity, fungal growth, stress responses, and virulence. The potential of HDAC inhibitors, currently under development for cancer therapy, as novel alternative antifungal agents against IA is discussed.