RNA polymerase III (Pol III) synthesizes tRNAs and other small noncoding RNAs to regulate protein synthesis. Dysregulation of Pol III transcription has been linked to cancer, and germline mutations in genes encoding Pol III subunits or tRNA processing factors cause neurogenetic disorders in humans, such as hypomyelinating leukodystrophies and pontocerebellar hypoplasia. Here we describe an autosomal recessive disorder characterized by cerebellar hypoplasia and intellectual disability, as well as facial dysmorphic features, short stature, microcephaly, and dental anomalies. Whole-exome sequencing revealed biallelic missense alterations of BRF1 in three families. In support of the pathogenic potential of the discovered alleles, suppression or CRISPR-mediated deletion of brf1 in zebrafish embryos recapitulated key neurodevelopmental phenotypes; in vivo complementation showed all four candidate mutations to be pathogenic in an apparent isoform-specific context. BRF1 associates with BDP1 and TBP to form the transcription factor IIIB (TFIIIB), which recruits Pol III to target genes. We show that disease-causing mutations reduce Brf1 occupancy at tRNA target genes in Saccharomyces cerevisiae and impair cell growth. Moreover, BRF1 mutations reduce Pol III-related transcription activity in vitro. Taken together, our data show that BRF1 mutations that reduce protein activity cause neurodevelopmental anomalies, suggesting that BRF1-mediated Pol III transcription is required for normal cerebellar and cognitive development.