To achieve sample sizes necessary for effectively conducting genome-wide association studies (GWASs), researchers often combine data from samples possessing multiple potential sources of heterogeneity. This is particularly relevant for psychiatric disorders, where symptom self-report, differing assessment instruments, and diagnostic comorbidity complicates the phenotypes and contribute to difficulties with detecting and replicating genetic association signals. We investigated sources of heterogeneity of anxiety disorders (ADs) across five large cohorts used in a GWAS meta-analysis project using a dimensional structural modeling approach including confirmatory factor analyses (CFAs) and measurement invariance (MI) testing. CFA indicated a single-factor model provided the best fit in each sample with the same pattern of factor loadings. MI testing indicated degrees of failure of metric and scalar invariance which depended on the inclusion of the effects of sex and age in the model. This is the first study to examine the phenotypic structure of psychiatric disorder phenotypes simultaneously across multiple, large cohorts used for GWAS. The analyses provide evidence for higher order invariance but possible break-down at more detailed levels that can be subtly influenced by included covariates, suggesting caution when combining such data. These methods have significance for large-scale collaborative studies that draw on multiple, potentially heterogeneous datasets. Copyright © 2016 John Wiley & Sons, Ltd.