The dose-dependent toxicity of the main psychoactive component of cannabis in brain regions rich in cannabinoid CB1 receptors is well known in animal studies. However, research in humans does not show common findings across studies regarding the brain regions that are affected after long-term exposure to cannabis. In the present study, we investigate (using Voxel-based Morphometry) gray matter changes in a group of regular cannabis smokers in comparison with a group of occasional smokers matched by the years of cannabis use. We provide evidence that regular cannabis use is associated with gray matter volume reduction in the medial temporal cortex, temporal pole, parahippocampal gyrus, insula, and orbitofrontal cortex; these regions are rich in cannabinoid CB1 receptors and functionally associated with motivational, emotional, and affective processing. Furthermore, these changes correlate with the frequency of cannabis use in the 3 months before inclusion in the study. The age of onset of drug use also influences the magnitude of these changes. Significant gray matter volume reduction could result either from heavy consumption unrelated to the age of onset or instead from recreational cannabis use initiated at an adolescent age. In contrast, the larger gray matter volume detected in the cerebellum of regular smokers without any correlation with the monthly consumption of cannabis may be related to developmental (ontogenic) processes that occur in adolescence.