Experiments previously reported in the literature suggest that people with dyslexia have a deficit in categorical perception. However, it is still unclear whether the deficit is specific to the perception of speech sounds or whether it more generally affects auditory function. In order to investigate the relationship between categorical perception and dyslexia, as well as the nature of this categorization deficit, speech specific or not, the discrimination responses of children who have dyslexia and those of average readers to sinewave analogues of speech sounds were compared. These analogues were presented in two different conditions, either as nonspeech whistles or as speech sounds. Results showed that children with dyslexia are less categorical than average readers in the speech condition, mainly because they are better at discriminating acoustic differences between stimuli belonging to the same category. In the nonspeech condition, discrimination was also better for children with dyslexia, but differences in categorical perception were less clear-cut. Further, the location of the categorical boundary on the stimulus continuum differed between speech and nonspeech conditions. As a whole, this study shows that categorical deficit in children with dyslexia results primarily from an increased perceptibility of within-category differences and that it has a speech-specific component. These findings may have profound implications for learning and re-education.