Neuroscience findings have recently received critique on the lack of replications. To examine the reproducibility of brain indices of speech sound discrimination and their role in dyslexia, a specific reading difficulty, brain event-related potentials using EEG were measured using the same cross-linguistic passive oddball paradigm in about 200 dyslexics and 200 typically reading 8-12-year-old children from four countries with different native languages. Brain responses indexing speech and non-speech sound discrimination were extremely reproducible, supporting the validity and reliability of cognitive neuroscience methods. Significant differences between typical and dyslexic readers were found when examined separately in different country and language samples. However, reading group differences occurred at different time windows and for different stimulus types between the four countries. This finding draws attention to the limited generalizability of atypical brain response findings in children with dyslexia across language environments and raises questions about a common neurobiological factor for dyslexia. Our results thus show the robustness of neuroscience methods in general while highlighting the need for multi-sample studies in the brain research of language disorders.