A brief exposure to rightward prismatic adaptation (PA) was shown to shift visual field representation within the inferior parietal lobule (IPL) from the right to the left hemisphere. This change in hemispheric dominance could be interpreted as (1) a general effect of discrepancy in visuomotor alignment caused by PA or (2) a direction-specific effect of rightward PA. To test these hypotheses, we compared the effects of rightward and leftward PA on visual representation in normal human subjects. Three groups of normal subjects underwent an fMRI evaluation using a simple visual detection task before and after brief PA exposure using leftward- or rightward-deviating prisms or no prisms (L-PA, R-PA, neutral groups). A two-way ANOVA group × session revealed a significant interaction suggesting that PA-induced modulation is direction specific. <i>Post hoc</i> analysis showed that L-PA enhanced the representation of the right visual field within the right IPL. Thus, a brief exposure to L-PA enhanced right hemispheric dominance within the ventral attentional system, which is the opposite effect of the previously described shift in hemispheric dominance following R-PA. The direction-specific effects suggest that the underlying neural mechanisms involve the fine-tuning of specific visuomotor networks. The enhancement of right hemispheric dominance following L-PA offers a parsimonious explanation for neglect-like symptoms described previously in normal subjects.