The role of the prefrontal cortex remains controversial. Neuroimaging studies support modality-specific and process-specific functions related to working memory and attention. Its role may also be defined by changes in its influence over other brain regions including sensory and motor cortex. We used functional magnetic imaging (fMRI) to study the free selection of actions and colours. Control conditions used externally specified actions and colours. The prefrontal cortex was activated during free selection, regardless of modality, in contrast to modality-specific activations outside prefrontal cortex. Structural equation modelling (SEM) of fMRI data was used to test the hypothesis that although the same regions of prefrontal cortex may be active in tasks within different domains, there is task-dependent effective connectivity between prefrontal cortex and non-prefrontal cortex. The SEM included high-order interactions between modality, selection and regional activity. There was greater coupling between prefrontal cortex and motor cortex during free selection and action tasks, and between prefrontal cortex and visual cortex during free selection of colours. The results suggest that the functions of the prefrontal cortex may be defined not only by selection-specific rather than modality-specific processes, but also by changing patterns of effective connectivity from prefrontal cortex to motor and sensory cortices.