Using numerical simulations, we investigate the underlying physical effects responsible for the overall organization of chromosomal territories in interphase nuclei. In particular, we address the following three questions: (i) why are chromosomal territories with relatively high transcriptional activity on average, closer to the centre of cell's nucleus than those with the lower activity? (ii) Why are actively transcribed genes usually located at the periphery of their chromosomal territories? (iii) Why are pair-wise contacts between active and inactive genes less frequent than those involving only active or only inactive genes? We show that transcription factories-mediated contacts between active genes belonging to different chromosomal territories are instrumental for all these features of nuclear organization to emerge spontaneously due to entropic effects arising when chromatin fibres are highly crowded.