Despite recent therapeutic advances, prostate cancer remains a leading cause of cancer-related death. A subset of castration resistant prostate cancers become androgen receptor (AR) signaling-independent and develop neuroendocrine prostate cancer (NEPC) features through lineage plasticity. These NEPC tumors, associated with aggressive disease and poor prognosis, are driven, in part, by aberrant expression of N-Myc, through mechanisms that remain unclear. Integrative analysis of the N-Myc transcriptome, cistrome and interactome using in vivo, in vitro and ex vivo models (including patient-derived organoids) identified a lineage switch towards a neural identity associated with epigenetic reprogramming. N-Myc and known AR-co-factors (e.g., FOXA1 and HOXB13) overlapped, independently of AR, at genomic loci implicated in neural lineage specification. Moreover, histone marks specifically associated with lineage-defining genes were reprogrammed by N-Myc. We also demonstrated that the N-Myc-induced molecular program accurately classifies our cohort of patients with advanced prostate cancer. Finally, we revealed the potential for EZH2 inhibition to reverse the N-Myc-induced suppression of epithelial lineage genes. Altogether, our data provide insights on how N-Myc regulates lineage plasticity and epigenetic reprogramming associated with lineage-specification. The N-Myc signature we defined could also help predict the evolution of prostate cancer and thus better guide the choice of future therapeutic strategies.