Screening of lymph node metastases in colorectal cancer (CRC) can be a cumbersome task, but it is amenable to artificial intelligence (AI)-assisted diagnostic solution. Here, we propose a deep learning-based workflow for the evaluation of CRC lymph node metastases from digitized hematoxylin and eosin-stained sections. A segmentation model was trained on 100 whole-slide images (WSIs). It achieved a Matthews correlation coefficient of 0.86 (±0.154) and an acceptable Hausdorff distance of 135.59 μm (±72.14 μm), indicating a high congruence with the ground truth. For metastasis detection, 2 models (Xception and Vision Transformer) were independently trained first on a patch-based breast cancer lymph node data set and were then fine-tuned using the CRC data set. After fine-tuning, the ensemble model showed significant improvements in the F1 score (0.797-0.949; P <.00001) and the area under the receiver operating characteristic curve (0.959-0.978; P <.00001). Four independent cohorts (3 internal and 1 external) of CRC lymph nodes were used for validation in cascading segmentation and metastasis detection models. Our approach showed excellent performance, with high sensitivity (0.995, 1.0) and specificity (0.967, 1.0) in 2 validation cohorts of adenocarcinoma cases (n = 3836 slides) when comparing slide-level labels with the ground truth (pathologist reports). Similarly, an acceptable performance was achieved in a validation cohort (n = 172 slides) with mucinous and signet-ring cell histology (sensitivity, 0.872; specificity, 0.936). The patch-based classification confidence was aggregated to overlay the potential metastatic regions within each lymph node slide for visualization. We also applied our method to a consecutive case series of lymph nodes obtained over the past 6 months at our institution (n = 217 slides). The overlays of prediction within lymph node regions matched 100% when compared with a microscope evaluation by an expert pathologist. Our results provide the basis for a computer-assisted diagnostic tool for easy and efficient lymph node screening in patients with CRC.