Approaches to analyze and cluster T-cell receptor (TCR) repertoires to reflect antigen specificity are critical for the diagnosis and prognosis of immune-related diseases and the development of personalized therapies. Sequence-based approaches showed success but remain restrictive, especially when the amount of experimental data used for the training is scarce. Structure-based approaches which represent powerful alternatives, notably to optimize TCRs affinity toward specific epitopes, show limitations for large-scale predictions. To handle these challenges, TCRpcDist is presented, a 3D-based approach that calculates similarities between TCRs using a metric related to the physico-chemical properties of the loop residues predicted to interact with the epitope. By exploiting private and public datasets and comparing TCRpcDist with competing approaches, it is demonstrated that TCRpcDist can accurately identify groups of TCRs that are likely to bind the same epitopes. Importantly, the ability of TCRpcDist is experimentally validated to determine antigen specificities (neoantigens and tumor-associated antigens) of orphan tumor-infiltrating lymphocytes (TILs) in cancer patients. TCRpcDist is thus a promising approach to support TCR repertoire analysis and TCR deorphanization for individualized treatments including cancer immunotherapies.