Comparing methods of detecting and segmenting unruptured intracranial aneurysms on TOF-MRAS: The ADAM challenge


Ressource 1Download: Timmins2021ADAMchallenge-published.pdf (3959.46 [Ko])
State: Public
Version: Final published version
License: CC BY 4.0
Serval ID
Article: article from journal or magazin.
Comparing methods of detecting and segmenting unruptured intracranial aneurysms on TOF-MRAS: The ADAM challenge
Timmins Kimberley M., Schaaf Irene C., Bennink Edwin, Ruigrok Ynte M., An Xingle, Baumgartner Michael, Bourdon Pascal, De Feo Riccardo, Di Noto Tommaso, Dubost Florian, Fava-Sanches Augusto, Feng Xue, Giroud Corentin, Hu Minghui, Jaeger Paul F., Kaiponen Juhana, Klimont Michał, Li Yuexiang, Li Hongwei, Lin Yi, Loehr Timo, Ma Jun, Maier-Hein Klaus H., Marie Guillaume, Menze Bjoern, Richiardi Jonas, Rjiba Saifeddine, Shah Dhaval, Shit Suprosanna, Tohka Jussi, Urruty Thierry, Walińska Urszula, Yang Xiaoping, Yang Yunqiao, Yin Yin, Velthuis Birgitta K., Kuijf Hugo J.
Working group(s)
Group, Inteneural
Publication state
Issued date
Accurate detection and quantification of unruptured intracranial aneurysms (UIAs) is important for rupture risk assessment and to allow an informed treatment decision to be made. Currently, 2D manual measures used to assess UIAs on Time-of-Flight magnetic resonance angiographies (TOF-MRAs) lack 3D information and there is substantial inter-observer variability for both aneurysm detection and assessment of aneurysm size and growth. 3D measures could be helpful to improve aneurysm detection and quantification but are time-consuming and would therefore benefit from a reliable automatic UIA detection and segmentation method. The Aneurysm Detection and segMentation (ADAM) challenge was organised in which methods for automatic UIA detection and segmentation were developed and submitted to be evaluated on a diverse clinical TOF-MRA dataset. A training set (113 cases with a total of 129 UIAs) was released, each case including a TOF-MRA, a structural MR image (T1, T2 or FLAIR), annotation of any present UIA(s) and the centre voxel of the UIA(s). A test set of 141 cases (with 153 UIAs) was used for evaluation. Two tasks were proposed: (1) detection and (2) segmentation of UIAs on TOF-MRAs. Teams developed and submitted containerised methods to be evaluated on the test set. Task 1 was evaluated using metrics of sensitivity and false positive count. Task 2 was evaluated using dice similarity coefficient, modified hausdorff distance (95th percentile) and volumetric similarity. For each task, a ranking was made based on the average of the metrics. In total, eleven teams participated in task 1 and nine of those teams participated in task 2. Task 1 was won by a method specifically designed for the detection task (i.e. not participating in task 2). Based on segmentation metrics, the top two methods for task 2 performed statistically significantly better than all other methods. The detection performance of the top-ranking methods was comparable to visual inspection for larger aneurysms. Segmentation performance of the top ranking method, after selection of true UIAs, was similar to interobserver performance. The ADAM challenge remains open for future submissions and improved submissions, with a live leaderboard to provide benchmarking for method developments at
Challenge, Segmentation, Detection, Aneurysms, Angiography
Create date
03/06/2021 13:47
Last modification date
04/06/2021 7:08
Usage data