On the Scale Invariance in State of the Art CNNs Trained on ImageNet

Details

Serval ID
serval:BIB_4E9A1C9BC5FA
Type
Article: article from journal or magazin.
Collection
Publications
Institution
Title
On the Scale Invariance in State of the Art CNNs Trained on ImageNet
Journal
Machine Learning and Knowledge Extraction
Author(s)
Graziani Mara, Lompech Thomas, Müller Henning, Depeursinge Adrien, Andrearczyk Vincent
ISSN
2504-4990
Publication state
Published
Issued date
03/04/2021
Volume
3
Number
2
Pages
374-391
Language
english
Abstract
The diffused practice of pre-training Convolutional Neural Networks (CNNs) on large natural image datasets such as ImageNet causes the automatic learning of invariance to object scale variations. This, however, can be detrimental in medical imaging, where pixel spacing has a known physical correspondence and size is crucial to the diagnosis, for example, the size of lesions, tumors or cell nuclei. In this paper, we use deep learning interpretability to identify at what intermediate layers such invariance is learned. We train and evaluate different regression models on the PASCAL-VOC (Pattern Analysis, Statistical modeling and ComputAtional Learning-Visual Object Classes) annotated data to (i) separate the effects of the closely related yet different notions of image size and object scale, (ii) quantify the presence of scale information in the CNN in terms of the layer-wise correlation between input scale and feature maps in InceptionV3 and ResNet50, and (iii) develop a pruning strategy that reduces the invariance to object scale of the learned features. Results indicate that scale information peaks at central CNN layers and drops close to the softmax, where the invariance is reached. Our pruning strategy uses this to obtain features that preserve scale information. We show that the pruning significantly improves the performance on medical tasks where scale is a relevant factor, for example for the regression of breast histology image magnification. These results show that the presence of scale information at intermediate layers legitimates transfer learning in applications that require scale covariance rather than invariance and that the performance on these tasks can be improved by pruning off the layers where the invariance is learned. All experiments are performed on publicly available data and the code is available on GitHub.
Keywords
NEURAL-NETWORKS, scale invariance, deep learning, interpretability, medical imaging
Web of science
Open Access
Yes
Funding(s)
Swiss National Science Foundation / 05320_179069
Create date
28/05/2021 17:13
Last modification date
16/12/2021 7:32
Usage data