Evaluating generic AutoML tools for computational pathology

Lars Ole Schwen¹, Daniela Schacherer¹, Christian Geißler², André Homeyer¹

¹ Fraunhofer MEVIS, Institute for Digital Medicine, Germany
² DAI-Labor, Technische Universität Berlin, Germany

Introduction

Image analysis tasks in computational pathology are commonly solved
using convolutional neural networks (CNNs). The selection of a
suitable CNN architecture and hyperparameters is usually done by
exploratory iterative optimization, which is computationally intensive
and requires a lot of manual work. There are tools for automated
architecture search and hyperparameter optimization, but these are
generic and not designed specifically for computational pathology.

Material and methods

We performed a comprehensive evaluation [Schwen et al., Informatics in
Medicine Unlocked 2022, http://doi.org/10/gn8zkh] of two generic
AutoML tools by repeating experiments from literature for three common
classification tasks for histological images: tissue classification
[Coudray et al., Nature Medicine 2018, http://doi.org/10/ctzr],
mutation prediction [Kather et al., Nature Medicine 2019,
http://doi.org/10/ggsd8z], and grading [Arvaniti et al., Scientific
Reports 2018, http://doi.org/10/gd4gcs]. One tool ran on-premises
(AutoGluon) and the other in the cloud (Google AutoML Vision).

Results

We found that the default CNN architectures and parameterizations of
the evaluated AutoML tools already yielded classification performance
on par with the original publications. Despite additional
computational effort, hyperparameter optimization for these tasks did
not substantially improve performance. However, performance varied
substantially between classifiers obtained from individual AutoML runs
due to non-deterministic effects.

Conclusion

Generic CNN architectures and AutoML tools can provide a viable
alternative to manual optimization of CNN architectures and
parametrizations. This allows developers of software solutions for
computational pathology to focus on more difficult-to-automate tasks,
such as data curation.

Keywords

Convolutional neural networks, AutoML, Tissue classification,
Hyperparameter optimization, Reproducibility