CytoSAE: Interpretable Cell Embeddings for Hematology

tl;dr: CytoSAE applies sparse autoencoders to hematology foundation models, discovering interpretable morphological concepts from peripheral blood and bone marrow cytology images — with sub-cellular explainability.

Abstract

Sparse autoencoders (SAEs) emerged as a promising tool for mechanistic interpretability of transformer-based foundation models. Very recently, SAEs were also adopted for the visual domain, enabling the discovery of visual concepts and their patch-wise attribution to tokens in the transformer model. While a growing number of foundation models emerged for medical imaging, tools for explaining their inferences are still lacking. In this work, we show the applicability of SAEs for hematology. We propose CytoSAE, a sparse autoencoder which is trained on over 40,000 peripheral blood single-cell images. CytoSAE generalizes to diverse and out-of-domain datasets, including bone marrow cytology, where it identifies morphologically relevant concepts which we validated with medical experts. Furthermore, we demonstrate scenarios in which CytoSAE can generate patient-specific and disease-specific concepts, enabling the detection of pathognomonic cells and localized cellular abnormalities at the patch level. We quantified the effect of concepts on a patient-level AML subtype classification task and show that CytoSAE concepts reach performance comparable to the state-of-the-art, while offering explainability on the sub-cellular level.

Citation

@inproceedings{dasdelen2025cytosae,
    title={CytoSAE: Interpretable Cell Embeddings for Hematology},
    author={Muhammed Furkan Dasdelen and Hyesu Lim and Michele Buck and Katharina S. G{\"o}tze and Carsten Marr and Steffen Schneider},
    year={2025},
    booktitle={International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)},
    url={https://arxiv.org/abs/2507.12464}
}