My watch list

For a long time, doctors have been diagnosing disorders of the body’s hematopoietic system using a light microscope. The analysis of individual blood cells is largely performed manually. Now, artificial intelligence can lend them a digital hand.

Diagnosing malignant diseases of the hematopoietic system requires a range of different laboratory techniques. In addition to modern immunological and molecular genetic methods, the process of morphologically diagnosing white blood cells using a light microscope, as introduced by the pathologist Rudolf Virchow over 170 years ago [1], continues to play an essential role in this effort.

A cornerstone of hematological diagnostics: Cytomorphology

In contrast to many other molecular methods which have been computerized for a long time now, morphological diagnostics still rely on assessments performed by human cytologists. In this process, leukocytes from peripheral blood or bone marrow are applied to a slide, stained and subsequently manually screened and classified at high optical magnification. Current standards stipulate that at least 200 cells should be evaluated per patient [2], which can be very time-consuming and limit the number of available examinations. Furthermore, the results are subject to the personal interpretation and professional experience of the respective examiner. This diminishes the reproducibility of morphological diagnostics [3].

Assistance from novel software: Using deep learning for image analysis

In this type of situation, it makes sense to resort to advances in automated image analysis. Over the last few years, scientists have succeeded in using artificial neural networks and methods known as “deep learning” to develop automated image analysis systems that can classify image content with the same accuracy as a human examiner [4]. These methods are also successfully used on data generated by medical imaging, e.g. in radiology, pathology and dermatology.

One of the decisive advantages of neural networks is that the software automatically identifies relevant structural elements in images, based on the training data it receives as input. Steps such as explicitly measuring the size of the nucleus or staining the cytoplasm, for example, are not required.

Data quality and quantity must fulfill high standards

Successfully employing neural networks in image analysis hinges on the availability of large amounts of image data, which, like in routine diagnostics, must be annotated by cytologists. For this purpose, our working group created a data set containing 18,000 images of individual leukocytes taken from 100 patients diagnosed with acute myeloid leukemia (AML) and from 100 control patients. The specimens were digitized at 100x magnification with the help of a slide scanner. Like in routine diagnostics, the image data were then annotated by two medical technicians with extensive experience in cytological analysis on the basis of a standard 15-category scheme used in hematology. This allowed us to compare the consistency of the annotations from two independent sources [5].

One major prerequisite for applying neural networks is the availability of high-quality image data and annotations. For medical image data, however, meeting the necessary requirements in terms of the quality and quantity of image data can be particularly difficult. In order to provide a starting point for further advances in the field, the authors therefore published the image data set on a platform called “The Cancer Imaging Archive,” which is run by the U.S. National Cancer Institute [6].

Cell identification with human accuracy

This image data set was then used to train a number of different neural networks to classify individual white blood cells in the standard diagnostic scheme comprising 15 morphological categories. Furthermore, the networks can be used to find answers to two questions that are of central importance to diagnostics and therapy: i) whether a particular cell is typical of a blast cell, i.e. whether it possesses an AML-like shape, and ii) whether the cell would be present in the specimen in normal cases. The neural network answered both of these questions with the accuracy of a human annotator. Also, with regard to the direct classification of individual cells, the neural network outperformed other approaches. The success of deep learning can therefore be applied to the morphological examination of leukocytes.

Peering into the “black box”

Fig. 1 Sample image from the single-cell data set next to a saliency map, which illustrates how relevant individual pixels are for the network’s decision-making process when classifying these cells. The lighter a pixel, the more relevant it is for cell classification.

One aspect of neural networks that could potentially be problematic when it comes to applying these networks in the medical field is that, on the one hand, they allow for a very precise classification of image data, but on the other, they offer no explanation as to why each cell is put into a particular category. This property is sometimes called the “black box” of neural networks. For scientists, however, it would be desirable if the networks went beyond merely classifying the image content and also revealed how the classification decision was made. Over the last years, this question has developed into an active field of research in its own right [7]. With regard to the classification of leukocytes, we calculated what are known as saliency maps [8] that allowed us to illustrate which pixels of an image the network considered particularly relevant when making a classification decision (Fig. 1). As a result, it emerged that the network had learned to analyze the same areas of images that would also be of importance for human cytologists, such as the structure of the cell nucleus and the cytoplasm.

Conclusion: Opportunities and challenges

When classifying leukocytes to diagnose types of leukemia, neural networks performed at a very high level in answering clinically relevant questions, for example related to the presence of blast cells. They also classified cells into standard morphological categories with great precision. Examinations of such networks indicate that they classify cells based on images structures that are also relevant for human examiners.

Despite these successes, however, there remains a potential for errors to be made. For example, this method imposes a high level of requirements on the whole process of pre-analytics, staining and digitization, e.g. to rule out systematic distortions of the training data. Efforts to standardize many of these aspects have been made over the past years [9].

Even very successful networks can potentially make an erroneous classification, which is why the results provided by the network must always be viewed in the context of other lab results and clinical parameters. If these aspects are taken into account, however, neural networks have the potential to provide valuable assistance in making diagnostic decisions.


The work presented is a joint project of the Helmholtz Zentrum München, German Research Center for Environmental Health, Munich, Germany, and the University Hospital of Ludwig-Maximilians-Universität (LMU) München, Munich, led by Dr. Carsten Marr and Dr. Christian Matek from the Institute of Computational Biology at Helmholtz Zentrum München as well as Prof. Dr. med Karsten Spiekermann from the Department of Medicine III, University Hospital, LMU Munich, Campus Großhadern.


Category: Bioinformatics | Diagnosis

[1] Virchow R. (1845) Weisses Blut, Froriep’s Notizen, 36, 151-156
[2] Wei, A. H., Löwenberg, B., Bloomfield, C. D. (2017) Diagnosis and management of AML in adults: 2017 ELN recommendations from an international expert panel, Blood, 129, 424-447, DOI: 10.1182/blood-2016-08-733196
[3] Font, P. et al. (2013) Inter-observer variance with the diagnosis of myelodysplastic syndromes (MDS) following the 2008 WHO classification, Ann. Hematol., 92, 19-24, DOI: 10.1007/s00277-012-1565-4
[4] Goodfellow, I., Bengio, Y., Courville, A. (2016) Deep Learning, Cambridge, MA, MIT Press
[5] Matek C., Schwarz, S., Spiekermann, K., Marr C. (2019) Human-level recognition of blast cells in acute myeloid leukemia with convolutional neural networks, Nat. Mach. Intell. 1, 538-544, DOI: 10.1038/s42256-019-0101-9
[6] Matek, C., Schwarz, S., Spiekermann, K., Marr, C. A single-cell morphological dataset of leukocytes from AML patients and non-malignant controls (AML-Cytomorphology_LMU). The Cancer Imaging Archive, on the internet at: https://doi.org/10.7937/tcia.2019.36f5o9ld (as of Sep 20, 2020)
[7] Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., Müller, K. -R. (Eds.) (2019) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Lecture Notes in Artificial Intelligence, 11700, Heidelberg, Springer
[8] Simonyan, K., Vedaldi, A., Zisserman, A. (2014) In “Deep inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps,” pp. 1–8, International Conference on Learning Representations (ICLR)
[9] Haroske, G., Zwönitzer, R., Hufnagl, P. et al. (2018) Leitfaden „Digitale Pathologie in der Diagnostik“, Pathologe, 39, 216-221, DOI: 10.1007/s00292-018-0433-y

Date of publication: 12-Nov-2020

Facts, background information, dossiers

  • diagnosis
  • hematopoietic system
  • blood cells
  • morphological diagnosis
  • leukocytes
  • leukemia
  • hematological diagnosis
  • light microscopes
  • artificial intelligence
  • deep learning
  • image analysis
  • neural networks

More about Helmholtz Zentrum München

  • News

    Node-centric expression models (NCEMs): Graph-neural networks reveal communication between cells

    How do single cells communicate in a tissue? How can these interactions be modeled, while retaining information of spatial context? Researchers around Fabian Theis from Helmholtz Munich Computational Health Center and Technical University of Munich (TUM) have generated a new method to repre ... more

    How tiny changes help T cells survive

    The chemical modification of nucleic acids, known as methylation, exists not only on DNA but also on RNA. So far, it is still unclear whether this methylation is important for certain cell types and what effects it has on the interaction of cells in the body. The most abundant modification ... more

    The oat genome unlocks the unique health benefits of oats

    Researchers have succeeded in sequencing and characterizing the entire genome of oat. Compared to other cereals and humans, the oat genome architecture is very complex. Scientists from Helmholtz Munich, Lund University and the ScanOats network finally elucidated at the genetic level why oat ... more

  • Authors

    Dr. Carsten Marr

    Carsten Marr, born in 1977, received his diploma in general physics from the Technische Universität München in 2002. He wrote his diploma thesis at the Max-Planck-Institute for Quantum Optics, Garching, Germany, and in 2003 visited the Quantum Information and Quantum Optics Theory Group at ... more

    Dr. Christian Matek

    Christian Matek, born in 1986, received undergraduate degrees in both Physics and Medicine in Munich. He then moved to the UK and finished his DPhil in Theoretical Physics at Oxford University in 2014. Since 2017, his main research interest has been applying artificial intelligence and mach ... more

q&more – the networking platform for quality excellence in lab and process

The q&more concept is to increase the visibility of recent research and innovative solutions, and support the exchange of knowledge. In the broad spectrum of subjects covered, the focus is on achieving maximum quality in highly innovative sectors. As a modern knowledge platform, q&more offers market participants one-of-a-kind networking opportunities. Cutting-edge research is presented by authors of international repute. Attractively presented in a high-quality context, and published in German and English, the original articles introduce new concepts and highlight unconventional solution strategies.

> more about q&more

q&more is supported by:


Your browser is not current. Microsoft Internet Explorer 6.0 does not support some functions on Chemie.DE