In a world of evolving diagnostics, histology is leading the charge. Far from being superseded by artificial intelligence (AI), molecular diagnostics, or precision medicine, tissue diagnostics are an integral part of these new advances. With just fewer than 13 billion clinical laboratory tests performed annually in the US, demand is only increasing. So, what is histology doing to keep up?
AI is hitting its stride
Squamous cell carcinoma
AI is on the rise worldwide—and the field of histology is no exception to this trend. In a new study from Japan, researchers sought to improve the ability of deep learning models to provide diagnostic histopathology support.1 To do this, they asked pathologists to prepare oral squamous cell carcinoma samples, which were then digitized, labeled, and presented to existing convolutional neural networks VGG16 and ResNet50. Though several sets of parameters were tested, the highest-performing combination was VGG16 used with a learning rate scheduler and the spectral angle mapper optimizer.
“It’s clear that AI-based decision support has the capacity to support diagnostic accuracy and treatment selection across a wide range of diseases.”
But does such a tool really support histopathologists? The results indicate that it does. Across six pathologists, the average performance (measured by area under the curve) without diagnostic assistance was 0.84 for normal tissue, 0.88 for squamous cell carcinoma, and 0.81 for other entities. With diagnostic assistance, those numbers increased to 0.94 for normal tissue, 0.96 for squamous cell carcinoma, and 0.92 for other entities. Not only were the differences statistically significant, but the relationship between diagnostic assistance and pathologist performance was strong, with effect sizes from “very large” to “huge.”
Another group explored AI’s ability to predict progression in prostate cancer based on biopsy images. Felix Feng, MD, and colleagues digitized H&E-stained slides from the SPARTAN trial2 of apalutamide treatment for prostate cancer, then applied a digital histopathology-based multimodal AI (MMAI) algorithm to identify patients at risk of progression.3 The outcome: the algorithm accurately divided patients into high-risk and non-high-risk groups based on a combination of clinical and histologic information. High-risk status was associated with shorter metastasis- and progression-free survival times in the untreated group, but not in patients treated with apalutamide. This success means that MMAI may be a promising clinical decision support tool, supporting apalutamide treatment for the patients most likely to respond and allowing those at lower risk to avoid unnecessary chemotherapy.
Non-small cell lung cancer
In non-small cell lung cancer (NSCLC), AI offers similar assistance. An international team of scientists trained a supervised deep learning algorithm, Deep-IO, on H&E slides from patients with advanced NSCLC based on their responses to immune checkpoint inhibitor monotherapy.4 Deep-IO alone showed greater predictive accuracy for treatment response than PD-L1 expression (measured by immunohistochemistry) or tumor mutational burden (area under the curve 0.75 versus 0.67, and 0.61, respectively). The best performance came from a combination of Deep-IO and PD-L1 scoring, which yielded an accuracy score of 0.82, with a sensitivity of 0.70 and a specificity of 0.88. These numbers suggest that such algorithms can support improved treatment response prediction, allowing therapies to be targeted at those most likely to benefit.
Chronic kidney disease
“By implementing AI for histology-based molecular biomarker detection, busy labs will be able to quickly identify non-targetable patients, prioritize cases for further testing or review, and shorten their turnaround times.”
A research group from the University of Michigan applied machine learning to the histopathological assessment of chronic kidney disease. To do so, they developed an entirely new type of framework, known as clustering-based spatial analysis (CluSA), capable of learning spatial relationships between patterns in kidney tissue.5 This capability allows the algorithm to learn from unlabeled images, eliminating the costly and time-consuming image annotation step from the training process—and its ability to automatically identify patterns and their spatial relationships is particularly valuable in kidney disease, where spatial context is key.6 Ultimately, CluSA was able to predict eGFR at the time of biopsy with an accuracy of 0.95, sensitivity of 0.97, and specificity of 0.90. Not only that, but its ability to predict eGFR changes over one year’s time showed an accuracy of 0.84, sensitivity of 0.83, and specificity of 0.85. This level of success means that, by looking at the algorithm’s prioritized predictive features, researchers may be able to identify the most important characteristics for pathologists to assess in the diagnostic and prognostic process.
Although more research is needed to fully explore the technology’s potential, it’s clear that AI-based decision support has the capacity to support diagnostic accuracy and treatment selection across a wide range of diseases, reducing the likelihood of errors or unnecessary treatment and potentially improving patient outcomes.
Emerging techniques and technologies
Shortening turnaround times
Clinical labs are busier than ever, especially with cancer incidence on a steady rise. But with staffing unable to meet demand, the hunt is on for new ways to optimize pathology workflows. Seeing this need, researchers from Israel combined histology and AI algorithms for faster diagnosis and treatment selection in NSCLC.7 The researchers de-identified whole slide images from 435 patients with documented EGFR, ALK, and ROS1 test results—some of the most commonly tested alterations in NSCLC—and ran them through an AI classifier. The classifier, generated using multiple algorithms trained on existing H&E slides, delivered a positive or negative result for each actionable alteration.
Ultimately, the model identified actionable alterations in just over 70 percent of the patient cohort. Of the 127 patients reported as negative for all mutations, only two were later found to have actionable mutations both in EGFR for an accuracy rate of over 98 percent. The researchers hope that by implementing AI for histology-based molecular biomarker detection, busy labs will be able to quickly identify non-targetable patients, prioritize cases for further testing or review, and shorten their turnaround times.
Combining 3D imaging with computational pathology
“Combining 3D imaging with computational pathology not only allows more comprehensive sampling and analysis of tumors and their microenvironments, but may eventually enable more precise histologic assessment, diagnosis, and prognosis.”
But AI isn’t the only technology advancing tissue diagnostics. Current histology techniques introduce a risk of undersampling; viewing only one or a few biopsy sections that only represent a small fraction of the overall tumor means that important characteristics of the tumor or its tissue microenvironment may be missed. Comprehensive three-dimensional (3D) imaging of intact samples could help alleviate this risk, which is why a recent study focused on 3D assessment of NSCLC.8 The authors used hybrid open-top light-sheet microscopy to examine 20 samples at 2 µm/pixel resolution, reimaging specific regions of interest at 0.17 µm/pixel resolution.
With this approach, they were able to obtain 3D images of samples as large as 5 mm3, which they further categorized and quantified using computational pathology approaches such as machine learning. Regions as small as 0.0045 mm3 were examined, showing healthy lymphocytes and tumor cells only a few microns apart. Further analysis is underway with the aim of linking these tissue characteristics to patient outcomes, but these early results suggest that combining 3D imaging with computational pathology not only allows more comprehensive sampling and analysis of tumors and their microenvironments, but may eventually enable more precise histologic assessment, diagnosis, and prognosis.
Advancing precision oncology
Precision oncology is another rising focus in histology. A US team recently developed hybrid multiplex panels that combine RNA in situ hybridization with immunofluorescence for detailed insights into the spatial biology of the tumor microenvironment.9 Applying this technology to NSCLC and breast cancer samples allowed the researchers to probe specific immuno-oncology biomarkers, such as CD3, CD68, CD163, and PAX5, as well as identify the levels and cellular sources of cytokines such as interleukins and TNFα.
The results: NSCLC tumor cells showed greater expression of IL-2, IL-6, and TNFα than breast cancer cells, but both tumor types showed high levels of CD3+ T cells and low levels of PAX5+ B cells. Spatially speaking, CD163+ macrophages predominated within both tumor types, whereas CD68+ macrophages were found peripherally near the non-tumor microenvironment. This combination of spatial insight and semiquantitative immuno-oncology could eventually lead to a better understanding of treatment response and potentially improve therapeutic and prognostic decision-making.
Ongoing improvements: does overfixation affect biomarkers?
New technologies need not overtake old—after all, “If it ain’t broke, don’t fix it.” There is such a thing as too much fixing; fortunately, new research suggests that overfixation may have little negative effect on tissue-based biomarker assessment.
“New research suggests that overfixation may have little negative effect on tissue-based biomarker assessment.”
In a recent collaboration, researchers from France and Canada examined the impact of overfixation on the results of immunohistochemistry (IHC) testing in NSCLC.10 To do so, they fixed 25 tissue samples for an appropriate length of time (24–48 hours) and an additional five samples for 52–108 hours (overfixation). Following fixation, IHC was performed and scored in accordance with standard practice. In a long list of actionable biomarkers and immune checkpoint inhibitor targets, most showed no differences between the correctly fixed and overfixed specimens. The researchers did note a decrease in multiplex PD-L1 expression in the overfixed specimens but were surprised to see that cMet and HER2 expression was actually stronger with overfixation. However, the researchers highlight that these samples did not have corresponding specimens fixed for 24–48 hours, so further testing is needed to assess whether this increase is a true result of overfixation or an inherent characteristic of the samples themselves.
Notwithstanding the lack of impact on most biomarkers assessed in the study, the reduction in PD-L1 expression scored by cell density may suggest that other markers may see similar decreases. As a result, the authors emphasize that consistent tissue fixation times and processes are critical in all biomarker characterization, validation, and assessment studies.
The future of histology
The field is rife with new advances—all of which are underpinned by tissue. “H&E is like soul food; it’s comfort for pathologists, it tells us where we are,” said Sandro Santagata, PhD, MD, associate professor of pathology at Brigham and Women’s Hospital in a recent press statement.11 “But now we can layer on molecular information, which is a very powerful capability. We’re trying to bring new tools to the group of people that are working hard every day to diagnose tumors and other diseases.” The same can be said of new algorithms, imaging techniques, spatial biology, and even incremental improvements to existing techniques and protocols.
Where will histology go next? That remains to be seen. In Santagata’s words, “This is a whole new world, and each person has their own long list of things they’d like to see.”
- Sukegawa S et al. Effectiveness of deep learning classifiers in histopathological diagnosis of oral squamous cell carcinoma by pathologists. Sci Rep. 2023;13(1):11676. doi:10.1038/s41598-023-38343-y.
- Smith MR et al. Apalutamide treatment and metastasis-free survival in prostate cancer. N Engl J Med. 2018;378(15):1408–18. doi: 10.1056/NEJMoa1715546.
- Feng FY et al. Digital histopathology-based multimodal artificial intelligence scores predict risk of progression in a randomized phase III trial in patients with nonmetastatic castration-resistant prostate cancer. J Clin Oncol. 2023;41(16_suppl):5035. doi:10.1200/JCO.2023.41.16_suppl.5035.
- Rakaee M et al. Artificial intelligence algorithm developed to predict immune checkpoint inhibitors efficacy in non–small-cell lung cancer. J Clin Oncol. 2023;41(16_suppl):9132. doi:10.1200/JCO.2023.41.16_suppl.9132.
- Lee J et al. Clustering-based spatial analysis (CluSA) framework through graph neural network for chronic kidney disease prediction using histopathology images. Sci Rep. 2023;13(1):12701. doi:10.1038/s41598-023-39591-8.
- Noel T et al. Principles of Spatial Transcriptomics Analysis: A Practical Walk-Through in Kidney Tissue. Front Physiol. 2022;12:809346. doi:10.3389/fphys.2021.809346.
- Ofek E et al. High-confidence AI-based biomarker profiling for H&E slides to optimize pathology workflow in lung cancer. J Clin Oncol. 2023;41(16_suppl):e21207. doi:10.1200/JCO.2023.41.16_suppl.e21207.
- Stoltzfus CR et al. Quantitative 3D assessment of the lung cancer microenvironment using multi-resolution open-top light-sheet microscopy. J Clin Oncol. 2023;41(16_suppl):2616. doi:10.1200/JCO.2023.41.16_suppl.2616.
- Patel B et al. Precision immuno-oncology (IO) ISH/IF multiplex panel: Spatial detection of cytokines and IO biomarkers. J Clin Oncol. 2023;41(16_suppl):e14654. doi:10.1200/JCO.2023.41.16_suppl.e14654.
- Gérus-Durand M et al. Impact of overfixation on actionable biomarkers and checkpoint inhibitor (CKI) targets with immunohistochemistry (IHC) in a patient population with non-small cell lung cancer (NSCLC). J Clin Oncol. 2023;41(16_suppl):e15144. doi:10.1200/JCO.2023.41.16_suppl.e15144.
- Caruso C. A New Tool for Diagnosing Cancer. June 22, 2023. https://hms.harvard.edu/news/new-tool-diagnosing-cancer.