Yushan Zheng

The construction of medical centers is rapidly advancing, generating a vast amount of whole slide images (WSIs). Content-based histopathological image retrieval (CBHIR) unlocks the rich digital morphologic content of WSIs previously confined to glass slides. Foundation models trained on large-scale pathology data have shown remarkable generalization and transfer capabilities, providing a powerful basis for CBHIR. However, deploying pathology foundation models across different centers remains challenging due to cross-center domain shifts and continual data expansion, which can lead to feature drift during long-term model adaptation. To address these issues, we present a continual learning framework that adapts pathology foundation models for continual cross-center WSI retrieval (CCBHIR). Our framework aligns outputs of pre-trained pathology foundation models into a unified latent domain by generating instance-wise prompts that dynamically mitigate domain discrepancies. In addition, an embedding consistency replay mechanism enables stable and efficient feature rehearsal without rebuilding the entire index, thus preserving both forward and backward retrieval compatibility across centers. Evaluations on a large-scale continual retrieval dataset comprising 10,837 WSIs from TCGA projects demonstrate that our framework achieves superior intra-center and cross-center retrieval performance compared with state-of-the-art continual learning methods. This work provides an effective strategy for adapting pathology foundation models to real-world, multi-center deployment scenarios, bridging the gap between foundation model research and practical computational pathology applications. The code is available at https://github.com/OliverZXY/CCBHIR.

Fusing histopathology images and genomics data with deep learning has significantly advanced precision oncology. However, genomics data is often missing due to its high acquisition cost and complexity in real-world clinical scenarios. Existing solutions aim to reconstruct genomics data from histopathology images. Nevertheless, these methods typically relied only on individual cases and overlooked the potential relationships among cases. Additionally, they failed to take advantage of the authentic genomics data of diagnostically related cases that are accessible from training for inference. In this work, we propose a novel Multi-modal Structural Representation Learning (MSRL) framework for data-efficient precision oncology. We pre-train a histopathology-genomics multi-modal representation graph adopting Graph Structure Learning (GSL) to construct inter-case relevance based on the data inherently. During the fine-tuning stage, we dynamically capture structural relevance between the training cases and the acquired authentic cases for precise prediction. MSRL leverages prior inter-case associations and authentic genomics data from diagnosed cases based on the graph, which contributes to effective inference based on the single histopathology image modality. We evaluated MSRL on public TCGA datasets with 7,263 cases across various tasks, including survival prediction, cancer grading, and gene mutation prediction. The results demonstrate that MSRL significantly outperforms existing missing-genomics generation approaches with improvements of 1.44% to 3.12% in C-Index on survival prediction tasks and achieves comparable performance to multi-modal fusion methods.

Gigapixel image analysis, particularly for whole slide images (WSIs), often relies on multiple instance learning (MIL). Under the paradigm of MIL, patch image representations are extracted and then fixed during the training of the MIL classifiers for efficiency consideration. However, the invariance of representations makes it difficult to perform data augmentation for WSI-level model training, which significantly limits the performance of the downstream WSI analysis. The current data augmentation methods for gigapixel images either introduce additional computational costs or result in a loss of semantic information, which is hard to meet the requirements for efficiency and stability needed for WSI model training. In this paper, we propose a Promptable Representation Distribution Learning framework (PRDL) for both patch-level representation learning and WSI-level data augmentation. Meanwhile, we explore the use of prompts to guide data augmentation in feature space, which achieves promptable data augmentation for training robust WSI-level models. The experimental results have demonstrated that the proposed method stably outperforms state-of-the-art methods.

Fine-grained cross-modal retrieval (FGCR) for digital pathology plays a crucial role in diagnostic support systems, enabling targeted searches using detailed descriptive queries. However, current research on FGCR for histopathology images faces three main challenges: the domain gap between medical language and histopathology images, the lack of robust fine-grained feature representation, and the absence of a comprehensive dataset to support such analysis. In this paper, we present a novel medical language and image representation learning framework specifically designed for FGCR in digital pathology. Our approach addresses the domain gap through a novel Knowledge-guided Contrastive Learning (KCL) strategy for representation alignment between texts and images in the histopathology domain. We also introduce a Dual-transformer network that enhances multimodal feature learning by integrating both global context and local details from pathology images. To facilitate research in this area, we have created the BreaKHis Cross-Modal Retrieval (BKCMR) dataset, containing over 5,000 image-text pairs with detailed morphological descriptions created by pathologists. Extensive experiments demonstrate that our method significantly outperforms current state-of-the-art approaches in cross-modal retrieval tasks, achieving substantial improvements across all evaluation metrics. This research represents an important step toward enabling more precise and effective cross-modal searches in digital pathology, with potential applications in diagnostic assistance and educational platforms.

Attention-based transformer networks have shown immense promise for enhancing feature representation by capturing token correlations. However, the standard transformer architecture may not fully exploit the complex contextual dependencies or spatial relationships present in histopathology whole slide image (WSI) patches. This paper introduces a kernel attention transformer (KAT) architecture designed specifically for histopathology WSI analysis. Unlike traditional transformers, our KAT network learns kernel-based attention patterns that offer more varied attention to effectively model the complex relationships in histopathology images. Specifically, we introduced a designed kernel structure to represent attention patterns, which significantly expands the representational capacity beyond what the traditional dot-product attention allows, while maintaining training efficiency. Further, to improve its computational efficiency on high-resolution histopathology WSIs, we developed a memory-token-based KAT classifier that efficiently models global dependencies across the entire slide while being computationally tractable. The proposed method was evaluated on four public histopathology WSI datasets for classification. Experimental results demonstrated that our method outperforms state-of-the-art approaches, achieving significantly better accuracy, particularly on the most challenging histopathology tasks.