Enhancing Structured Pathology Report Generation With Foundation Model and Modular Design

Abstract

Pathology report generation, which requires analyzing giga-pixel Whole Slide Images (WSIs), is a complex and labor-intensive task. Despite this, it is crucial for clinical decision-making. An AI-driven report generation system can assist in producing accurate and concise reports. To address these needs, we designed a two-stage framework consisting of image analysis and text generation. This framework improves diagnostic performance and optimizes it for real-world clinical application. In the first stage, using 752 high resolution WSIs from bladder tumor tissue samples collected at Korea University Medical Center, we leveraged foundation models to extract robust features. Then, we built a multi-label classification based on key diagnostic elements to achieve high performance with an Attention-based Multi-Instance Learning (MIL) model and a Transformer-based MIL with Knowledge Distillation. For the report generation stage, we designed a T5-based text-to-text model, simplifying input representations while integrating data augmentation to improve stability and generalization. The proposed model achieved a ROUGE score of 0.87, a BLEU-4 score of 0.94, a Jaccard score of 0.89 and a BioLLM score of 0.97. An additional evaluation conducted by the institution maintaining the exclusive K-MEDICON test datasets confirmed a consistent performance yielding an overall score of 0.88, calculated as a weighted sum of aforementioned metrics. Its consistency validated through a Mean Opinion Score (MOS) evaluation by pathologists. By systematically integrating foundation models and modular structure, the benchmark results demonstrate that the task can be effectively solved with limited data and computational resources, indicating promising potential for real-world clinical adaptation.