Liu Pei, Ji Luping, Ye Feng, Fu Bo
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Xiyuan Ave, Chengdu, 611731, Sichuan, China.
Institute of Clinical Pathology, West China Hospital, Sichuan University, Guo Xue Xiang, Chengdu, 610041, Sichuan, China.
Med Image Anal. 2024 Jan;91:103020. doi: 10.1016/j.media.2023.103020. Epub 2023 Nov 2.
The survival analysis on histological whole-slide images (WSIs) is one of the most important means to estimate patient prognosis. Although many weakly-supervised deep learning models have been developed for gigapixel WSIs, their potential is generally restricted by classical survival analysis rules and fully-supervised learning requirements. As a result, these models provide patients only with a completely-certain point estimation of time-to-event, and they could only learn from the labeled WSI data currently at a small scale. To tackle these problems, we propose a novel adversarial multiple instance learning (AdvMIL) framework. This framework is based on adversarial time-to-event modeling, and integrates the multiple instance learning (MIL) that is much necessary for WSI representation learning. It is a plug-and-play one, so that most existing MIL-based end-to-end methods can be easily upgraded by applying this framework, gaining the improved abilities of survival distribution estimation and semi-supervised learning. Our extensive experiments show that AdvMIL not only could often bring performance improvement to mainstream WSI survival analysis methods at a relatively low computational cost, but also enables these methods to effectively utilize unlabeled data via semi-supervised learning. Moreover, it is observed that AdvMIL could help improving the robustness of models against patch occlusion and two representative image noises. The proposed AdvMIL framework could promote the research of survival analysis in computational pathology with its novel adversarial MIL paradigm.
对组织学全切片图像(WSIs)进行生存分析是评估患者预后的最重要手段之一。尽管已经针对数十亿像素的WSIs开发了许多弱监督深度学习模型,但其潜力通常受到经典生存分析规则和全监督学习要求的限制。因此,这些模型只为患者提供事件发生时间的完全确定的点估计,并且目前只能从小规模的标记WSI数据中学习。为了解决这些问题,我们提出了一种新颖的对抗性多实例学习(AdvMIL)框架。该框架基于对抗性事件发生时间建模,并集成了WSI表示学习中非常必要的多实例学习(MIL)。它是即插即用的,因此通过应用此框架可以轻松升级大多数现有的基于MIL的端到端方法,从而获得改进的生存分布估计能力和半监督学习能力。我们广泛的实验表明,AdvMIL不仅通常可以以相对较低的计算成本为主流WSI生存分析方法带来性能提升,还能使这些方法通过半监督学习有效地利用未标记数据。此外,观察到AdvMIL有助于提高模型对切片遮挡和两种代表性图像噪声的鲁棒性。所提出的AdvMIL框架以其新颖的对抗性MIL范式可以推动计算病理学中生存分析的研究。