分享:
分享到微信朋友圈
X
特别关注
基于DCE-MRI图像深度学习模型鉴别诊断乳腺良恶性肿瘤的价值分析
罗文斌 郑晔 刘欣 王蕾 段少银

Cite this article as: LUO W B, ZHENG Y, LIU X, et al. Value analysis of deep learning model based on DCE-MRI images in the differential diagnosis of benign and malignant breast tumors[J]. Chin J Magn Reson Imaging, 2024, 15(10): 22-29.本文引用格式:罗文斌, 郑晔, 刘欣, 等. 基于DCE-MRI图像深度学习模型鉴别诊断乳腺良恶性肿瘤的价值分析[J]. 磁共振成像, 2024, 15(10): 22-29. DOI:10.12015/issn.1674-8034.2024.10.005.


[摘要] 目的 探讨基于动态对比增强磁共振成像(dynamic contrast-enhanced magnetic resonance imaging, DCE-MRI)图像深度学习模型鉴别诊断乳腺良恶性肿瘤的价值。材料与方法 回顾性分析2018年9月至2022年12月厦门医学院附属第二医院病理学确诊303例乳腺肿瘤患者资料,良性144例,恶性159例。按7∶3的比例分层随机抽样患者,分成训练集212例、测试集91例。构建六个深度学习模型:50层深度残差网络(50-layer deep residual network, ResNet-50)、Inception-V3、Googlenet,密集连接的卷积网络(densely connected convolutional networks, DenseNet)-121、视觉几何组(visual geometry group, VGG)-19和移动神经网络(mobile neural network, MobileNet)-V3,同时应用梯度加权类激活映射(gradient-weighted class activation mapping, Grad-CAM)对模型进行可视化。最后通过第一、二轮阅片比较了深度学习模型、初级和高级放射科医师的诊断结果。通过受试者工作特征(receive operating characteristic, ROC)曲线、准确度、敏感度、特异度、阴性预测值(negative predictive value, NPV)及阳性预测值(positive predictive value, PPV)对不同深度学习模型及两轮阅片的诊断效能进行分析,计算各深度学习模型曲线下面积(area under the curve, AUC),使用DeLong检验对各模型间ROC曲线进行比较,使用配对卡方检验比较两轮阅片的诊断效能。结果 训练集ResNet-50、Inception-V3、Googlenet、DenseNet-121、VGG-19和MobileNet-V3六种深度学习模型AUC分别为0.874 [95%置信区间(confidence interval, CI):0.828~0.920]、0.771(95% CI:0.707~0.834)、0.993(95% CI:0.986~0.999)、0.926(95% CI:0.888~0.958)、0.947(95% CI:0.918~0.975)及0.945(95% CI:0.918~0.973)。测试集ResNet-50、Inception-V3、Googlenet、DenseNet-121、VGG -19和MobileNet-V3六种深度学习模型AUC分别为0.841(95% CI:0.755~0.927)、0.746(95% CI:0.641~0.851)、0.822(95% CI:0.736~0.909)、0.752(95% CI:0.650~0.855)、0.827(95% CI:0.737~0.918)及0.779(95% CI:0.685~0.874)。ResNet-50模型Grad-CAM可视化图像显示乳腺恶性肿瘤呈病灶中央激活,良性肿瘤呈周边激活。第一轮阅片,ResNet-50深度学习模型的准确度、特异度及敏感度分别为80.2%、86.7%及73.9%,初级医师的准确度、特异度及敏感度为73.6%、73.3%及73.9%,高级医师的准确度、特异度及敏感度为80.2%、80.0%及80.4%。第二轮阅片,在ResNet-50模型辅助下,初级医师准确度、特异度及敏感度增加15.4%、17.8%、13.1%(P<0.05),高级医师准确度、特异度及敏感度增加12.1%、13.3%、10.9%(P=0.001、0.031、0.063)。结论 ResNet-50模型鉴别诊断良恶性乳腺肿瘤性能最佳, 可视化图像可能成为影像诊断依据。借助该模型放射科医师鉴别诊断乳腺肿瘤良、恶性准确性明显提高,为临床决策提供客观依据。
[Abstract] Objective To explore the value of image deep learning model based on dynamic contrast-enhanced magnetic resonance imaging in differential diagnosis of benign and malignant breast tumors.Materials and Methods A total of 303 breast tumor patients diagnosed pathologically in the Second Affiliated Hospital of Xiamen Medical College from September 2018 to December 2022 were retrospectively collected, including 144 benign and 159 malignant. Stratified random sampling patients were divided into 212 training set and 91 test set according to the ratio of 7:3. Six DCE-MRI Deep learning models were constructed: 50-layer deep residual network (ResNet-50), Inception-V3, Googlenet, Densely connected convolutional networks (DenseNet)-121, visual geometry group (VGG)-19 and mobile neural network (MobileNet)-V3 were used to visualize the model simultaneously with gradient-weighted class activation mapping. Finally, the diagnostic results of the deep learning model, junior and senior radiologists were compared by the first and second rounds of reading. According to the receiver operating characteristic (ROC) curve, accuracy, sensitivity, specificity, negative predictive value and positive predictive value analyze the diagnostic efficiency of different deep learning models and two rounds of reading, calculate the area under the curve of each deep learning model, compare the ROC curves among the models with DeLong test, and compare the diagnostic efficiency of two rounds of reading with paired chi-square test.Results The AUC of the six deep learning models ResNet-50, Inception-V3, Googlenet, DenseNet-121, VGG-19 and MobileNet-V3 was 0.874 [95% confidence interval (CI): 0.828-0.920], 0.771 (95% CI: 0.707-0.834), 0.993 (95% CI: 0.986-0.999), 0.926 (95% CI: 0.888-0.958), 0.947 (95% CI: 0.918-0.975) and 0.945 (95% CI: 0.918-0.973). The test sets of ResNet-50, Inception-V3, Googlenet, DenseNet-121, VGG-19 and MobileNet-V3 had an AUC of 0.841 (95% CI: 0.755-0.927), 0.746 (95% CI: 0.641-0.851), 0.822 (95% CI: 0.736-0.909), 0.752 (95% CI: 0.650-0.855), 0.827 (95% CI: 0.737-0.918) and 0.779 (95% CI: 0.685-0.874). ResNet-50 model Grad-CAM visualization images showed that malignant breast tumors were activated in the center and benign tumors were activated in the periphery. In the first round of reading, the accuracy, specificity and sensitivity of ResNet-50 deep learning model were 80.2%, 86.7% and 73.9%, junior doctors were 73.6%, 73.3% and 73.9%, and senior doctors were 80.2%, 80.0% and 80.4%, respectively. In the second round of reading, with the assistance of ResNet-50 model, the accuracy, specificity and sensitivity of junior doctors increased by 15.4%, 17.8% and 13.1% (P<0.05), while the accuracy, specificity and sensitivity of senior doctors increased by 12.1%, 13.3% and 10.9% (P=0.001, 0.031, 0.063).Conclusions ResNet-50 model has the best performance in differential diagnosis of benign and malignant breast tumors, and visual images may become the basis of imaging diagnosis. With the help of this model, radiologists significantly improve the accuracy of differential diagnosis of benign and malignant breast tumors, which provides an objective basis for clinical decision-making.
[关键词] 乳腺肿瘤;辅助诊断;深度学习;动态增强磁共振成像;卷积神经网络
[Keywords] breast neoplasms;auxiliary diagnosis;deep learning;dynamic contrast-enhanced magnetic resonance imaging;convolutional neural network

罗文斌 1   郑晔 1   刘欣 1   王蕾 1   段少银 2*  

1 厦门医学院附属第二医院放射一科,厦门 361021

2 厦门大学附属中山医院影像科,厦门 361004

通信作者:段少银,E-mail: xmdsy@xmu.edu.cn

作者贡献声明:段少银设计本研究的方案,对稿件重要内容进行最终审阅修改;罗文斌实施研究,采集数据,起草和撰写稿件,获取、分析及解释本研究的数据,获得了厦门市医疗卫生指导性项目资助;郑晔参与本研究数据的分析和解释,对稿件的重要内容进行修改,得到了福建省自然科学基金面上项目的资助;刘欣实施研究,采集、分析数据,对稿件的材料与方法、结果部分进行撰写;王蕾对本研究病理结果进行确认、解释,并参与本研究讨论部分的撰写;全体作者都同意发表最后的修改稿,同意对本研究的所有方面负责,确保本研究的准确性和诚信。


基金项目: 福建省自然科学基金面上项目 2022J011387 厦门市医疗卫生指导性项目 3502Z20214ZD1198
收稿日期:2024-05-28
接受日期:2024-09-06
中图分类号:R445.2  R737.9 
文献标识码:A
DOI: 10.12015/issn.1674-8034.2024.10.005
本文引用格式:罗文斌, 郑晔, 刘欣, 等. 基于DCE-MRI图像深度学习模型鉴别诊断乳腺良恶性肿瘤的价值分析[J]. 磁共振成像, 2024, 15(10): 22-29. DOI:10.12015/issn.1674-8034.2024.10.005.

0 引言

       乳腺癌是女性最常见的癌症,同时也是女性癌症死亡的主要原因[1]。乳腺癌的术前精准诊断是一个临床亟须解决的难题,研究表明,动态对比增强磁共振成像(dynamic contrast-enhanced magnetic resonance imaging, DCE-MRI)是乳腺癌的主要筛查方式[2, 3, 4],对患有乳腺癌的年轻女性进行术前MRI评估,可以降低重复手术率而不增加总体乳房切除率[5]。放射科医师可以根据DCE-MRI强化方式及MRI特征识别病变,但当表现出相似的MRI影像特征,难以鉴别乳腺良恶性肿瘤[6]。刘鹏等[7]研究表明基于DCE-MRI的影像组学可以为肉芽肿性乳腺炎与乳腺癌的鉴别诊断提供较高价值,逻辑回归模型诊断效能和稳定性最佳[训练组:曲线下面积(area under the curve, AUC)0.928,准确度85.5%,敏感度83.7%,特异度88.5%;测试组:AUC 0.933,准确度83.3%,敏感度89.5%,特异度72.7%]。韩珺琪等[8]研究表明乳腺MRI影像组学和临床影像学特征的联合模型鉴别乳腺导管原位癌与导管原位癌伴微浸润有良好效能(AUC为0.840)。但基于机器学习的影像组学研究需要耗时的病灶勾画,并不可避免地受到操作者间和操作者内可重复性的影响。基于深度学习模型的研究方法可以解决这个问题。

       近年来,深度学习在医学影像领域受到广泛关注。研究表明,计算机辅助诊断模型对放射科医师诊断疾病有很大帮助[9]。目前,基于DCE-MRI的深度学习模型主要用于乳腺癌分子分型[10]、淋巴血管浸润[11]、新辅助化疗疗效[12]及腋窝淋巴结转移预测[13]。尽管深度学习模型有其优势,但放射科医师仍需要做出最终决定。这些研究很少评估放射科医师使用深度学习模型的实际益处。国内关于DEC-MRI图像深度学习模型鉴别诊断乳腺良恶性肿瘤的研究尚未见报道。本研究开发一种基于DCE-MRI图像的深度学习模型,用于鉴别诊断乳腺良恶性肿瘤,并验证该深度学习模型协助放射科医师做出决策的可行性。本研究旨为临床术前以非侵入性方式准确鉴别诊断乳腺肿瘤良恶性肿瘤提供帮助。

1 材料与方法

1.1 研究对象

       本研究遵守《赫尔辛基宣言》,已获厦门医学院附属第二医院伦理委员会批准,免除受试者知情同意,批准文号:2021027。回顾性分析2018年9月至2022年12月在厦门医学院附属第二医院接受DCE-MRI检查并符合以下标准的乳腺病变患者。纳入标准:(1)在DCE-MRI后接受活检手术的单发肿瘤患者;(2)患者具有完整的影像学和临床数据;(3)未经任何治疗。排除标准:(1)图像质量不佳;(2)随访记录丢失。最终共纳入303例患者,均为女性,年龄16~73(44±12岁)。乳腺良性肿瘤144例(纤维腺瘤114例、良性叶状肿瘤及导管内乳头状瘤30例),恶性肿瘤159例(浸润性导管癌146例,原位癌5例、浸润性小叶癌3例、浸润性黏液腺癌2例、大汗腺分化癌2例、恶性叶状肿瘤1例)。

1.2 检查方法

       MRI检查采用美国GE Discovery 750 3.0 T MR扫描仪,选择8通道双侧乳腺专用线圈,患者取俯卧位。扫描序列:轴位T2WI采用脂肪抑制快速自旋回波序列。轴位扩散加权成像(diffusion weighted imaging, DWI)使用脂肪抑制回波平面成像序列,b值取0和1000 s/mm2。增强扫描使用乳腺容积动态增强(volume imaging for breast assessment, VIBRANT)序列,静脉注射钆喷酸葡胺(佳迪显,江苏恒瑞医药股份有限公司,中国),剂量为0.1 mmol/kg,以2.5 mL/s的流率注射,增强前先行蒙片扫描,注射后连续增强扫描522 s,共9个时相,每个动态时相持续58 s,层厚2.8 mm,矩阵为512×512,视野360 mm×360 mm。

1.3 深度学习模型图像预处理

       使用PyRadiomics进行图像预处理,并将所有图像重新采样至1 mm×1 mm×1 mm的体素间距,图像中的像素归一化为0~2900。使用3D-Slicer从DCE-MRI第二期图像手动裁剪整个肿瘤区域及其边界区域,结合肿瘤的3D分割,从原始图像中裁剪所有肿瘤最大截面的2D矩形感兴趣区域(region of interest, ROI)。通过分层随机抽样,将所有患者按7∶3的比例分为训练集212例和测试集91例。

1.4 深度模型开发及模型可解释性

       本研究中使用的深度学习模型包括50层深度残差网络(50-layer deep residual network, ResNet-50)、Inception-V3、Googlenet、密集连接的卷积网络(densely connected convolutional networks, DenseNet)-121、视觉几何组(visual geometry group, VGG)-19和移动神经网络(mobile neural network, MobileNet)-V3,这些学习模型都在ImageNet数据集上进行了预训练以获得初始权重值。训练前所有卷积神经网络(convolutional neural network, CNN)模型ROI图像大小调整为224×224像素。采用随机梯度下降优化器更新模型参数,初始学习率为0.001,通过余弦算法衰减,共1000个周期,4000个迭代步骤,批量大小为64。使用L2正则化防止过拟合,训练的损失值用于评估模型性能。

       为了使模型具有可解释性,使用了梯度加权类激活映射(gradient-weighted class activation mapping,Grad-CAM)对模型进行可视化。利用CNN最后一卷积层的梯度信息进行加权融合,得到突出分类目标图像重要区域的类激活图。

1.5 放射科医师的视觉评估和深度学习模型辅助诊断

       所有MRI图像均由两名放射科医师(分别具有5年和10年乳腺磁共振诊断的主治医师及副主任医师)采用盲法独立阅片,两名医师回顾了患者的临床病史和所有DCE-MRI、DWI及T2WI图像以做出诊断。他们分别进行了两次阅片评估(第一次没有深度学习模型辅助阅片,第二次在深度学习模型辅助下阅片),每次评估间隔1个月,在整个过程中,两名医师对原始诊断报告和最终病理结果未知。

1.6 统计学分析

       使用SPSS 23.0及Python 3.7软件统计学分析;计量资料先使用Kolmogorov-Smirnov检验评价数据分布的正态性,符合正态分布的资料使用均数±标准差表示,检验方法采用独立样本t检验;不符合正态分布的非参数计量资料采用中位数(上四分位数,下四分位数)表示,采用Mann-Whitney U检验分析。计数资料采用频数(构成比)表示,卡方检验及Fisher检验用来验证数据分布情况。绘制受试者工作特征(receive operating characteristic, ROC)曲线,计算AUC、准确度、敏感度、特异度、阴性预测值(negative predictive value, NPV)及阳性预测值(positive predictive value, PPV),评价各模型预测乳腺良恶性病变的效能。使用DeLong检验比较模型间AUC差异,采用配对卡方检验比较两轮阅片的准确度、敏感度和特异度,P<0.05认为差异具有统计学意义。

2 结果

2.1 深度学习模型的性能

       训练损失图中可以看到ResNet-50(蓝色区域)位于图形最下方(图1),代表ResNet-50在训练中损失值最小。训练集ResNet-50与Inception-V3、Googlenet、VGG-19和MobileNet-V3模型之间AUC差异有统计学意义(Z=2.739、5.110、2.728、2.693;P=0.006、P<0.000 1、P=0.006、P=0.007),训练集ResNet-50与DenseNet-121模型之间AUC差异无统计学意义(Z=1.820,P=0.069)。测试集ResNet-50 AUC高于Inception-V3、Googlenet、DenseNet-121、VGG-19和MobileNet-V3深度学习模型(Z=1.488、0.324、1.387、0.236、1.106;P=0.137、0.746、0.166、0.813及0.269),ResNet-50模型准确度、特异度及敏感度分别为80.2%、86.7%及73.9%(表1图2)。

图2  深度学习模型工作特征曲线。2A:训练集;2B:测试集。
Fig. 2  The receiver operating characteristic curves of deep learning models. 2A: Training set; 2B: Validation set.
图1  训练集中不同深度学习模型随迭代步骤变化的损失图。蓝色、橙色、黄色、绿色,淡绿色及红色区域分别代表ResNet-50、DenseNet-121、Googlenet、Inception-V3、MobileNet-V3和VGG-19模型的损失值,ResNet-50模型(蓝色区域)位于图形最下方。
Fig. 1  Loss diagram of different depth learning models in training set with iterative steps. Blue, orange, yellow, green, light green and red area represent the loss values of ResNet-50, DenseNet-121, Googlenet, Inception-V3, MobileNet-V3 and VVG-19 models respectively, ResNet-50 model (blue area) is located at the bottom of the drawing.
表1  训练集及测试集各深度学习模型效能对比
Tab. 1  Comparison of the efficiency of deep learning models in training set and test set

2.2 深度学习模型辅助放射医师的诊断结果

       第一轮阅片,ROC曲线显示,ResNet-50模型的准确度、特异度及敏感度分别为80.2%、86.7%及73.9%,初级医师为73.6%、73.3%及73.9%,高级医师为80.2%、80.0%及80.4%。第二轮阅片,在ResNet-50模型辅助下,初级医师准确度、特异度及敏感度增加15.4%、17.8%、13.1%(χ2=12.07、6.13、4.17;P<0.001、P=0.008、P=0.031),高级医师准确度、特异度及敏感度增加12.1%、13.3%、10.9%(χ2=9.09、4.17、3.20;P=0.001、0.031、0.063)(表2图3)。

图3  深度学习模型与医师两轮阅片工作特征曲线比较,两轮阅片医师准确度、特异度及敏感度变化。3A:测试集ResNet-50、初级医师(junior)、初级医师+ResNet-50、高级医师(senior)、高级医师+ResNet-50各模型的工作特征曲线比较,第一轮阅片,初级医师(粉红色圆点)及高级医师(绿色圆点)数值低于ResNet-50模型工作特征曲线的左上角;第二轮阅片,初级医师+ResNet-50模型(蓝色方形点)及高级医师+ResNet-50模型(橙色三角形点)数值高于ResNet-50 模型工作特征曲线的左上角;3B~3D:初级医师、高级医师有及没有ResNet-50模型时的准确度(3B)、特异度(3C)及敏感度(3D)变化。
Fig. 3  Comparing the deep learning model with the characteristic curve of doctors' two rounds of reading, the accuracy, specificity and sensitivity of doctors in two rounds of film reading changed. 3A: Comparison of the models of ResNet-50, junior, junior +ResNet-50, senior and senior +ResNet-50 in the test set. In the first round of film reading, the values of junior (pink dots) and senior (green dots) are lower than the upper left corner of the working characteristic curve of ResNet-50 model; In the second round of film reading, the values of junior physician +ResNet-50 model (blue square point) and senior physician +ResNet-50 model (orange triangle point) are higher than the upper left corner of the working characteristic curve of ResNet-50 model; 3B-3D: The accuracy (3B), specificity (3C) and sensitivity (3D) of ResNet-50 model have changed between junior doctors and senior doctors.
表2  测试集中ResNet-50 深度学习模型、初级和高级放射科医师的诊断效能
Tab. 2  Test set ResNet-50 deep learning model, diagnostic efficiency of junior and senior radiologists

2.3 Grad-CAM模型可视化结果

       三例乳腺恶性肿瘤类激活图显示病灶中央区呈激活改变(图4),三例乳腺良性肿瘤类激活图显示病灶周边呈激活改变(图5)。典型病例示放射医师根据深度学习模型概率值及类激活图辅助下做出正确的诊断(图6)。

图4  乳腺恶性肿瘤的原始感兴趣区(ROI)图、肿瘤分割图、最大截面矩形ROI图及ResNet-50模型类激活图。4A~4D 女,57岁,右乳腺浸润性导管癌。4A:原始ROI图;4B:红色区域为肿瘤分割图;4C:最大截面矩形ROI图;4D:类激活图显示病灶中央区呈激活改变,突出显示的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)。4E~4H,女,52岁,左乳腺浸润性导管癌。4E:原始ROI图;4F:红色区域为肿瘤分割图;4G:最大截面矩形ROI图;4H:类激活图显示病灶中央区呈激活改变,突出显示的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)。图4I~4L,女,55岁,左乳腺浸润性导管癌。4I:原始ROI图;4J:红色区域为肿瘤分割图;4K:最大截面矩形ROI图;4L:类激活图显示病灶中央区呈激活改变,突出显示的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)。
Fig. 4  Original region of interest (ROI) map, tumor segmentation map, maximum cross-sectional rectangular ROI map, and ResNet-50 model class activation map of breast malignant tumors. 4A-4D: A 57 years old female with right invasive ductal carcinoma of the breast. 4A: Original ROI map; 4B: Red area shows tumor segmentation map; 4C: Maximum cross-sectional rectangular ROI map; 4D: Class activation map shows activation changes in the central area of the lesion, with the highlighted central area mainly consisting of low enhancement areas (blue), surrounded by adjacent high enhancement areas (light yellow and red). 4E-4H: A 52 years old female with left invasive ductal carcinoma of the breast. 4E: Original ROI map; 4F: Red area represents tumor segmentation map; 4G: Maximum cross-sectional rectangular ROI map; 4H: Class activation map shows activation changes in the central area of the lesion, with the highlighted central area mainly consisting of low enhancement areas (blue), surrounded by adjacent high enhancement areas (light yellow and red). 4I-4L: A 55 years old female with left invasive ductal carcinoma of the breast. 4I: Original ROI map; 4J: Red area represents tumor segmentation map; 4K: Maximum cross-sectional rectangular ROI map; 4L: Class activation map shows activation changes in the central area of the lesion, with low enhancement areas (blue), surrounded by adjacent high enhancement areas (light yellow and red).
图5  乳腺良性肿瘤的原始感兴趣区(ROI)、肿瘤分割图、最大截面矩形ROI及ResNet-50模型类激活图。5A~5D:女,47岁,右乳腺纤维腺瘤。5A:原始ROI图;5B:红色区域为肿瘤分割图;5C:最大截面矩形ROI图;5D:类激活图显示病灶周围处呈激活改变,激活的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)。5E~5H,女,40岁,左乳腺良性叶状肿瘤。5E:原始感兴趣区域图;5F:红色区域为肿瘤分割图;5G:最大截面矩形ROI图;5H:类激活图显示病灶周围处呈激活改变,激活的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)。5I~5L 女,43岁,左乳腺纤维腺瘤。5I:原始感兴趣区域图;5J:红色区域为肿瘤分割图;5K:最大截面矩形ROI图;5L:类激活图显示病灶周围处呈激活改变,激活的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)。
Fig. 5  Original region of interest (ROI) map, tumor segmentation map, maximum cross-sectional rectangular ROI map, and ResNet-50 model class activation map of benign breast tumors. 5A-5D: Female, 47 years old, with right fibroadenoma of the breast. 5A: Original ROI map; 5B: Red area shows tumor segmentation map; 5C: Maximum cross-sectional rectangular ROI map; 5D: Class activation map shows activation changes around the lesion, with low enhancement areas (blue) as the main central area of activation, and adjacent high enhancement areas (light yellow and red) around it. 5E-5H: Female, 40 years old, with left benign lobular breast tumor; 5E: Original ROI map; 5F: Red area shows tumor segmentation map; 5G: Maximum cross-sectional rectangular ROI map; 5H: Class activation map shows activation changes around the lesion, with low enhancement areas (blue) as the main central area of activation, and adjacent high enhancement areas (light yellow and red) around it. 5I-5L: Female, 43 years old, with left fibroadenoma of the breast; 5I: Original ROI map; 5J: Red area shows tumor segmentation map; 5K: Maximum cross-sectional rectangular ROI map; 5L: Class activation map shows activation changes around the lesion, with low enhancement areas (blue) as the main central area of activation, and adjacent high enhancement areas (light yellow and red) around it.
图6  ResNet-50模型辅助放射科医师做出诊断的典型病例。6A:女,38岁,病理诊断为右乳腺浸润性导管癌。初级及高级医师第一轮评估时诊断乳腺纤维腺瘤,通过深度学习(DL)模型信息[良性概率值为0.040,恶性概率值为0.960,类激活图位于肿瘤中心,激活的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)],第二轮评估均诊断乳腺癌。6B:女,50岁,病理诊断为左乳导管内乳头状瘤。初级及高级医师第一轮评估时诊断为乳腺癌及乳腺良性肿瘤,通过深度学习模型信息[良性概率值为0.885,恶性概率值为0.115,类激活图位于肿瘤周围,激活的中央区域以低增强区域为主(蓝色),周围有相邻的高增强区域(淡黄色及红色)],第二轮评估均诊断良性肿瘤。
Fig. 6  ResNet-50 model assists radiologists in diagnosing typical cases. 6A: A 38 years old female with pathological diagnosis of infiltrating ductal carcinoma of the right breast. Primary and senior physicians diagnosed breast fibroadenoma in the first round of evaluation. Through deep learning (DL) model information [benign probability value is 0.040, malignant probability value is 0.960, quasi activation map is located in the center of the tumor, the activated central area is mainly low enhancement area (blue), and adjacent high enhancement area (light yellow and red)], breast cancer is diagnosed in the second round of evaluation. 6B: A 50 years old female with pathological diagnosis of left ductal papillary carcinoma. Primary and senior physicians diagnosed breast cancer and benign breast tumors in the first round of evaluation. Through deep learning model information [benign probability value is 0.885, malignant probability value is 0.115, class activation map is located around the tumor, the activated central area is dominated by low enhancement areas (blue), and adjacent high enhancement areas (light yellow and red)], benign tumors are diagnosed in the second round of evaluation.

3 讨论

       本研究探讨基于DCE-MRI图像深度学习模型鉴别诊断乳腺良恶性肿瘤的价值,其中ResNet-50模型性能最佳。借助ResNet-50模型放射科医师诊断的准确性明显提高,可视化图像可能成为影像诊断依据,为临床决策提供客观依据。

3.1 深度学习模型在乳腺肿瘤的应用

       与传统的机器学习相比,深度学习可以自动学习神经网络隐藏层的语义和空间特征[14]。与需要耗时数据标记的传统机器算法不同,深度学习算法更加灵活和适应性强,因为它们可以从未标记或结构化的数据中学习[15]。关于深度学习在乳腺癌方面已经有一些研究。PARK等[16]所提出的弱注释深度学习模型在使用DCE-MRI对乳腺癌进行3D分割方面表现出良好的性能。GUO等[17]研究表明深度学习放射组学模型有效区分了乳腺癌患者的HER-2状态,尤其是HER-2低阳性状态。JING等[18]提出CNN可用于在高敏感度的MRI筛查中自动定位乳腺病变。CHEN等[19]研究显示基于DCE-MRI图像的CNN模型对于前哨淋巴结转移在内部验证集、外部测试集1及2取得了令人鼓舞的预测性能(AUC=0.89、0.885、0.768),帮助外科医师对乳腺癌患者进行个性化无创腋窝治疗。HUANG等[20]基于纵向MRI的影像组学和深度学习融合新模型可以精准预测新辅助化疗乳腺癌的病理完全缓解。上述研究表明深度学习模型在乳腺癌应用方面有良好的预测性能,我们研究的一个创新点是六种深度学习模型可以准确鉴别乳腺良恶性肿瘤,测试集AUC为0.746~0.841,准确度为71.4%~81.3%。

3.2 ResNet-50深度学习模型诊断乳腺良恶性肿瘤效能评估

       ResNet-50作为训练效率均衡和性能的深度网络,是计算机视觉研究中最常使用的深度网络[21, 22]。LIU等[23]研究基于乳腺MRI的弱监督深度学习可以很好地区分恶性和良性病变,CNN模型的AUC为0.92,准确度为94%,敏感度为74%,特异度为95%。WU等[24]研究CNN模型融合肿瘤几何信息的分析特征和肿瘤组织的药代动力学特性,用于区分71名恶性肿瘤和59名良性肿瘤,CNN模型实现了87.7%的总体诊断准确率、91.2%的准确率、86.1%的敏感度。ZHOU等[25]研究表明使用ResNet-50评估了133个病变(91个恶性病变和62个良性病变),与基于ROI和影像组学的模型相比,深度学习在区分良性和恶性病变方面具有更高的准确性。而ZHOU等[26]研究表明在鉴别非肿块强化的良恶性病变中,基于DCE-MRI的ResNet-50深度学习模型较5种不同的机器学习模型具有更好的诊断性能,准确度训练集为91.5%,测试集为83.3%。本研究与上述研究结果一致,ResNet-50模型在训练中损失值最小,这意味着它在训练过程中学习到的错误更少,并且比任何其他CNN表现出更快的收敛速度。训练集ResNet-50模型与大部分模型之间AUC差异有统计学意义,同时其测试集实现了最高的AUC(0.841)及PPV(85%)。ResNet-50模型测试集准确度为80.2%,高于Inception-V3、Googlenet,DenseNet-121和MobileNet-V3模型,因此本研究中ResNet-50为性能最佳模型,比其他深度学习模型具有更强的泛化和预测能力[27]。MAO等[28]基于对比增强光谱乳腺X线摄影的DenseNet 121、Xception和ResNet 50三种CNN作为骨干架构,区分乳腺良性和恶性乳腺病变,性能最好的模型为Xception,外部测试集AUC为0.970,敏感度为84.8%,特异度为100.0%,准确度为0.891,高于其他CNN、影像组学模型和放射科医生,其与本研究结果不同,分析原因可能是MAO等的研究基于对比增强光谱乳腺X线摄影,而不是基于MRI的深度学习模型鉴别乳腺良恶性病变,其次MAO等的研究中模型是在多中心数据上进行了测试,相对来说具有很强的泛化能力。

3.3 ResNet-50深度学习模型辅助医师诊断乳腺良恶性肿瘤效能评估

       本研究的另外一个创新点是初级和高级放射科医师进行了两轮阅片,分析鉴别诊断乳腺良恶性肿瘤的效能。深度学习模型可以进一步学习和使用人类无法识别的高级抽象特征来识别病变,从而超越放射医师的诊断性能[29, 30]。与上述研究一致,第一轮阅片,ROC曲线显示初级医师及高级医师数值低于ResNet-50模型ROC曲线的左上角,深度学习模型准确度及特异度为80.2%,86.7%,高于初级医师(73.6%,73.3%),准确度相当于高级医师(80.2%),特异度高于高级医师(80.0%),ResNet-50模型诊断效能优于放射医师阅片。然而深度学习特征的可解释性仍处于起步阶段[31, 32],这些影像组学特征背后的生物学机制仍未得到充分探索。在本研究中,深度学习模型概率值会告知放射科医师定量计算机分析结果,同时类激活图可以引导他们的注意力集中在DCE-MRI图像中的突出显示区域上,以便他们更有效地重新评估图像。第二轮阅片,在ResNet-50模型辅助下,初级医师准确度、特异度及敏感度增加15.4%、17.8%及13.1%,高级医师准确度、特异度及敏感度增加12.1%、13.3%及10.9%,初级医师两轮阅片的准确度、特异度及敏感度差异有统计学意义,高级医师两轮阅片的准确度及特异度差异有统计学意义,放射科医师在ResNet-50模型辅助下诊断良恶性肿瘤实现了更高的诊断性能。

3.4 ResNet-50模型诊断乳腺良恶性肿瘤的特征可视化

       与良性肿瘤相比,恶性肿瘤具有更不规则的形状、模糊的边界和复杂的内部肿瘤异质性[33]。既往的研究表明,模型可视化的Grad-CAM生成的类激活图在良性肿瘤和恶性肿瘤图像中具有不同的模式[34, 35],本研究ResNet-50深度学习模型的Grad-CAM生成的类激活图可视化结果为恶性肿瘤中央区呈激活改变,良性肿瘤周边呈激活改变。

3.5 本研究的局限性

       本研究具有以下局限性:(1)单中心回顾性研究,303例乳腺良性与恶性肿瘤的数据来自同一机构,可能会导致结果过拟合,为了弥补这一局限性,在后续的研究中,应增加外部多中心验证集病例;(2)本研究基于 DEC-MRI单一序列进行建模,可能会对结果造成一定的影响,在后续的研究中,多模态序列应纳入研究。

4 结论

       综上所述,ResNet-50深度学习模型在区分乳腺良恶性肿瘤方面表现出最优异的性能。放射科医师在ResNet-50深度学习模型概率值及类激活图辅助下实现了更高的诊断性能,有望为乳腺肿瘤患者的临床决策提供更有价值的信息。

[1]
SIEGEL R L, GIAQUINTO A N, JEMAL A. Cancer statistics, 2024[J]. CA Cancer J Clin, 2024, 74(1): 12-49. DOI: 10.3322/caac.21820.
[2]
MANN R M, KUHL C K, MOY L. Contrast-enhanced MRI for breast cancer screening[J]. J Magn Reson Imaging, 2019, 50(2): 377-390. DOI: 10.1002/jmri.26654.
[3]
HUSSEIN H, ABBAS E, KESHAVARZI S, et al. Supplemental breast cancer screening in women with dense breasts and negative mammography: a systematic review and meta-analysis[J/OL]. Radiology, 2023, 306(3): e221785 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/36719288/. DOI: 10.1148/radiol.221785.
[4]
周涵, 唐婉, 李志恒, 等. 基于多参数MRI联合临床特征构建的列线图模型鉴别BI-RADS 4类病灶良性与恶性的价值[J]. 中华放射学杂志, 2024, 58(4): 388-393. DOI: 10.3760/cma.j.cn12149-20231115-00390.
ZHOU H, TANG W, LI Z H, et al. Nomogram model based on multiparametric MRI combined with clinical features in identifying benign and malignant BI-RADS 4 lesions[J]. Chin J Radiol, 2024, 58(4): 388-393. DOI: 10.3760/cma.j.cn12149-20231115-00390.
[5]
PARK A R, CHAE E Y, CHA J H, et al. Preoperative breast MRI in women 35 years of age and younger with breast cancer: benefits in surgical outcomes by using propensity score analysis[J]. Radiology, 2021, 300(1): 39-45. DOI: 10.1148/radiol.2021204124.
[6]
CAO Y, HUANG Y, CHEN X L, et al. Optimizing ultrafast dynamic contrast-enhanced MRI scan duration in the differentiation of benign and malignant breast lesions[J/OL]. Insights Imaging, 2024, 15(1): 112 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/38713334/. DOI: 10.1186/s13244-024-01697-6.
[7]
刘鹏, 于晓晶, 李春志, 等. 基于动态增强磁共振成像的影像组学模型对肉芽肿性乳腺炎与乳腺癌的鉴别诊断价值[J]. 中国医学影像学杂志, 2024, 32(2): 144-149. DOI: 10.3969/j.issn.1005-5185.2024.02.007.
LIU P, YU X J, LI C Z, et al. Differential diagnosis of dynamic contrast-enhanced-MRI-based radiomics model for granulomatous mastitis and breast cancer[J]. Chin J Med Imag, 2024, 32(2): 144-149. DOI: 10.3969/j.issn.1005-5185.2024.02.007.
[8]
韩珺琪, 华辉, 王晓琳, 等. 基于MRI的影像组学模型在鉴别乳腺导管原位癌与导管原位癌伴微浸润中的价值[J]. 临床放射学杂志, 2024, 43(2): 196-203. DOI: 10.1007/s00432-021-03606-6.
HAN J Q, HUA H, WANG X L, et al. Radiomics nomogram for diagnosis of DCIS and DCIS with microinvasion on MRI[J]. J Clin Radiol, 2024, 43(2): 196-203. DOI: 10.1007/s00432-021-03606-6.
[9]
RICCIARDI R, METTIVIER G, STAFFA M, et al. A deep learning classifier for digital breast tomosynthesis[J/OL]. Phys Med, 2021, 83: 184-193 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/33798904/. DOI: 10.1016/j.ejmp.2021.03.021.
[10]
HUANG G L, DU S Y, GAO S, et al. Molecular subtypes of breast cancer identified by dynamically enhanced MRI radiomics: the delayed phase cannot be ignored[J/OL]. Insights Imaging, 2024, 15(1): 127 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/38816553/. DOI: 10.1186/s13244-024-01713-9.
[11]
YANG X Q, FAN X H, LIN S Y, et al. Assessment of lymphovascular invasion in breast cancer using a combined MRI morphological features, radiomics, and deep learning approach based on dynamic contrast-enhanced MRI[J]. J Magn Reson Imaging, 2024, 59(6): 2238-2249. DOI: 10.1002/jmri.29060.
[12]
岳文怡, 张洪涛, 高珅, 等. 基于机器学习的组合模型在预测乳腺癌新辅助化疗疗效中的价值[J]. 磁共振成像, 2024, 15(3): 93-99, 106. DOI: 10.12015/issn.1674-8034.2024.03.016.
YUE W Y, ZHANG H T, GAO S, et al. Radiomics based on combined machine learning models for prediction of the response to neoadjuvant chemotherapy in mass enhancement breast cancer using magnetic resonance imaging[J]. Chin J Magn Reson Imag, 2024, 15(3): 93-99, 106. DOI: 10.12015/issn.1674-8034.2024.03.016.
[13]
LI X, YANG L F, JIAO X. Comparison of traditional radiomics, deep learning radiomics and fusion methods for axillary lymph node metastasis prediction in breast cancer[J]. Acad Radiol, 2023, 30(7): 1281-1287. DOI: 10.1016/j.acra.2022.10.015.
[14]
ZHENG X Y, YAO Z, HUANG Y N, et al. Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer[J/OL]. Nat Commun, 2020, 11(1): 1236 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/32144248/. DOI: 10.1038/s41467-020-15027-z.
[15]
KOH D M, PAPANIKOLAOU N, BICK U, et al. Artificial intelligence and machine learning in cancer imaging[J/OL]. Commun Med, 2022, 2: 133 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/36310650/. DOI: 10.1038/s43856-022-00199-0.
[16]
PARK G E, KIM S H, NAM Y, et al. 3D breast cancer segmentation in DCE-MRI using deep learning with weak annotation[J]. J Magn Reson Imaging, 2024, 59(6): 2252-2262. DOI: 10.1002/jmri.28960.
[17]
GUO Y, XIE X T, TANG W J, et al. Noninvasive identification of HER2-low-positive status by MRI-based deep learning radiomics predicts the disease-free survival of patients with breast cancer[J]. Eur Radiol, 2024, 34(2): 899-913. DOI: 10.1007/s00330-023-09990-6.
[18]
JING X P, DORRIUS M D, ZHENG S Y, et al. Localization of contrast-enhanced breast lesions in ultrafast screening MRI using deep convolutional neural networks[J]. Eur Radiol, 2024, 34(3): 2084-2092. DOI: 10.1007/s00330-023-10184-3.
[19]
CHEN M Z, KONG C L, LIN G H, et al. Development and validation of convolutional neural network-based model to predict the risk of sentinel or non-sentinel lymph node metastasis in patients with breast cancer: a machine learning study[J/OL]. EClinicalMedicine, 2023, 63: 102176 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/37662514/. DOI: 10.1016/j.eclinm.2023.102176.
[20]
HUANG Y H, ZHU T, ZHANG X L, et al. Longitudinal MRI-based fusion novel model predicts pathological complete response in breast cancer treated with neoadjuvant chemotherapy: a multicenter, retrospective study[J/OL]. EClinicalMedicine, 2023, 58: 101899 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/37007742/. DOI: 10.1016/j.eclinm.2023.101899.
[21]
SENG H S, ARGHA A, LIAW S T, et al. Machine and deep learning for tuberculosis detection on chest X-rays: systematic literature review[J/OL]. J Med Internet Res, 2023, 25: e43154 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/37399055/. DOI: 10.2196/43154.
[22]
黄翔, 曹康养, 邹玉坚, 等. 基于 T2WI 的 ResNet 深度学习模型术前预测膀胱癌病理分级的研究[J]. 磁共振成像, 2024, 15(1): 125-131. DOI: 10.12015/issn.1674-8034.2024.01.020
HUANG X, CAO K Y, ZOU Y J, et al. Preoperative prediction of pathological grading in bladder cancer by deep residual network model based on T2WI[J]. Chin J Magn Reson Imag, 2024, 15(1): 125-131. DOI: 10.12015/issn.1674-8034.2024.01.020.
[23]
LIU M Z, SWINTELSKI C, SUN S, et al. Weakly supervised deep learning approach to breast MRI assessment[J/OL]. Acad Radiol, 2022, 29(Suppl 1): S166-S172 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/34108114/. DOI: 10.1016/j.acra.2021.03.032.
[24]
WU Y N, WU J, DOU Y T, et al. A deep learning fusion model with evidence-based confidence level analysis for differentiation of malignant and benign breast tumors using dynamic contrast enhanced MRI[J/OL]. Biomed Signal Process Contr, 2022, 72: 103319 [2024-05-27]. https://www.sciencedirect.com/science/article/abs/pii/S1746809421009162. DOI: 10.1016/j.bspc.2021.103319.
[25]
ZHOU J J, ZHANG Y, CHANG K T, et al. Diagnosis of benign and malignant breast lesions on DCE-MRI by using radiomics and deep learning with consideration of peritumor tissue[J]. J Magn Reson Imaging, 2020, 51(3): 798-809. DOI: 10.1002/jmri.26981.
[26]
ZHOU J J, LIU Y L, ZHANG Y, et al. BI-RADS reading of non-mass lesions on DCE-MRI and differential diagnosis performed by radiomics and deep learning[J/OL]. Front Oncol, 2021, 11: 728224 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/34790569/. DOI: 10.3389/fonc.2021.728224.
[27]
YU Q, NING Y Q, WANG A R, et al. Deep learning-assisted diagnosis of benign and malignant parotid tumors based on contrast-enhanced CT: a multicenter study[J]. Eur Radiol, 2023, 33(9): 6054-6065. DOI: 10.1007/s00330-023-09568-2.
[28]
MAO N, ZHANG H C, DAI Y, et al. Attention-based deep learning for breast lesions classification on contrast enhanced spectral mammography: a multicentre study[J]. Br J Cancer, 2023, 128(5): 793-804. DOI: 10.1038/s41416-022-02092-y.
[29]
QIAN X J, PEI J, ZHENG H, et al. Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning[J]. Nat Biomed Eng, 2021, 5(6): 522-532. DOI: 10.1038/s41551-021-00711-2.
[30]
CHEN C, JIANG Y T, YAO J C, et al. Deep learning to assist composition classification and thyroid solid nodule diagnosis: a multicenter diagnostic study[J]. Eur Radiol, 2024, 34(4): 2323-2333. DOI: 10.1007/s00330-023-10269-z.
[31]
HOLZINGER A, LANGS G, DENK H, et al. Causability and explainability of artificial intelligence in medicine[J/OL]. Wiley Interdiscip Rev Data Min Knowl Discov, 2019, 9(4): e1312 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/32089788/. DOI: 10.1002/widm.1312.
[32]
JIN W N, FATEHI M, GUO R, et al. Evaluating the clinical utility of artificial intelligence assistance and its explanation on the glioma grading task[J/OL]. Artif Intell Med, 2024, 148: 102751 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/38325929/. DOI: 10.1016/j.artmed.2023.102751.
[33]
MOHAMED MUSTHAFA M, T R M, VINOTH KUMAR V, et al. Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50[J/OL]. BMC Med Imaging, 2024, 24(1): 107 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/38734629/. DOI: 10.1186/s12880-024-01292-7.
[34]
ZHANG H J, OGASAWARA K. Grad-CAM-based explainable artificial intelligence related to medical text processing[J/OL]. Bioengineering, 2023, 10(9): 1070 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/37760173/. DOI: 10.3390/bioengineering10091070.
[35]
TONG T, GU J H, XU D, et al. Deep learning radiomics based oncontrast-enhanced ultrasound images for assisted diagnosis of pancreatic ductal adenocarcinoma and chronic pancreatitis[J/OL]. BMC Med, 2022, 20(1): 74 [2024-05-27]. https://pubmed.ncbi.nlm.nih.gov/35232446/. DOI: 10.1186/s12916-022-02258-8.

上一篇 基于深度学习重建的扩散加权成像在肺部病变良恶性鉴别中的应用价值
下一篇 基于深度学习重建和传统TSE序列在直肠癌磁共振检查的对比研究
  
诚聘英才 | 广告合作 | 免责声明 | 版权声明
联系电话:010-67113815
京ICP备19028836号-2