Share：
Share this content in WeChat
X
[Chinese][PDF]6424
• Review •
Research progress of deep learning brain tumor MRI image classification
ZHANG Heng  ZHANG Sai  SUN Jiawei  LU Zhengda  NI Xinye 

Cite this article as: ZHANG H, ZHANG S, SUN J W, et al. Research progress of deep learning brain tumor MRI image classification[J]. Chin J Magn Reson Imaging, 2023, 14(1): 166-171, 193. DOI:10.12015/issn.1674-8034.2023.01.031.


[Abstract] In the big data environment, tumor cases provide huge data resources for the clinical diagnosis of tumors. Meanwhile, the development of artificial intelligence technology promotes the continuous improvement of the application level of deep learning and promotes the rapid and accurate classification of tumor images in the era of deep learning. This paper is mainly divided into the following four parts. The first part reviews the current mainstream deep learning MRI image classification models: convolutional neural network, deep belief network, deep residual network, and Vision Transformer. Firstly, the historical lineage, the initial problems, and the main ideas of each model are described. Secondly, the network architecture of the model is summarized and its latest application in MRI image classification is discussed. Then, the characteristics, limitations, and development trends of the models are analyzed. The second part discusses some key factors that affect classification performance. In the third part, some widely used performance enhancement techniques are proposed. Finally, the main limitations of deep learning classification of MRI images in clinical practice are discussed, and future research directions have been prospected. The results presented here can provide researchers with a comprehensive comparison as well as the effectiveness of various deep learning models, which is expected to promote the progress of brain tumor research.
[Keywords] deep learning；brain tumor；image classification；magnetic resonance imaging；artificial intelligence；neural networks

ZHANG Heng1, 2, 3, 4   ZHANG Sai1, 2, 3, 4   SUN Jiawei1, 2, 3, 4   LU Zhengda1, 2, 3, 4   NI Xinye1, 2, 3, 4*  

1 Department of Radiotherapy, Changzhou Second Peope's Hospital, Nanjing Medical University, Changzhou 213003, China

2 Jiangsu Province Engineering Research Center of Medical Physics, Changzhou 213003, China

3 Medical Physics Research Center, Nanjing Medical University, Changzhou 213003, China

4 Key Laboratory of Medical Physics of Changzhou, Changzhou 213003, China

Corresponding author: Ni XY, E-mail: nxy@njmu.edu.cn

Conflicts of interest   None.

ACKNOWLEDGMENTS Jiangsu Provincial Key Research and Development Program Social Development Project (No. BE2022720); General Program of Jiangsu Provincial Health Commission (No. M2020006).
Received  2022-08-16
Accepted  2022-11-29
DOI: 10.12015/issn.1674-8034.2023.01.031
Cite this article as: ZHANG H, ZHANG S, SUN J W, et al. Research progress of deep learning brain tumor MRI image classification[J]. Chin J Magn Reson Imaging, 2023, 14(1): 166-171, 193. DOI:10.12015/issn.1674-8034.2023.01.031.

[1]
SIEGEL R L, MILLER K D, JEMAL A. Cancer statistics, 2016[J]. CA A Cancer J Clin, 2016, 66(1): 7-30. DOI: 10.3322/caac.21332.
[2]
BIRATU E S, SCHWENKER F, AYANO Y M, et al. A survey of brain tumor segmentation and classification algorithms[J/OL]. J Imaging, 2021, 7(9): 179 [2022-08-15]. https://doi.org/10.3390/jimaging7090179. DOI: 10.3390/jimaging7090179.
[3]
NAZIR M, SHAKIL S, KHURSHID K. Role of deep learning in brain tumor detection and classification (2015 to 2020): a review[J/OL]. Comput Med Imaging Graph, 2021, 91: 101940 [2022-08-15]. https://doi.org/10.1016/j.compmedimag.2021.101940. DOI: 10.1016/j.compmedimag.2021.101940.
[4]
SAJID S, HUSSAIN S, SARWAR A. Brain tumor detection and segmentation in MR images using deep learning[J]. Arab J Sci Eng, 2019, 44(11): 9249-9261. DOI: 10.1007/s13369-019-03967-8.
[5]
CHAN H P, SAMALA R K, HADJIISKI L M, et al. Deep learning in medical image analysis[J/OL]. Adv Exp Med Biol, 2020, 1213: 3-21 [2022-08-15]. https:/doi.org/10.1007/978-3-030-33128-3_1. DOI: 10.1007/978-3-030-33128-3_1.
[6]
ZHENG F, CHEN X Z. Research progress of deep learning in glioblastoma[J]. Chin J Magn Reson Imaging, 2022, 13(3): 115-117. DOI: 10.12015/issn.1674-8034.2022.03.028.
[7]
HUBEL D H, WIESEL T N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex[J]. J Physiol, 1962, 160(1): 106-154. DOI: 10.1113/jphysiol.1962.sp006837.
[8]
LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444. DOI: 10.1038/nature14539.
[9]
BERNAL J, KUSHIBAR K, ASFAW D S, et al. Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review[J]. Artif Intell Med, 2019, 95: 64-81. DOI: 10.1016/j.artmed.2018.08.008.
[10]
KHAN H A, JUE W, MUSHTAQ M, et al. Brain tumor classification in MRI image using convolutional neural network[J]. Math Biosci Eng, 2020, 17(5): 6203-6216. DOI: 10.3934/mbe.2020328.
[11]
ANILKUMAR B. Tumor Classification using Block wise fine tuning and Transfer learning of Deep Neural Network and KNN classifier on MR Brain Images[J]. Int J Emerg Trends Eng Res, 2020, 8(2): 574-583. DOI: 10.30534/ijeter/2020/48822020.
[12]
AYADI W, ELHAMZI W, CHARFI I, et al. Deep CNN for brain tumor classification[J]. Neural Process Lett, 2021, 53(1): 671-700. DOI: 10.1007/s11063-020-10398-2.
[13]
KADER I ABD EL, XU G, SHUAI Z, et al. Differential deep convolutional neural network model for brain tumor classification[J/OL]. Brain Sci, 2021, 11(3): 352 [2022-08-15]. https://doi.org/10.3390/brainsci11030352. DOI: 10.3390/brainsci11030352.
[14]
AMIN J, SHARIF M, GUL N, et al. Brain tumor classification based on DWT fusion of MRI sequences using convolutional neural network[J]. Pattern Recognit Lett, 2020, 129: 115-122. DOI: 10.1016/j.patrec.2019.11.016.
[15]
HAN X F, LAGA H, BENNAMOUN M. Image-based 3D object reconstruction: state-of-the-art and trends in the deep learning era[J]. IEEE Trans Pattern Anal Mach Intell, 2021, 43(5): 1578-1604. DOI: 10.1109/TPAMI.2019.2954885.
[16]
RUSSO C, LIU S D, DI IEVA A. Spherical coordinates transformation pre-processing in Deep Convolution Neural Networks for brain tumor segmentation in MRI[J]. Med Biol Eng Comput, 2022, 60(1): 121-134. DOI: 10.1007/s11517-021-02464-1.
[17]
ANDRESEN J, KEPP T, EHRHARDT J, et al. Deep learning-based simultaneous registration and unsupervised non-correspondence segmentation of medical images with pathologies[J]. Int J Comput Assist Radiol Surg, 2022, 17(4): 699-710. DOI: 10.1007/s11548-022-02577-4.
[18]
HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507. DOI: 10.1126/science.1127647.
[19]
HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Comput, 2006, 18(7): 1527-1554. DOI: 10.1162/neco.2006.18.7.1527.
[20]
REDDY A V N, KRISHNA C P, MALLICK P K, et al. Analyzing MRI scans to detect glioblastoma tumor using hybrid deep belief networks[J/OL]. J Big Data, 2020, 7: 35 [2022-08-15]. https://link.springer.com/article/10.1186/s40537-020-00311-y. DOI: 10.1186/s40537-020-00311-y.
[21]
SATHIES KUMAR T, ARUN C, EZHUMALAI P. An approach for brain tumor detection using optimal feature selection and optimized deep belief network[J/OL]. Biomed Signal Process Control, 2022, 73: 103440 [2022-08-15]. https://doi.org/10.1016/j.bspc.2021.103440. DOI: 10.1016/j.bspc.2021.103440.
[22]
SANTHOSH KUMAR H S, KARIBASAPPA K. An effective hybrid deep learning with adaptive search and rescue for brain tumor detection[J]. Multimed Tools Appl, 2022, 81(13): 17669-17701. DOI: 10.1007/s11042-022-12474-2.
[23]
ZENG N Y, LI H, PENG Y H. A new deep belief network-based multi-task learning for diagnosis of Alzheimer's disease[J/OL]. Neural Comput & Applic, 2021: 1-12 [2022-08-15]. https://doi.org/10.1007/s00521-021-06149-6. DOI: 10.1007/s00521-021-06149-6.
[24]
HASSANZADEH H R, WANG M D. An integrated deep network for cancer survival prediction using omics data[J/OL]. Front Big Data, 2021, 4: 568352 [2022-08-15]. https://doi.org/10.3389/fdata.2021.568352. DOI: 10.3389/fdata.2021.568352.
[25]
KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Commun ACM, 2012, 60: 84-90. DOI: 10.1145/3065386.
[26]
ANWAR S M, MAJID M, QAYYUM A, et al. Medical image analysis using convolutional neural networks: a review[J/OL]. J Med Syst, 2018, 42(11): 226 [2022-08-15]. https://doi.org/10.1007/s10916-018-1088-1. DOI: 10.1007/s10916-018-1088-1.
[27]
CUI H, YUAN G G, LIU N, et al. Convolutional neural network for recognizing highway traffic congestion[J]. J Intell Transp Syst, 2020, 24(3): 279-289. DOI: 10.1080/15472450.2020.1742121.
[28]
HE K M, ZHANG X Y, REN S Q, et al. Identity mappings in deep residual networks[EB/OL]. [2022-08-15]. https://link.springer.com/chapter/10.1007/978-3-319-46493-0_38.
[29]
HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016, 770-778. DOI: 10.1109/CVPR.2016.90.
[30]
ÇINAR A, YILDIRIM M. Detection of tumors on brain MRI images using the hybrid convolutional neural network architecture[J/OL]. Med Hypotheses, 2020, 139: 109684 [2022-08-15]. https://doi.org/10.1016/j.mehy.2020.109684. DOI: 10.1016/j.mehy.2020.109684.
[31]
ABDELAZIZ ISMAEL S A, MOHAMMED A, HEFNY H. An enhanced deep learning approach for brain cancer MRI images classification using residual networks[J/OL]. Artif Intell Med, 2020, 102: 101779 [2022-08-15]. https:doi.org/10.1016/j.artmed.2019.101779. DOI: 10.1016/j.artmed.2019.101779.
[32]
KAKARLA J, VENKATESWARLU ISUNURI B, et al. Multi-class brain tumor classification using residual network and global average pooling[J]. Multimed Tools Appl, 2021, 80(9): 13429-13438. DOI: 10.1007/s11042-020-10335-4.020-10335-4.
[33]
KABIR ANARAKI A, AYATI M, KAZEMI F. Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms[J]. Biocybern Biomed Eng, 2019, 39(1): 63-74. DOI: 10.1016/j.bbe.2018.10.004.
[34]
SWATI Z N K, ZHAO Q H, KABIR M, et al. Brain tumor classification for MR images using transfer learning and fine-tuning[J/OL]. Comput Med Imaging Graph, 2019, 75: 34-46 [2022-08-15]. https://doi.org/10.1016.j.compmedimag.2019.05.001. DOI: 10.1016/j.compmedimag.2019.05.001.
[35]
SHIN I, KIM H, AHN S S, et al. Development and validation of a deep learning-based model to distinguish glioblastoma from solitary brain metastasis using conventional MR images[J]. AJNR Am J Neuroradiol, 2021, 42(5): 838-844. DOI: 10.3174/ajnr.A7003.
[36]
ZEINELDIN R A, KARAR M E, COBURGER J, et al. DeepSeg: deep neural network framework for automatic brain tumor segmentation using magnetic resonance FLAIR images[J]. Int J Comput Assist Radiol Surg, 2020, 15(6): 909-920. DOI: 10.1007/s11548-020-02186-z.
[37]
LIU W T, LU X M. Research progress of transformer based on computer vision[J]. Comput Eng Appl, 2022, 58(6): 1-16. DOI: 10.3778/j.issn.1002-8331.2106-0442.
[38]
DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. 2020: arXiv: 2010.11929. https://arxiv.org/abs/2010.11929
[39]
ZHANG J J, ZHOU L P, WANG L, et al. Diffusion kernel attention network for brain disorder classification[J]. IEEE Trans Med Imaging, 2022, 41(10): 2814-2827. DOI: 10.1109/TMI.2022.3170701.
[40]
CHEN L Y, YOU Z Y, ZHANG N, et al. UTRAD: anomaly detection and localization with U-transformer[J/OL]. Neural Netw, 2022, 147: 53-62 [2022-08-15]. https://doi.org/10.1016/j.neunet.2021.12.008. DOI: 10.1016/j.neunet.2021.12.008.
[41]
DAI Y, GAO Y, LIU F. TransMed: transformers advance multi-modal medical image classification[J/OL]. Diagnostics (Basel), 2021, 11(8): 1384 [2022-08-15]. https://doi.org/10.3390/diagnostics11081384. DOI: 10.3390/diagnostics11081384.
[42]
MOHAN G, SUBASHINI M M. MRI based medical image analysis: survey on brain tumor grade classification[J]. Biomed Signal Process Control, 2018, 39: 139-161. DOI: 10.1016/j.bspc.2017.07.007.
[43]
NADEEM M W, GHAMDI M A A, HUSSAIN M, et al. Brain tumor analysis empowered with deep learning: a review, taxonomy, and future challenges[J/OL]. Brain Sci, 2020, 10(2): 118 [2022-08-15]. https://www.mdpi.com/2076-3425/10/2/118. DOI: 10.3390/brainsci10020118.
[44]
JIN L, MIN L, WANG J X, et al. A survey of MRI-based brain tumor segmentation methods[J]. Tsinghua Sci Technol, 2014, 19(6): 578-595. DOI: 10.1109/TST.2014.6961028.
[45]
ABD-ELLAH M K, AWAD A I, KHALAF A A M, et al. A review on brain tumor diagnosis from MRI images: practical implications, key achievements, and lessons learned[J]. Magn Reson Imaging, 2019, 61: 300-318. DOI: 10.1016/j.mri.2019.05.028.
[46]
SHARIF M I, LI J P, NAZ J, et al. A comprehensive review on multi-organs tumor detection based on machine learning[J]. Pattern Recognit Lett, 2020, 131: 30-37. DOI: 10.1016/j.patrec.2019.12.006.
[47]
ALARAIMI S, OKEDU K E, TIANFIELD H, et al. Transfer learning networks with skip connections for classification of brain tumors[J]. Int J Imaging Syst Technol, 2021, 31(3): 1564-1582. DOI: 10.1002/ima.22546.
[48]
SRINIVAS C, K S N P, ZAKARIAH M, et al. Deep transfer learning approaches in performance analysis of brain tumor classification using MRI images[J/OL]. J Healthc Eng, 2022, 2022: 3264367 [2022-08-15]. https://doi.org/10.1155/2022/3264367. DOI: 10.1155/2022/3264367.
[49]
BHATELE K R, BHADAURIA S S. Brain structural disorders detection and classification approaches: a review[J]. Artif Intell Rev, 2020, 53(5): 3349-3401. DOI: 10.1007/s10462-019-09766-9.

PREV Research progress of magnetic resonance imaging in the identification and evaluation of ischemic penumbra in acute cerebral infarction
NEXT Research progress in imaging of carotid calcified plaque
  


LINK


Tel & Fax: +8610-67113815    E-mail: editor@cjmri.cn