Share：
Share this content in WeChat
X
[Chinese][PDF]3581
• Review •
Application of generative adversarial networks in cardiac magnetic resonance
LIU Weixiao  FANG Jin  WANG Ying  MO Xiaokai  ZHANG Shuixing 

Cite this article as: LIU W X, FANG J, WANG Y, et al. Application of generative adversarial networks in cardiac magnetic resonance[J]. Chin J Magn Reson Imaging, 2023, 14(6): 139-144. DOI:10.12015/issn.1674-8034.2023.06.025.


[Abstract] Cardiac magnetic resonance (CMR) is an important examination method for evaluating cardiovascular structure and function. Deep learning has been widely used in CMR image processing. Generative adversarial network (GAN), as a new type of network in deep learning, uses the adversarial game between the generator responsible for image generation and the discriminator responsible for judging the authenticity of images to create image processing models with powerful generation and generalization capabilities. We summarized the main applications of GAN in CMR image segmentation and synthesis, image reconstruction, super-resolution reconstruction, and virtual native enhancement in this paper. Combined with the current clinical application requirements of CMR, we analyzed the challenges and future prospects faced by GAN models in order to improve their practical application value as soon as possible.
[Keywords] cardiac magnetic resonance；image processing；generative adversarial network；deep learning；artificial intelligence；magnetic resonance imaging

LIU Weixiao   FANG Jin   WANG Ying   MO Xiaokai   ZHANG Shuixing*  

Department of Radiology, the First Affiliated Hospital of Jinan University, Guangzhou 510000, China

Corresponding author: Zhang SX, E-mail: shui7515@126.com

Conflicts of interest   None.

ACKNOWLEDGMENTS National Natural Science Foundation of China (No. 81871323); Medical Science and Technology Research Fund of Guangdong Province (No. A2022267); Fundamental Research Funds for the Central Universities (No. 21621050).
Received  2023-01-11
Accepted  2023-04-23
DOI: 10.12015/issn.1674-8034.2023.06.025
Cite this article as: LIU W X, FANG J, WANG Y, et al. Application of generative adversarial networks in cardiac magnetic resonance[J]. Chin J Magn Reson Imaging, 2023, 14(6): 139-144. DOI:10.12015/issn.1674-8034.2023.06.025.

[1]
CHONG J H, ABDULKAREEM M, PETERSEN S E, et al. Artificial intelligence and cardiovascular magnetic resonance imaging in myocardial infarction patients[J/OL]. Curr Probl Cardiol, 2022, 47(12): 101330 [2023-01-10]. https://pubmed.ncbi.nlm.nih.gov/35870544/. DOI: 10.1016/j.cpcardiol.2022.101330.
[2]
TING D S W, CHEUNG C Y L, LIM G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes[J]. JAMA, 2017, 318(22): 2211-2223. DOI: 10.1001/jama.2017.18152.
[3]
ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639): 115-118. DOI: 10.1038/nature21056.
[4]
BERA K, BRAMAN N, GUPTA A, et al. Predicting cancer outcomes with radiomics and artificial intelligence in radiology[J]. Nat Rev Clin Oncol, 2022, 19(2): 132-146. DOI: 10.1038/s41571-021-00560-7.
[5]
WANG S, CHAUHAN D, PATEL H, et al. Assessment of right ventricular size and function from cardiovascular magnetic resonance images using artificial intelligence[J/OL]. J Cardiovasc Magn Reson, 2022, 24(1): 27 [2023-01-10]. https://pubmed.ncbi.nlm.nih.gov/35410226/. DOI: 10.1186/s12968-022-00861-5.
[6]
WANG J, BRAVO L, ZHANG J Q, et al. Radiomics analysis derived from LGE-MRI predict sudden cardiac death in participants with hypertrophic cardiomyopathy[J/OL]. Front Cardiovasc Med, 2021, 8: 766287 [2023-01-10]. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8702805/. DOI: 10.3389/fcvm.2021.766287.
[7]
RAVÌ D, WONG C, DELIGIANNI F, et al. Deep learning for health informatics[J]. IEEE J Biomed Health Inform, 2016, 21(1): 4-21. DOI: 10.1109/JBHI.2016.2636665.
[8]
POLONSKY T S, MCCLELLAND R L, JORGENSEN N W, et al. Coronary artery calcium score and risk classification for coronary heart disease prediction[J]. JAMA, 2010, 303(16): 1610-1616. DOI: 10.1001/jama.2010.461.
[9]
BHUVA A N, BAI W J, LAU C, et al. A multicenter, scan-rescan, human and machine learning CMR study to test generalizability and precision in imaging biomarker analysis[J/OL]. Circ Cardiovasc Imaging, 2019, 12(10): e009214 [2023-01-07]. https://www.ahajournals.org/doi/10.1161/CIRCIMAGING.119.009214. DOI: 10.1161/CIRCIMAGING.119.009214.
[10]
GREENSPAN H, VAN GINNEKEN B, SUMMERS R M. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique[J]. IEEE Trans Med Imaging, 2016, 35(5): 1153-1159. DOI: 10.1109/TMI.2016.2553401.
[11]
GOODFELLOW I, POUGET J, MIRZA M, et al. Generative adversarial networks[J]. Commun ACM, 2020, 63(11): 139-144. DOI: 10.1145/3422622.
[12]
ZHAO L J, BAI H H, LIANG J, et al. Simultaneous color-depth super-resolution with conditional generative adversarial networks[J]. Pattern Recognit, 2019, 88: 356-369. DOI: 10.1016/j.patcog.2018.11.028.
[13]
WANG Q, FAN H J, SUN G, et al. Recurrent generative adversarial network for face completion[J]. IEEE Trans Multimed, 2021, 23: 429-442. DOI: 10.1109/TMM.2020.2978633.
[14]
LIU M Y, HUANG X, MALLYA A, et al. Few-shot unsupervised image-to-image translation[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27 - November 2, 2019, Seoul, Korea (South). IEEE, 2020: 10550-10559. DOI: 10.1109/ICCV.2019.01065.
[15]
YI X, WALIA E, BABYN P. Generative adversarial network in medical imaging: a review[J/OL]. Med Image Anal, 2019, 58: 101552 [2023-01-08]. https://pubmed.ncbi.nlm.nih.gov/31521965/. DOI: 10.1016/j.media.2019.101552.
[16]
GHEORGHIȚĂ B A, ITU L M, SHARMA P, et al. Improving robustness of automatic cardiac function quantification from cine magnetic resonance imaging using synthetic image data[J/OL]. Sci Rep, 2022, 12(1): 2391 [2023-01-08]. https://pubmed.ncbi.nlm.nih.gov/35165324/. DOI: 10.1038/s41598-022-06315-3.
[17]
PENA H, GOMEZ S, ROMO-BUCHELI D, et al. Cardiac disease representation conditioned by spatio-temporal priors in cine-MRI sequences using generative embedding vectors[J]. Annu Int Conf IEEE Eng Med Biol Soc, 2021, 2021: 5570-5573. DOI: 10.1109/EMBC46164.2021.9630115.
[18]
GUARICCI A I, MASCI P G, MUSCOGIURI G, et al. CarDiac magnEtic Resonance for prophylactic Implantable-cardioVerter defibrillAtor ThErapy in Non-Ischaemic dilated CardioMyopathy: an international Registry[J]. Europace, 2021, 23(7): 1072-1083. DOI: 10.1093/europace/euaa401.
[19]
LEINER T, BOGAERT J, FRIEDRICH M G, et al. SCMR Position Paper (2020) on clinical indications for cardiovascular magnetic resonance[J/OL]. J Cardiovasc Magn Reson, 2020, 22(1): 76 [2023-01-08]. https://pubmed.ncbi.nlm.nih.gov/33161900/. DOI: 10.1186/s12968-020-00682-4.
[20]
WANG S, PATEL H, MILLER T, et al. AI based CMR assessment of biventricular function: clinical significance of intervendor variability and measurement errors[J]. JACC Cardiovasc Imaging, 2022, 15(3): 413-427. DOI: 10.1016/j.jcmg.2021.08.011.
[21]
ZHANG Y T, FENG J Q, GUO X, et al. Comparative analysis of U-Net and TLMDB GAN for the cardiovascular segmentation of the ventricles in the heart[J/OL]. Comput Methods Programs Biomed, 2022, 215: 106614 [2023-01-08]. https://www.sciencedirect.com/science/article/abs/pii/S016926072100688X. DOI: 10.1016/j.cmpb.2021.106614.
[22]
WU H S, LU X H, LEI B Y, et al. Automated left ventricular segmentation from cardiac magnetic resonance images via adversarial learning with multi-stage pose estimation network and co-discriminator[J/OL]. Med Image Anal, 2021, 68: 101891 [2023-01-09]. https://www.sciencedirect.com/science/article/abs/pii/S1361841520302553?via%3Dihub. DOI: 10.1016/j.media.2020.101891.
[23]
UPENDRA R R, DANGI S, LINTE C A. An adversarial network architecture using 2D U-net models for segmentation of left ventricle from cine cardiac MRI[J]. Funct Imaging Model Heart, 2019, 11504: 415-424. DOI: 10.1007/978-3-030-21949-9_45.
[24]
DECOURT C, DUONG L. Semi-supervised generative adversarial networks for the segmentation of the left ventricle in pediatric MRI[J/OL]. Comput Biol Med, 2020, 123: 103884 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/32658792/. DOI: 10.1016/j.compbiomed.2020.103884.
[25]
YANG X Y, ZHANG Y, LO B, et al. DBAN: adversarial network with multi-scale features for cardiac MRI segmentation[J]. IEEE J Biomed Health Inform, 2021, 25(6): 2018-2028. DOI: 10.1109/JBHI.2020.3028463.
[26]
KANAKATTE A, BHATIA D, GHOSE A. 3D cardiac substructures segmentation from CMRI using generative adversarial network (GAN)[J]. Annu Int Conf IEEE Eng Med Biol Soc, 2022, 2022: 1698-1701. DOI: 10.1109/EMBC48229.2022.9871950.
[27]
CUI H F, YUWEN C, JIANG L, et al. Bidirectional cross-modality unsupervised domain adaptation using generative adversarial networks for cardiac image segmentation[J/OL]. Comput Biol Med, 2021, 136: 104726 [2023-01-09]. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7935602/. DOI: 10.1016/j.compbiomed.2021.104726.
[28]
CHEN J, YANG G, KHAN H, et al. JAS-GAN: generative adversarial network based joint atrium and scar segmentations on unbalanced atrial targets[J]. IEEE J Biomed Health Inform, 2022, 26(1): 103-114. DOI: 10.1109/JBHI.2021.3077469.
[29]
AMIRRAJAB S, KHALIL Y AL, LORENZ C, et al. Label-informed cardiac magnetic resonance image synthesis through conditional generative adversarial networks[J/OL]. Comput Med Imaging Graph, 2022, 101: 102123 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/36174308/. DOI: 10.1016/j.compmedimag.2022.102123.
[30]
DILLER G P, VAHLE J, RADKE R, et al. Utility of deep learning networks for the generation of artificial cardiac magnetic resonance images in congenital heart disease[J/OL]. BMC Med Imaging, 2020, 20(1): 113 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/33032536/. DOI: 10.1186/s12880-020-00511-1.
[31]
KARIMI-BIDHENDI S, ARAFATI A, CHENG A L, et al. Fully-automated deep-learning segmentation of pediatric cardiovascular magnetic resonance of patients with complex congenital heart diseases[J/OL]. J Cardiovasc Magn Reson, 2020, 22(1): 80 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/33256762/. DOI: 10.1186/s12968-020-00678-0.
[32]
VAN DER GRAAF A W, BHAGIRATH P, GHOERBIEN S, et al. Cardiac magnetic resonance imaging: artefacts for clinicians[J]. Neth Heart J, 2014, 22(12): 542-549. DOI: 10.1007/s12471-014-0623-z.
[33]
ZHANG L, GOOYA A, PEREANEZ M, et al. Automatic assessment of full left ventricular coverage in cardiac cine magnetic resonance imaging with fisher discriminative 3D CNN[J/OL]. IEEE Trans Biomed Eng, 2018 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/30475705/. DOI: 10.1109/TBME.2018.2881952.
[34]
YUAN Z M, JIANG M F, WANG Y M, et al. SARA-GAN: self-attention and relative average discriminator based generative adversarial networks for fast compressed sensing MRI reconstruction[J/OL]. Front Neuroinform, 2020, 14: 611666 [2023-01-09]. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7726262/. DOI: 10.3389/fninf.2020.611666.
[35]
XIA Y, ZHANG L, RAVIKUMAR N, et al. Recovering from missing data in population imaging - Cardiac MR image imputation via conditional generative adversarial nets[J/OL]. Med Image Anal, 2021, 67: 101812 [2023-01-09]. https://linkinghub.elsevier.com/retrieve/pii/S1361-8415(20)30176-6. DOI: 10.1016/j.media.2020.101812.
[36]
LYU Q, SHAN H M, XIE Y B, et al. Cine cardiac MRI motion artifact reduction using a recurrent neural network[J]. IEEE Trans Med Imaging, 2021, 40(8): 2170-2181. DOI: 10.1109/TMI.2021.3073381.
[37]
HUANG J H, DING W P, LV J, et al. Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information[J].Appl Intell, 2022, 52(13): 14693-14710. DOI: 10.1007/s10489-021-03092-w.
[38]
LV J, WANG C Y, YANG G. PIC-GAN: a parallel imaging coupled generative adversarial network for accelerated multi-channel MRI reconstruction[J/OL]. Diagnostics, 2021, 11(1): 61 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/33401777/. DOI: 10.3390/diagnostics11010061.
[39]
GHODRATI V, BYDDER M, BEDAYAT A, et al. Temporally aware volumetric generative adversarial network-based MR image reconstruction with simultaneous respiratory motion compensation: initial feasibility in 3D dynamic cine cardiac MRI[J]. Magn Reson Med, 2021, 86(5): 2666-2683. DOI: 10.1002/mrm.28912.
[40]
TSAI R Y, HUANG T S. Multiframe image restoration and registration[J]. Adv Comput. Vis Image Process, 1984, 1(2): 317-339.
[41]
SHI J S, YE Y G, LIU H F, et al. Super-resolution reconstruction of Pneumocystis carinii pneumonia images based on generative confrontation network[J/OL]. Comput Methods Programs Biomed, 2022, 215: 106578 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/34998168/. DOI: 10.1016/j.cmpb.2021.106578.
[42]
SONG T A, CHOWDHURY S R, YANG F, et al. PET image super-resolution using generative adversarial networks[J]. Neural Netw, 2020, 125: 83-91. DOI: 10.1016/j.neunet.2020.01.029.
[43]
DELANNOY Q, PHAM C H, CAZORLA C, et al. SegSRGAN: Super-resolution and segmentation using generative adversarial networks-Application to neonatal brain MRI[J/OL]. Comput Biol Med, 2020, 120: 103755 [2023-01-09]. https://pubmed.ncbi.nlm.nih.gov/32421654/. DOI: 10.1016/j.compbiomed.2020.103755.
[44]
MAHAPATRA D, BOZORGTABAR B, GARNAVI R. Image super-resolution using progressive generative adversarial networks for medical image analysis[J/OL]. Comput Med Imaging Graph, 2019, 71: 30-39 [2023-01-10]. https://linkinghub.elsevier.com/retrieve/pii/S0895-6111(18)30587-1. DOI: 10.1016/j.compmedimag.2018.10.005.
[45]
JIANG M F, ZHI M H, WEI L Y, et al. FA-GAN: Fused attentive generative adversarial networks for MRI image super-resolution[J/OL]. Comput Med Imaging Graph, 2021, 92: 101969 [2023-01-10]. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8453331/. DOI: 10.1016/j.compmedimag.2021.101969.
[46]
ZHAO M, LIU X H, LIU H, et al. Super-resolution of cardiac magnetic resonance images using Laplacian Pyramid based on Generative Adversarial Networks[J/OL]. Comput Med Imaging Graph, 2020, 80: 101698 [2023-01-10]. https://linkinghub.elsevier.com/retrieve/pii/S089561112030001X. DOI: 10.1016/j.compmedimag.2020.101698.
[47]
KÜSTNER T, MUNOZ C, PSENICNY A, et al. Deep-learning based super-resolution for 3D isotropic coronary MR angiography in less than a minute[J]. Magn Reson Med, 2021, 86(5): 2837-2852. DOI: 10.1002/mrm.28911.
[48]
RUDNICK M R, WAHBA I M, LEONBERG-YOO A K, et al. Risks and options with gadolinium-based contrast agents in patients with CKD: a review[J]. Am J Kidney Dis, 2021, 77(4): 517-528. DOI: 10.1053/j.ajkd.2020.07.012.
[49]
PASQUINI L, NAPOLITANO A, VISCONTI E, et al. Gadolinium-based contrast agent-related toxicities[J]. CNS Drugs, 2018, 32(3): 229-240. DOI: 10.1007/s40263-018-0500-1.
[50]
ZHANG Q, BURRAGE M K, SHANMUGANATHAN M, et al. Artificial intelligence for contrast-free MRI: scar assessment in myocardial infarction using deep learning-based virtual native enhancement[J]. Circulation, 2022, 146(20): 1492-1503. DOI: 10.1161/CIRCULATIONAHA.122.060137.
[51]
BUSTAMANTE M, VIOLA F, CARLHÄLL C J, et al. Using deep learning to emulate the use of an external contrast agent in cardiovascular 4D flow MRI[J]. J Magn Reson Imaging, 2021, 54(3): 777-786. DOI: 10.1002/jmri.27578.

PREV Research progresses of cardiac magnetic resonance in 2022
NEXT Research progress on the application of magnetic resonance T2 mapping technology in heart disease
  


LINK


Tel & Fax: +8610-67113815    E-mail: editor@cjmri.cn