Original paper| Volume 67, P58-69, November 2019

Download started.


MFP-Unet: A novel deep learning based approach for left ventricle segmentation in echocardiography

Published:October 28, 2019DOI:


      • A deep learning-based approach is proposed for left ventricle segmentation.
      • U-net architecture is modified and combined with feature pyramid network.
      • Feature maps in all levels of the decoder path of U-net are concatenated and resized.
      • Semantic segmentation is performed on this stack of feature maps.
      • The performance of the model is evaluated using a publicly and a prepared dataset.


      Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements such as area, volume, and ejection fraction. However, the automatic LV segmentation in 2D echocardiographic images is a challenging task due to ill-defined borders, and operator dependence issues (insufficient reproducibility). U-net, which is a well-known architecture in medical image segmentation, addressed this problem through an encoder-decoder path. Despite outstanding overall performance, U-net ignores the contribution of all semantic strengths in the segmentation procedure. In the present study, we have proposed a novel architecture to tackle this drawback. Feature maps in all levels of the decoder path of U-net are concatenated, their depths are equalized, and up-sampled to a fixed dimension. This stack of feature maps would be the input of the semantic segmentation layer. The performance of the proposed model was evaluated using two sets of echocardiographic images: one public dataset and one prepared dataset. The proposed network yielded significantly improved results when comparing with results from U-net, dilated U-net, Unet++, ACNN, SHG, and deeplabv3. An average Dice Metric (DM) of 0.953, Hausdorff Distance (HD) of 3.49, and Mean Absolute Distance (MAD) of 1.12 are achieved in the public dataset. The correlation graph, bland-altman analysis, and box plot showed a great agreement between automatic and manually calculated volume, area, and length.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Physica Medica: European Journal of Medical Physics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Zipes D.P.
        • Libby P.
        • Bonow R.O.
        • Mann D.L.
        • Tomaselli G.F.
        • Braunwald E.
        Braunwald’s heart disease: a textbook of cardiovascular medicine.
        Saunders, 2018
        • Lang R.
        • Bierig M.
        • Devereux R.
        • Flachskampf F.
        • Foster E.
        • Pellikka P.
        • et al.
        Recommendations for chamber quantification.
        Eur J Echocardiogr. 2006; 7: 79-108
        • Noble J.A.
        • Boukerroui D.
        Ultrasound image segmentation: a survey.
        IEEE Trans Med Imaging. 2006; 25: 987-1010
        • Carneiro G.
        • Nascimento J.C.
        • Freitas A.
        • Nascimento J.C.
        The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods.
        IEEE Trans IMAGE Process. 2012; 21
        • Ghelich Oghli M.
        • Fallahi A.
        • Dehlaqi V.
        • Pooyan M.
        • Abdollahi N.
        A novel method for left ventricle volume measurement on short axis MRI images based on deformable superellipses.
        in: Int Joint Conf Adv Signal Process Inf Technol. Springer, 2012: 101-106
        • Bosch J.G.
        • Mitchell S.C.
        • Lelieveldt B.P.F.
        • Nijland F.
        • Kamp O.
        • Sonka M.
        • et al.
        Automatic segmentation of echocardiographic sequences by active appearance motion models.
        IEEE Trans Med Imaging. 2002; 21: 1374-1383
        • Mitchell S.C.
        • Bosch J.G.
        • Lelieveldt B.P.F.
        • van der Geest R.J.
        • Reiber J.H.C.
        • Sonka M.
        3-D active appearance models: segmentation of cardiac MR and ultrasound images.
        IEEE Trans Med Imaging. 2002; 21: 1167-1178
        • Lin N.
        • Yu W.
        • Duncan J.S.
        Combinative multi-scale level set framework for echocardiographic image segmentation.
        Med Image Anal. 2003; 7: 529-537
        • Wolf I.
        • Hastenteufel M.
        • De Simone R.
        • Vetter M.
        • Glombitza G.
        • Mottl-Link S.
        • et al.
        ROPES: a semiautomated segmentation method for accelerated analysis of three-dimensional echocardiographic data.
        IEEE Trans Med Imaging. 2002; 21: 1091-1104
        • Leung K.Y.E.
        • Danilouchkine M.G.
        • van Stralen M.
        • de Jong N.
        • van der Steen A.F.W.
        • Bosch J.G.
        Probabilistic framework for tracking in artifact-prone 3D echocardiograms.
        Med Image Anal. 2010; 14: 750-758
        • Ghelich Oghli M.
        • Mohammadzadeh A.
        • Kafieh R.
        • Kermani S.
        A hybrid graph-based approach for right ventricle segmentation in cardiac MRI by long axis information transition.
        Phys Medica. 2018; 54
        • Ghelich Oghli M.
        • Mohammadzadeh M.
        • Mohammadzadeh V.
        • Kadivar S.
        • Zadeh A.M.
        Left ventricle segmentation using a combination of region growing and graph based method.
        Iran J Radiol. 2017; 14
        • Yan J.
        • Zhuang T.
        Applying improved fast marching method to endocardial boundary detection in echocardiographic images.
        Pattern Recogn Lett. 2003; 24: 2777-2784
        • Chen Y.
        • Tagare H.D.
        • Thiruvenkadam S.
        • Huang F.
        • Wilson D.
        • Gopinath K.S.
        • et al.
        Using prior shapes in geometric active contours in a variational framework.
        Int J Comput Vis. 2002; 50: 315-328
      1. Georgescu B, Zhou XS, Comaniciu D, Gupta A. Database-Guided Segmentation of Anatomical Structures with Complex Appearance. 2005 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2, IEEE; n.d., p. 429–36. doi: 10.1109/CVPR.2005.119.

        • Krizhevsky A.
        • Sutskever I.
        • Hinton G.E.
        ImageNet classification with deep convolutional neural networks.
        in: Adv. Neural Inf. Process. Syst. 25. Curran Associates, Inc., 2012: 1097-1105
        • LeCun Y.
        • Bengio Y.
        • Hinton G.
        Deep learning.
        Nature. 2015; 521: 436-444
        • Shiri I.
        • Ghafarian P.
        • Geramifar P.
        • Leung K.H.-Y.
        • Ghelichoghli M.
        • Oveisi M.
        • et al.
        Direct attenuation correction of brain PET images using only emission data via a deep convolutional encoder-decoder (Deep-DAC).
        Eur Radiol. 2019; : 1-13
        • Smistad E.
        • Ostvik A.
        • Haugen B.O.
        • Lovstakken L.
        2D left ventricle segmentation using deep learning.
        in: 2017 IEEE Int. Ultrason. Symp. IEEE, 2017: 1-4
        • Ronneberger O.
        • Fischer P.
        • Brox T.
        U-Net: convolutional networks for biomedical image segmentation.
        in: Int. Conf. Med. image Comput. Comput. Interv. Springer, 2015: 234-241
        • Dice L.R.
        Measures of the amount of ecologic association between species.
        Ecology. 1945; 26: 297-302
        • Babalola K.O.
        • Patenaude B.
        • Aljabar P.
        • Schnabel J.
        • Kennedy D.
        • Crum W.
        • et al.
        Comparison and evaluation of segmentation techniques for subcortical structures in brain MRI.
        in: Med. Image Comput. Comput. Interv. – MICCAI 2008. Springer, Berlin, Heidelberg2008: 409-416
        • Zyuzin V.
        • Sergey P.
        • Mukhtarov A.
        • Chumarnaya T.
        • Solovyova O.
        • Bobkova A.
        • et al.
        Identification of the left ventricle endocardial border on two-dimensional ultrasound images using the convolutional neural network Unet.
        in: 2018 Ural Symp. Biomed. Eng. Radioelectron. Inf. Technol. IEEE, 2018: 76-78
        • Oktay O.
        • Ferrante E.
        • Kamnitsas K.
        • Heinrich M.
        • Bai W.
        • Caballero J.
        • et al.
        Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation.
        IEEE Trans Med Imaging. 2018; 37: 384-395
        • Çiçek Ö.
        • Abdulkadir A.
        • Lienkamp S.S.
        • Brox T.
        • Ronneberger O.
        3D U-Net: learning dense volumetric segmentation from sparse annotation.
        in: Int. Conf. Med. Image Comput. Comput. Interv. Springer, 2016: 424-432
        • Bernard O.
        • Bosch J.G.
        • Heyde B.
        • Alessandrini M.
        • Barbosa D.
        • Camarasu-Pop S.
        • et al.
        Standardized Evaluation System for Left Ventricular Segmentation Algorithms in 3D Echocardiography.
        IEEE Trans Med Imaging. 2016; 35: 967-977
        • Chen J.
        • Yang L.
        • Zhang Y.
        • Alber M.
        • Chen D.Z.
        Combining fully convolutional and recurrent neural networks for 3d biomedical image segmentation.
        Adv. Neural Inf. Process. Syst. 2016; : 3036-3044
        • Ghafoorian M.
        • Karssemeijer N.
        • Heskes T.
        • van Uder I.W.M.
        • de Leeuw F.E.
        • Marchiori E.
        • et al.
        Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation.
        in: IEEE 13th Int. Symp. Biomed. Imaging. IEEE, 2016: 1414-1417
        • Poudel R.P.K.
        • Lamata P.
        • Montana G.
        Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation.
        in: Reconstr. Segmentation Anal. Med. Images. Springer, 2017: 83-94
      2. Long J, Shelhamer E, Darrell T. Fully Convolutional Networks for Semantic Segmentation. IEEE Conf. Comput. Vis. pattern Recognit, 2015, p. 3431–40.

        • Lin T.-Y.
        • Dollár P.
        • Girshick R.
        • He K.
        • Hariharan B.
        • Belongie S.
        Feature pyramid networks for object detection.
        CVPR. 2017; : 2117-2125
        • Leclerc S.
        • Smistad E.
        • Pedrosa J.
        • Ostvik A.
        • Cervenansky F.
        • Espinosa F.
        • et al.
        Deep learning for segmentation using an open large-scale dataset in 2D echocardiography.
        IEEE Trans Med Imaging. 2019; : 1
        • Zhou Z.
        • Rahman Siddiquee M.M.
        • Tajbakhsh N.
        • Liang J.
        UNet++: a nested U-net architecture for medical image segmentation. Springer, Cham2018: 3-11
        • Newell A.
        • Yang K.
        • Deng J.
        Stacked hourglass networks for human pose estimation.
        in: Eur. Conf. Comput. Vis. Springer, Cham2016: 483-499
      3. Yu F, Koltun V. multi-scale context aggregation by dilated convolutions. ArXiv Prepr ArXiv151107122 2016.

      4. Chen L-C, Papandreou G, Schroff F, Adam H. Rethinking Atrous Convolution for Semantic Image Segmentation. ArXiv Prepr ArXiv170605587 2017.

        • Adelson E.H.
        • Anderson C.H.
        • Bergen J.R.
        • Burt P.J.
        • Ogden J.M.
        Pyramid methods in image processing.
        RCA Eng. 1984; 29: 33-41
        • Ghiasi G.
        • Fowlkes C.C.
        Laplacian pyramid reconstruction and refinement for semantic segmentation.
        in: Eur. Conf. Comput. Vis. Springer, Cham2016: 519-534
      5. Honari S, Yosinski J, Vincent P, Pal C. Recombinator Networks: Learning Coarse-To-Fine Feature Aggregation. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, p. 5743–52.

        • Pinheiro P.O.
        • Lin T.-Y.
        • Collobert R.
        • Dollár P.
        Learning to refine object segments.
        in: Eur. Conf. Comput. Vis. Springer, Cham2016: 75-91
        • Singh O.I.
        • Sinam T.
        • James O.
        • Singh T.R.
        Local contrast and mean based thresholding technique in image binarization.
        Int J Comput Appl. 2012; 51: 6-10
        • Niblack W.
        An introduction to digital image processing.
        Prentice-Hall Englewood Cliffs, 1986
        • Kikinis R.S.D.
        • Pieper K.G.
        Vosburgh 3D slicer: A platform for subject-specific image analysis, visualization, and clinical support. Intraoperative imaging image-guided ther. Springer New York, New York, NY2014: 277-289
        • Malm S.
        • Frigstad S.
        • Sagberg E.
        • Steen P.A.
        • Skjarpe T.
        Real-time simultaneous triplane contrast echocardiography gives rapid, accurate, and reproducible assessment of left ventricular volumes and ejection fraction: a comparison with magnetic resonance imaging.
        J Am Soc Echocardiogr. 2006; 19: 1494-1501
        • Vargas J.M.
        The Probabilistic Basis of Jaccard’s Index of Similarity.
        Artic Syst Biol. 1996; 45: 380-385
        • Babalola K.O.
        • Patenaude B.
        • Aljabar P.
        • Schnabel J.
        • Kennedy D.
        • Crum W.
        • et al.
        An evaluation of four automatic methods of segmenting the subcortical structures in the brain.
        Neuroimage. 2009; 47: 1435-1447
        • Wu C.J.
        • Hamada M.S.
        Experiments: planning, analysis, and parameter design optimization.
        Wiley, 2011
        • Bland J.M.
        • Altman D.G.
        Statistical methods for assessing agreement between two methods of clinical measurement.
        Int J Nurs Stud. 2010; 47: 931-936
        • Krähenbühl P.
        • Koltun V.
        Efficient inference in fully connected crfs with gaussian edge potentials.
        Adv Neural Inf Process Syst. 2011; : 109-117