Highlights
- •A deep learning-based approach is proposed for left ventricle segmentation.
- •U-net architecture is modified and combined with feature pyramid network.
- •Feature maps in all levels of the decoder path of U-net are concatenated and resized.
- •Semantic segmentation is performed on this stack of feature maps.
- •The performance of the model is evaluated using a publicly and a prepared dataset.
Abstract
Segmentation of the Left ventricle (LV) is a crucial step for quantitative measurements
such as area, volume, and ejection fraction. However, the automatic LV segmentation
in 2D echocardiographic images is a challenging task due to ill-defined borders, and
operator dependence issues (insufficient reproducibility). U-net, which is a well-known
architecture in medical image segmentation, addressed this problem through an encoder-decoder
path. Despite outstanding overall performance, U-net ignores the contribution of all
semantic strengths in the segmentation procedure. In the present study, we have proposed
a novel architecture to tackle this drawback. Feature maps in all levels of the decoder
path of U-net are concatenated, their depths are equalized, and up-sampled to a fixed
dimension. This stack of feature maps would be the input of the semantic segmentation
layer. The performance of the proposed model was evaluated using two sets of echocardiographic
images: one public dataset and one prepared dataset. The proposed network yielded
significantly improved results when comparing with results from U-net, dilated U-net,
Unet++, ACNN, SHG, and deeplabv3. An average Dice Metric (DM) of 0.953, Hausdorff
Distance (HD) of 3.49, and Mean Absolute Distance (MAD) of 1.12 are achieved in the
public dataset. The correlation graph, bland-altman analysis, and box plot showed
a great agreement between automatic and manually calculated volume, area, and length.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Physica Medica: European Journal of Medical PhysicsAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Braunwald’s heart disease: a textbook of cardiovascular medicine.Saunders, 2018
- Recommendations for chamber quantification.Eur J Echocardiogr. 2006; 7: 79-108https://doi.org/10.1016/j.euje.2005.12.014
- Ultrasound image segmentation: a survey.IEEE Trans Med Imaging. 2006; 25: 987-1010https://doi.org/10.1109/TMI.2006.877092
- The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods.IEEE Trans IMAGE Process. 2012; 21https://doi.org/10.1109/TIP.2011.2169273
- A novel method for left ventricle volume measurement on short axis MRI images based on deformable superellipses.in: Int Joint Conf Adv Signal Process Inf Technol. Springer, 2012: 101-106
- Automatic segmentation of echocardiographic sequences by active appearance motion models.IEEE Trans Med Imaging. 2002; 21: 1374-1383https://doi.org/10.1109/TMI.2002.806427
- 3-D active appearance models: segmentation of cardiac MR and ultrasound images.IEEE Trans Med Imaging. 2002; 21: 1167-1178https://doi.org/10.1109/TMI.2002.804425
- Combinative multi-scale level set framework for echocardiographic image segmentation.Med Image Anal. 2003; 7: 529-537https://doi.org/10.1016/S1361-8415(03)00035-5
- ROPES: a semiautomated segmentation method for accelerated analysis of three-dimensional echocardiographic data.IEEE Trans Med Imaging. 2002; 21: 1091-1104https://doi.org/10.1109/TMI.2002.804432
- Probabilistic framework for tracking in artifact-prone 3D echocardiograms.Med Image Anal. 2010; 14: 750-758https://doi.org/10.1016/j.media.2010.06.003
- A hybrid graph-based approach for right ventricle segmentation in cardiac MRI by long axis information transition.Phys Medica. 2018; 54https://doi.org/10.1016/j.ejmp.2018.09.011
- Left ventricle segmentation using a combination of region growing and graph based method.Iran J Radiol. 2017; 14https://doi.org/10.5812/iranjradiol.42272
- Applying improved fast marching method to endocardial boundary detection in echocardiographic images.Pattern Recogn Lett. 2003; 24: 2777-2784https://doi.org/10.1016/S0167-8655(03)00121-1
- Using prior shapes in geometric active contours in a variational framework.Int J Comput Vis. 2002; 50: 315-328https://doi.org/10.1023/A:1020878408985
Georgescu B, Zhou XS, Comaniciu D, Gupta A. Database-Guided Segmentation of Anatomical Structures with Complex Appearance. 2005 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2, IEEE; n.d., p. 429–36. doi: 10.1109/CVPR.2005.119.
- ImageNet classification with deep convolutional neural networks.in: Adv. Neural Inf. Process. Syst. 25. Curran Associates, Inc., 2012: 1097-1105
- Deep learning.Nature. 2015; 521: 436-444https://doi.org/10.1038/nature14539
- Direct attenuation correction of brain PET images using only emission data via a deep convolutional encoder-decoder (Deep-DAC).Eur Radiol. 2019; : 1-13https://doi.org/10.1007/s00330-019-06229-1
- 2D left ventricle segmentation using deep learning.in: 2017 IEEE Int. Ultrason. Symp. IEEE, 2017: 1-4https://doi.org/10.1109/ULTSYM.2017.8092573
- U-Net: convolutional networks for biomedical image segmentation.in: Int. Conf. Med. image Comput. Comput. Interv. Springer, 2015: 234-241https://doi.org/10.1007/978-3-319-24574-4_28
- Measures of the amount of ecologic association between species.Ecology. 1945; 26: 297-302https://doi.org/10.2307/1932409
- Comparison and evaluation of segmentation techniques for subcortical structures in brain MRI.in: Med. Image Comput. Comput. Interv. – MICCAI 2008. Springer, Berlin, Heidelberg2008: 409-416https://doi.org/10.1007/978-3-540-85988-8_49
- Identification of the left ventricle endocardial border on two-dimensional ultrasound images using the convolutional neural network Unet.in: 2018 Ural Symp. Biomed. Eng. Radioelectron. Inf. Technol. IEEE, 2018: 76-78https://doi.org/10.1109/USBEREIT.2018.8384554
- Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation.IEEE Trans Med Imaging. 2018; 37: 384-395https://doi.org/10.1109/TMI.2017.2743464
- 3D U-Net: learning dense volumetric segmentation from sparse annotation.in: Int. Conf. Med. Image Comput. Comput. Interv. Springer, 2016: 424-432https://doi.org/10.1007/978-3-319-46723-8_49
- Standardized Evaluation System for Left Ventricular Segmentation Algorithms in 3D Echocardiography.IEEE Trans Med Imaging. 2016; 35: 967-977https://doi.org/10.1109/TMI.2015.2503890
- Combining fully convolutional and recurrent neural networks for 3d biomedical image segmentation.Adv. Neural Inf. Process. Syst. 2016; : 3036-3044
- Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation.in: IEEE 13th Int. Symp. Biomed. Imaging. IEEE, 2016: 1414-1417https://doi.org/10.1109/ISBI.2016.7493532
- Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation.in: Reconstr. Segmentation Anal. Med. Images. Springer, 2017: 83-94https://doi.org/10.1007/978-3-319-52280-7_8
Long J, Shelhamer E, Darrell T. Fully Convolutional Networks for Semantic Segmentation. IEEE Conf. Comput. Vis. pattern Recognit, 2015, p. 3431–40.
- Feature pyramid networks for object detection.CVPR. 2017; : 2117-2125
- Deep learning for segmentation using an open large-scale dataset in 2D echocardiography.IEEE Trans Med Imaging. 2019; : 1https://doi.org/10.1109/TMI.2019.2900516
- UNet++: a nested U-net architecture for medical image segmentation. Springer, Cham2018: 3-11https://doi.org/10.1007/978-3-030-00889-5_1
- Stacked hourglass networks for human pose estimation.in: Eur. Conf. Comput. Vis. Springer, Cham2016: 483-499https://doi.org/10.1007/978-3-319-46484-8_29
Yu F, Koltun V. multi-scale context aggregation by dilated convolutions. ArXiv Prepr ArXiv151107122 2016.
Chen L-C, Papandreou G, Schroff F, Adam H. Rethinking Atrous Convolution for Semantic Image Segmentation. ArXiv Prepr ArXiv170605587 2017.
- Pyramid methods in image processing.RCA Eng. 1984; 29: 33-41
- Laplacian pyramid reconstruction and refinement for semantic segmentation.in: Eur. Conf. Comput. Vis. Springer, Cham2016: 519-534https://doi.org/10.1007/978-3-319-46487-9_32
Honari S, Yosinski J, Vincent P, Pal C. Recombinator Networks: Learning Coarse-To-Fine Feature Aggregation. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, p. 5743–52.
- Learning to refine object segments.in: Eur. Conf. Comput. Vis. Springer, Cham2016: 75-91https://doi.org/10.1007/978-3-319-46448-0_5
- Local contrast and mean based thresholding technique in image binarization.Int J Comput Appl. 2012; 51: 6-10https://doi.org/10.5120/8044-1362
- An introduction to digital image processing.Prentice-Hall Englewood Cliffs, 1986
- Vosburgh 3D slicer: A platform for subject-specific image analysis, visualization, and clinical support. Intraoperative imaging image-guided ther. Springer New York, New York, NY2014: 277-289https://doi.org/10.1007/978-1-4614-7657-3_19
- Real-time simultaneous triplane contrast echocardiography gives rapid, accurate, and reproducible assessment of left ventricular volumes and ejection fraction: a comparison with magnetic resonance imaging.J Am Soc Echocardiogr. 2006; 19: 1494-1501https://doi.org/10.1016/J.ECHO.2006.06.021
- The Probabilistic Basis of Jaccard’s Index of Similarity.Artic Syst Biol. 1996; 45: 380-385https://doi.org/10.1093/sysbio/45.3.380
- An evaluation of four automatic methods of segmenting the subcortical structures in the brain.Neuroimage. 2009; 47: 1435-1447https://doi.org/10.1016/j.neuroimage.2009.05.029
- Experiments: planning, analysis, and parameter design optimization.Wiley, 2011
- Statistical methods for assessing agreement between two methods of clinical measurement.Int J Nurs Stud. 2010; 47: 931-936https://doi.org/10.1016/J.IJNURSTU.2009.10.001
- Efficient inference in fully connected crfs with gaussian edge potentials.Adv Neural Inf Process Syst. 2011; : 109-117
Article info
Publication history
Published online: October 28, 2019
Accepted:
October 1,
2019
Received in revised form:
September 8,
2019
Received:
July 10,
2019
Identification
Copyright
© 2019 Associazione Italiana di Fisica Medica. Published by Elsevier Ltd. All rights reserved.
