Research Article| Volume 107, 102560, March 2023

Download started.


A novel deep learning model for breast lesion classification using ultrasound Images: A multicenter data evaluation

Published:March 04, 2023DOI:


      • We developed a promoted version of InceptionV3 network to classify breast lesions.
      • The main promotion is converting InceptionV3 modules to residual inception modules.
      • Five datasets (3 public and 2 private) were used for training and evaluation.
      • The model were compared with 24 CNN architectures.
      • It achieved the best results in terms of accuracy, AUC, RMSE, Cronbach's α, f1, etc.



      Breast cancer is one of the major reasons of death due to cancer in women. Early diagnosis is the most critical key for disease screening, control, and reducing mortality. A robust diagnosis relies on the correct classification of breast lesions. While breast biopsy is referred to as the “gold standard” in assessing both the activity and degree of breast cancer, it is an invasive and time-consuming approach.


      The current study’s primary objective was to develop a novel deep-learning architecture based on the InceptionV3 network to classify ultrasound breast lesions. The main promotions of the proposed architecture were converting the InceptionV3 modules to residual inception ones, increasing their number, and altering the hyperparameters. In addition, we used a combination of five datasets (three public datasets and two prepared from different imaging centers) for training and evaluating the model.


      The dataset was split into the train (80%) and test (20%) groups. The model achieved 0.83, 0.77, 0.8, 0.81, 0.81, 0.18, and 0.77 for the precision, recall, F1 score, accuracy, AUC, Root Mean Squared Error, and Cronbach’s α in the test group, respectively.


      This study illustrates that the improved InceptionV3 can robustly classify breast tumors, potentially reducing the need for biopsy in many cases.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Physica Medica: European Journal of Medical Physics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Siegel R.L.
        • Miller K.D.
        • Fuchs H.E.
        • Jemal A.
        Cancer statistics, 2021.
        CA Cancer J Clin. 2021; 71: 7-33
        • Yap Moi Hoon
        • Goyal Manu
        • Osman Fatima
        • Martí Robert
        • Denton Erika
        • Juette Arne
        • et al.
        Breast ultrasound region of interest detection and lesion localisation.
        Artif Intell Med. 2020; 107: 101880
        • Ahmed M.
        • Purushotham A.D.
        • Douek M.
        Novel techniques for sentinel lymph node biopsy in breast cancer: a systematic review.
        Lancet Oncol. 2014; 15: e351-e362
        • Kim G.R.
        • Choi J.S.
        • Han B.K.
        • Lee J.E.
        • Nam S.J.
        • Ko E.Y.
        • et al.
        Preoperative axillary US in early-stage breast cancer: Potential to prevent unnecessary axillary lymph node dissection.
        Radiology. 2018; 288: 55-63
      1. Xueyi Zheng Zhao Yao Yini Huang Yanyan Yu Yun Wang Yubo Liu et al. Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer Nat Commun 11 1.

      2. Samantha Bove Maria Colomba Comes Vito Lorusso Cristian Cristofaro Vittorio Didonna Gianluca Gatta et al. A ultrasound-based radiomic approach to predict the nodal status in clinically negative breast cancer patients Sci Rep 12 1.

        • Kim S.-Y.
        • Choi Y.
        • Kim E.-K.
        • Han B.-K.
        • Yoon J.H.
        • Choi J.S.
        • et al.
        Deep learning-based computer-aided diagnosis in screening breast ultrasound to reduce false-positive diagnoses.
        Sci Rep. 2021; 11: 1-11
        • Takemura A.
        • Shimizu A.
        • Hamamoto K.
        Discrimination of breast tumors in ultrasonic images using an ensemble classifier based on the AdaBoost algorithm with feature selection.
        IEEE Trans Med Imaging. 2010; 29: 598-609
        • Hamet P.
        • Tremblay J.
        Artificial intelligence in medicine.
        Metabolism. 2017; 69S: S36-S40
        • Tagliafico A.S.
        • Piana M.
        • Schenone D.
        • Lai R.
        • Massone A.M.
        • Houssami N.
        Overview of radiomics in breast cancer diagnosis and prognostication.
        Breast. 2020; 49: 74-80
        • Rodrigues R.
        • Pinheiro A.
        • Braz R.
        • Pereira M.
        • Moutinho J.
        Towards Breast Ultrasound Image Segmentation using Multi-resolution Pixel Descriptors.
        in: Proc 21st Int Conf Pattern Recognit. IEEE, 2012: 2833-2836
      3. Sadek I, Elawady M, Stefanovski V. Automated Breast Lesion Segmentation in Ultrasound Images 2016. doi:10.48550/arxiv.1609.08364.

      4. Shi X, Cheng HD, Hu L. MASS DETECTION AND CLASSIFICATION IN BREAST ULTRASOUND IMAGES USING FUZZY SVM. Proc 9th Jt Conf Inf Sci JCIS 2006 2006;2006:253–6. doi:10.2991/JCIS.2006.257.

        • Dhahri Habib
        • Al Maghayreh Eslam
        • Mahmood Awais
        • Elkilani Wail
        • Faisal Nagi Mohammed
        Automated breast cancer diagnosis based on machine learning algorithms.
        J Healthc Eng. 2019; 2019: 1-11
        • Zakeri F.S.
        • Behnam H.
        • Ahmadinejad N.
        Classification of benign and malignant breast masses based on shape and texture features in sonography images.
        J Med Syst. 2012; 36: 1621-1627
        • Hagiwara Yuki
        • Koh Joel En Wei
        • Tan Jen Hong
        • Bhandary Sulatha V.
        • Laude Augustinus
        • Ciaccio Edward J.
        • et al.
        Computer-aided diagnosis of glaucoma using fundus images: a review.
        Comput Methods Programs Biomed. 2018; 165: 1-12
        • Paterakis N.G.
        • Mocanu E.
        • Gibescu M.
        • Stappers B.
        • van Alst W.
        Deep learning versus traditional machine learning methods for aggregated energy demand prediction.
        IEEE PES Innov Smart Grid Technol Conf Eur. 2017; 2017: 1-6
        • Moon W.K.
        • Lo C.M.
        • Chang J.M.
        • Huang C.S.
        • Chen J.H.
        • Chang R.F.
        Computer-aided classification of breast masses using speckle features of automated breast ultrasound images.
        Med Phys. 2012; 39: 6465-6473
        • Gómez Flores W.
        • Pereira W.C.D.A.
        • Infantosi A.F.C.
        Improving classification performance of breast lesions on ultrasonography.
        Pattern Recognit. 2015; 48: 1125-1136
        • Byra Michał
        • Nowicki Andrzej
        • Wróblewska-Piotrzkowska Hanna
        • Dobruch-Sobczak Katarzyna
        Classification of breast lesions using segmented quantitative ultrasound maps of homodyned K distribution parameters.
        Med Phys. 2016; 43: 5561-5569
        • Uniyal Nishant
        • Eskandari Hani
        • Abolmaesumi Purang
        • Sojoudi Samira
        • Gordon Paula
        • Warren Linda
        • et al.
        Ultrasound RF time series for classification of breast lesions.
        IEEE Trans Med Imaging. 2015; 34: 652-661
        • Bellotti R.
        • Bagnasco S.
        • Bottigli U.
        • Castellano M.
        • Cataldo R.
        • Catanzariti E.
        • et al.
        The MAGIC-5 project: medical applications on a grid infrastructure connection.
        IEEE Nucl Sci Symp Conf Rec. 2004; 3: 1902-1906
        • Saw S.N.
        • Ng K.H.
        Current challenges of implementing artificial intelligence in medical imaging.
        Phys Med. 2022; 100: 12-17
        • Litjens Geert
        • Kooi Thijs
        • Bejnordi Babak Ehteshami
        • Setio Arnaud Arindra Adiyoso
        • Ciompi Francesco
        • Ghafoorian Mohsen
        • et al.
        A survey on deep learning in medical image analysis.
        Med Image Anal. 2017; 42: 60-88
        • Kermani Saeed
        • Ghelich Oghli Mostafa
        • Mohammadzadeh Ali
        • Kafieh Raheleh
        NF-RCNN: Heart localization and right ventricle wall motion abnormality detection in cardiac MRI.
        Phys Med. 2020; 70: 65-74
        • Comes Maria Colomba
        • Fanizzi Annarita
        • Bove Samantha
        • Didonna Vittorio
        • Diotaiuti Sergio
        • La Forgia Daniele
        • et al.
        Early prediction of neoadjuvant chemotherapy response by exploiting a transfer learning approach on breast DCE-MRIs.
        Sci Reports. 2021; 11: 1-12
        • Liu M.Z.
        • Mutasa S.
        • Chang P.
        • Siddique M.
        • Jambawalikar S.
        • Ha R.
        A novel CNN algorithm for pathological complete response prediction using an I-SPY TRIAL breast MRI database.
        Magn Reson Imaging. 2020; 73: 148-151
        • Ho P.-S.
        • Hwang Y.-S.
        • Tsai H.-Y.
        Machine learning framework for automatic image quality evaluation involving a mammographic American College of Radiology phantom.
        Phys Med. 2022; 102: 1-8
        • LeCun Y.
        • Bengio Y.
        • Hinton G.
        Deep learning.
        Nature. 2015; 521: 436-444
        • Mohammed M.A.
        • Al-Khateeb B.
        • Rashid A.N.
        • Ibrahim D.A.
        • Abd Ghani M.K.
        • Mostafa S.A.
        Neural network and multi-fractal dimension features for breast cancer classification from ultrasound images.
        Comput Electr Eng. 2018; 70: 871-882
        • Wang Y.
        • Choi E.J.
        • Choi Y.
        • Zhang H.
        • Jin G.Y.
        • Ko S.B.
        Breast cancer classification in automated breast ultrasound using multiview convolutional neural network with transfer learning.
        Ultrasound Med Biol. 2020; 46: 1119-1132
        • Shi J.
        • Zhou S.
        • Liu X.
        • Zhang Q.
        • Lu M.
        • Wang T.
        Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset.
        Neurocomputing. 2016; 194: 87-94
        • Cao Z.
        • Duan L.
        • Yang G.
        • Yue T.
        • Chen Q.
        An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures.
        BMC Med Imaging. 2019; 19: 1-9
      5. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the Inception Architecture for Computer Vision. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2015;2016-Decem:2818–26. doi:10.48550/arxiv.1512.00567.

      6. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size 2016. doi:10.48550/arxiv.1602.07360.

      7. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely Connected Convolutional Networks. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017 2016;2017-Janua:2261–9. doi:10.48550/arxiv.1608.06993.

      8. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications 2017. doi:10.48550/arxiv.1704.04861.

      9. Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. ArXiv:14091556 2014.

      10. Xie S, Girshick R, Dollár P, Tu Z, He K. Aggregated Residual Transformations for Deep Neural Networks. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017 2016;2017-Janua:5987–95. doi:10.48550/arxiv.1611.05431.

      11. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. 31st AAAI Conf Artif Intell AAAI 2017 2016:4278–84. doi:10.48550/arxiv.1602.07261.

        • He K.
        • Zhang X.
        • Ren S.
        • Sun J.
        Deep residual learning for image recognition.
        IEEE Conf Comput Vis pattern Recognit. 2016; : 770-778
        • Al-Dhabyani Walid
        • Gomaa Mohammed
        • Khaled Hussien
        • Fahmy Aly
        Dataset of breast ultrasound images.
        Data Br. 2020; 28: 104863
        • Yap Moi Hoon
        • Pons Gerard
        • Marti Joan
        • Ganau Sergi
        • Sentis Melcior
        • Zwiggelaar Reyer
        • et al.
        Automated breast ultrasound lesions detection using convolutional neural networks.
        IEEE J Biomed Heal Informatics. 2018; 22: 1218-1226
      12. Stanislav Makhanov. Ultrasound Images 2012.

        • Morid Mohammad Amin
        • Borjali Alireza
        • Del Fiol Guilherme
        A scoping review of transfer learning research on medical image analysis using ImageNet.
        Comput Biol Med. 2021; 128: 104115
      13. Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions. Proc - 30th IEEE Conf Comput Vis Pattern Recognition, CVPR 2017 2016;2017-Janua:1800–7. doi:10.48550/arxiv.1610.02357.

        • Mada R.O.
        • Lysyansky P.
        • Daraban A.M.
        • Duchenne J.
        • Voigt J.U.
        How to define end-diastole and end-systole?: Impact of timing on strain measurements.
        JACC Cardiovasc Imaging. 2015; 8: 148-157
      14. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, et al. This paper is included in the Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16). Open access to the Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation is sponsored by US n.d.

      15. Ting KM. Confusion Matrix. Encycl Mach Learn Data Min 2017:260.

        • Connelly L.M.
        Cronbach’s alpha.
        Medsurg Nurs. 2014; 20: 1357-1359
      16. Fürnkranz J, Chan PK, Craw S, Sammut C, Uther W, Ratnaparkhi A, et al. Mean Squared Error. Encycl Mach Learn 2011:653. doi:10.1007/978-0-387-30164-8_528.

      17. Breast Cancer: Statistics | Cancer.Net. CancerNet n.d. (accessed August 13, 2022).

        • Eroğlu Y.
        • Yildirim M.
        • Çinar A.
        Convolutional Neural Networks based classification of breast ultrasonography images by hybrid method with respect to benign, malignant, and normal using mRMR.
        Comput Biol Med. 2021; 133104407
        • Zhuang Z.
        • Yang Z.
        • Raj A.N.J.
        • Wei C.
        • Jin P.
        • Zhuang S.
        Breast ultrasound tumor image classification using image decomposition and fusion based on adaptive multi-model spatial feature fusion.
        Comput Methods Programs Biomed. 2021; 208106221
        • Polchai N.
        • Sa-Nguanraksa D.
        • Numprasit W.
        • Thumrongtaradol T.
        • O-Charoenrat E.
        • O-Charoenrat P. P.
        A Comparison between the online prediction models cancermath and PREDICT as prognostic tools in thai breast cancer patients.
        Cancer Manag Res. 2020; 12: 5549-5559
      18. Annarita Fanizzi Domenico Pomarico Angelo Paradiso Samantha Bove Sergio Diotaiuti Vittorio Didonna et al. Predicting of Sentinel Lymph Node Status in Breast Cancer Patients with Clinically Negative Nodes: A Validation Study Cancers 13 2 352.

        • Russakovsky Olga
        • Deng Jia
        • Su Hao
        • Krause Jonathan
        • Satheesh Sanjeev
        • Ma Sean
        • et al.
        ImageNet large scale visual recognition challenge.
        Int J Comput Vis. 2015; 115: 211-252
        • Li J.
        • Wang P.
        • Li Y.
        • Zhou Y.
        • Liu X.
        • Luan K.
        Transfer learning of pre-trained Inception-v3 model for colorectal cancer lymph node metastasis classification.
        IEEE Int Conf Mechatronics Autom. 2018; 2018: 1650-1654
        • Elmuogy Samir
        • Hikal Noha A.
        • Hassan Esraa
        An efficient technique for CT scan images classification of COVID-19.
        J Intell Fuzzy Syst. 2021; 40: 5225-5238
        • Shadin N.S.
        • Sanjana S.
        • Lisa N.J.
        COVID-19 diagnosis from chest X-ray images using convolutional neural network (CNN) and InceptionV3.
        Int Conf Inf Technol. 2021; 2021: 799-804
        • Vijayan T.
        • Sangeetha M.
        • Karthik B.
        Efficient analysis of diabetic retinopathy on retinal fundus images using deep learning techniques with inception v3 architecture.
        J Green Eng. 2020; 10: 9615-9625