Advertisement

Artificial intelligence applications in medical imaging: A review of the medical physics research in Italy

      Highlights

      • A systematic search for papers on applications of AI to medical imaging in Italy was performed.
      • 168 research papers were selected 65% using machine learning, 35% deep learning.
      • A rapid increase of interest in AI was observed in the last years.
      • Further collaborations, initiatives and guidelines are needed to develop the research on AI on Imaging.

      Abstract

      Purpose

      To perform a systematic review on the research on the application of artificial intelligence (AI) to imaging published in Italy and identify its fields of application, methods and results.

      Materials and Methods

      A Pubmed search was conducted using terms Artificial Intelligence, Machine Learning, Deep learning, imaging, and Italy as affiliation, excluding reviews and papers outside time interval 2015–2020. In a second phase, participants of the working group AI4MP on Artificial Intelligence of the Italian Association of Physics in Medicine (AIFM) searched for papers on AI in imaging.

      Results

      The Pubmed search produced 794 results. 168 studies were selected, of which 122 were from Pubmed search and 46 from the working group. The most used imaging modality was MRI (44%) followed by CT(12%) ad radiography/mammography (11%). The most common clinical indication were neurological diseases (29%) and diagnosis of cancer (25%). Classification was the most common task for AI (57%) followed by segmentation (16%). 65% of studies used machine learning and 35% used deep learning. We observed a rapid increase of research in Italy on artificial intelligence in the last 5 years, peaking at 155% from 2018 to 2019.

      Conclusions

      We are witnessing an unprecedented interest in AI applied to imaging in Italy, in a diversity of fields and imaging techniques. Further initiatives are needed to build common frameworks and databases, collaborations among different types of institutions, and guidelines for research on AI.

      Keywords

      Introduction

      Accurate and early diagnosis and prognosis are essential in many fields of healthcare. Artificial Intelligence (AI) applied to medical images allows for automated disease detection, characterization of histology, stage, or subtype, and patient classification according to therapy outcome or prognosis. It also permits outlining particular regions in the images, quantifying organ volumes, and extracting features from the images which, combined with machine learning algorithms, leads to quantification of image properties or image classification.
      In recent years an unprecedented amount of digital imaging data has become available in medicine thanks to digitalization, affordable data storage, and improved imaging techniques. This leads to an unprecedented interest in applications of AI to images which has boosted research efforts of medical physicists (MPs) in Italy.
      These efforts however, are more efficient when there is communication, collaboration, sharing of knowledge and common intents in the MP community. For these purposes, the Italian Association of Medical Physics (AIFM) [

      Associazione Italiana di Fisica Medica (AIFM). AIFM n.d. https://www.fisicamedica.it/en/ (accessed March 1, 2021).

      ] which is composed of 1284 medical physicists, has established the AI for Medical Physics (AI4MP) [

      Associazione Italiana di Fisica Medica e Sanitaria. Big Data e Intelligenza Artificiale: il gruppo di lavoro AIFM. AIFM 2020. https://www.fisicamedica.it/i-temi-della-fisica-medica/big-data-e-intelligenza-artificiale/big-data-e-intelligenza-artificiale-il-gruppo-di-lavoro-aifm/ (accessed March 1, 2021).

      ] task-group.
      The purpose of this review, performed by the imaging subgroup of AI4MP, is to describe the status of the research in Italy on AI applied to imaging, by systematically analyzing research published in this field in Italy in the last 5 years. This work, besides providing an overview of the fields of application, methods for AI used and the results achieved, will serve to define future goals for the community of MP and facilitate research on AI applied to imaging by MPs in Italy.

      Machine learning and deep learning in imaging

      Machine learning (ML) is a field of AI algorithms (Fig. 1) which can recognize patterns in medical images by analyzing voxel intensity values or quantitative imaging features, called also “radiomic features”, by identifying their best combination and building a model for classification or regression [

      Lella E, Lombardi A, Amoroso N, Diacono D, Maggipinto T, Monaco A, et al. Machine learning and DWI brain communicability networks for Alzheimer’s disease detection. Appl Sci 2020;10. https://doi.org/10.3390/app10030934.

      ,
      • Amoroso N.
      • Diacono D.
      • Fanizzi A.
      • La Rocca M.
      • Monaco A.
      • Lombardi A.
      • et al.
      Deep learning reveals Alzheimer’s disease onset in MCI subjects: results from an international challenge.
      ,

      Alongi P, Laudicella R, Stefano A, Caobelli F, Comelli A, Vento A, et al. Choline PET/CT features to predict survival outcome in high risk prostate cancer restaging: a preliminary machine-learning radiomics study. Q J Nucl Med Mol Imaging Off Publ Ital Assoc Nucl Med AIMN Int Assoc Radiopharmacol IAR Sect Soc Of 2020. https://doi.org/10.23736/S1824-4785.20.03227-6.

      ,

      D’Amico NC, Sicilia R, Cordelli E, Tronchin L, Greco C, Fiore M, et al. Radiomics-based prediction of overall survival in lung cancer using different volumes-of-interest. Appl Sci 2020;10. https://doi.org/10.3390/app10186425.

      ]. By ML, image features can also be combined with variables from other sources, such as dose distribution from the radiotherapy treatment [
      • Avanzo M.
      • Pirrone G.
      • Vinante L.
      • Caroli A.
      • Stancanello J.
      • Drigo A.
      • et al.
      Electron density and biologically effective dose (BED) radiomics-based machine learning models to predict late radiation-induced subcutaneous fibrosis.
      ]) or clinical variables [
      • Ricciardi C.
      • Cantoni V.
      • Improta G.
      • Iuppariello L.
      • Latessa I.
      • Cesarelli M.
      • et al.
      Application of data mining in a cohort of Italian subjects undergoing myocardial perfusion imaging at an academic medical center.
      ] to improve accuracy of classification. Supervised ML is frequently employed in imaging for classification [
      • Avanzo M.
      • Pirrone G.
      • Vinante L.
      • Caroli A.
      • Stancanello J.
      • Drigo A.
      • et al.
      Electron density and biologically effective dose (BED) radiomics-based machine learning models to predict late radiation-induced subcutaneous fibrosis.
      ] when the output variable is categorical, and for regression [
      • Amoroso N.
      • La Rocca M.
      • Bellantuono L.
      • Diacono D.
      • Fanizzi A.
      • Lella E.
      • et al.
      Deep learning and multiplex networks for accurate modeling of brain age.
      ] tasks when the output variable is continuous.
      A large number of supervised ML algorithms is available, as shown in Table 1. Parametric algorithms make an assumption about the functional form of the function that map covariates to the outcome, then learn a finite number of coefficients for the function from the training data. These algorithms, by using a pre-selected function, are generally faster and easier to interpret. Generalized Linear Model with LASSO [
      • Amoroso N.
      • La Rocca M.
      • Bellantuono L.
      • Diacono D.
      • Fanizzi A.
      • Lella E.
      • et al.
      Deep learning and multiplex networks for accurate modeling of brain age.
      ,
      • Garau N.
      • Paganelli C.
      • Summers P.
      • Choi W.
      • Alam S.
      • Lu W.
      • et al.
      External validation of radiomics-based predictive models in low-dose CT screening for early lung cancer diagnosis.
      ,

      Botta F, Raimondi S, Rinaldi L, Bellerba F, Corso F, Bagnardi V, et al. Association of a CT-based clinical and radiomics score of non-small cell lung cancer (NSCLC) with lymph node status and overall survival. Cancers 2020;12. https://doi.org/10.3390/cancers12061432.

      ,
      • Pantoni L.
      • Marzi C.
      • Poggesi A.
      • Giorgio A.
      • De Stefano N.
      • Mascalchi M.
      • et al.
      Fractal dimension of cerebral white matter: A consistent feature for prediction of the cognitive performance in patients with small vessel disease and mild cognitive impairment.
      ], Ridge or Elastic Net penalty and Logistic Regression (LR) belong to parametric ML. Non-parametric algorithms, by using a number of parameters which is not limited, are usually slower and require larger dataset. These include classification and Regression Trees (CART) [
      • Nero C.
      • Ciccarone F.
      • Boldrini L.
      • Lenkowicz J.
      • Paris I.
      • Capoluongo E.D.
      • et al.
      Germline BRCA 1–2 status prediction through ovarian ultrasound images radiogenomics: a hypothesis generating study (PROBE study).
      ,
      • Barricelli B.R.
      • Casiraghi E.
      • Gliozzo J.
      • Huber V.
      • Leone B.E.
      • Rizzi A.
      • et al.
      ki67 nuclei detection and ki67-index estimation: a novel automatic approach based on human vision modeling.
      ], K-Nearest Neighbours (KNN) [
      • Fanizzi A.
      • Losurdo L.
      • Basile T.M.A.
      • Bellotti R.
      • Bottigli U.
      • Delogu P.
      • et al.
      Fully automated support system for diagnosis of breast cancer in contrast-enhanced spectral mammography images.
      ,

      Lombardi A, Amoroso N, Diacono D, Monaco A, Tangaro S, Bellotti R. Extensive evaluation of morphological statistical harmonization for brain age prediction. Brain Sci 2020;10:10.3390/brainsci10060364.

      ], and Support Vector Machines (SVM). SVM, based on finding a hyperplane that best divides the data into two classes in the feature space, is among most popular ML algorithm and is employed for both classification [
      • Avanzo M.
      • Pirrone G.
      • Vinante L.
      • Caroli A.
      • Stancanello J.
      • Drigo A.
      • et al.
      Electron density and biologically effective dose (BED) radiomics-based machine learning models to predict late radiation-induced subcutaneous fibrosis.
      ,
      • Garau N.
      • Paganelli C.
      • Summers P.
      • Choi W.
      • Alam S.
      • Lu W.
      • et al.
      External validation of radiomics-based predictive models in low-dose CT screening for early lung cancer diagnosis.
      ,
      • Spera G.
      • Retico A.
      • Bosco P.
      • Ferrari E.
      • Palumbo L.
      • Oliva P.
      • et al.
      Evaluation of altered functional connections in male children with autism spectrum disorders on multiple-site data optimized with machine learning.
      ,

      Retico A, Giuliano A, Tancredi R, Cosenza A, Apicella F, Narzisi A, et al. The effect of gender on the neuroanatomy of children with autism spectrum disorders: a support vector machine case-control study. Mol Autism 2016;7:5-015-0067-3. eCollection 2016. https://doi.org/10.1186/s13229-015-0067-3.

      ,
      • Moccia S.
      • Mattos L.S.
      • Patrini I.
      • Ruperti M.
      • Poté N.
      • Dondero F.
      • et al.
      Computer-assisted liver graft steatosis assessment via learning-based texture analysis.
      ,
      • Galli M.
      • Zoppis I.
      • De Sio G.
      • Chinello C.
      • Pagni F.
      • Magni F.
      • et al.
      A support vector machine classification of thyroid bioptic specimens using MALDI-MSI data.
      ] and regression [

      Lombardi A, Amoroso N, Diacono D, Monaco A, Tangaro S, Bellotti R. Extensive evaluation of morphological statistical harmonization for brain age prediction. Brain Sci 2020;10:10.3390/brainsci10060364.

      ]. Stochastic search algorithms were developed in an effort to imitate the mechanics of natural selection and natural genetics [
      • Militello C.
      • Vitabile S.
      • Rundo L.
      • Russo G.
      • Midiri M.
      • Gilardi M.C.
      A fully automatic 2D segmentation method for uterine fibroid in MRgFUS treatment evaluation.
      ].
      Table 1Summary of research papers published in Italy in years 2015–2020 using MLapplied to imaging. For each paper, the first author, year of publication, image modality, software and algorithms used for feature selection and model building, methods for model evaluation and validation, and purpose of the study are reported.
      RefAuthorYearImagingPlatformClassification/regressionType of featuresOther variablesDimensionality reductionMLEvaluation metricsValidationIndication

      Lella E, Lombardi A, Amoroso N, Diacono D, Maggipinto T, Monaco A, et al. Machine learning and DWI brain communicability networks for Alzheimer’s disease detection. Appl Sci 2020;10. https://doi.org/10.3390/app10030934.

      Lella E., et al.2020MRIRClassificationGraph communicability of connectivity network//SVM, ANN, RFACC, AUC, sensitivity, specificity10-fold CVAlzheimer disease detection

      Lombardi A, Amoroso N, Diacono D, Monaco A, Tangaro S, Bellotti R. Extensive evaluation of morphological statistical harmonization for brain age prediction. Brain Sci 2020;10:10.3390/brainsci10060364.

      Lombardi A., et al.2020MRIRRegressionRadiomic/SVM-RFESVR, LASSO, RFMAE, coefficient of determination10-fold CVBrain age prediction
      • Pantoni L.
      • Marzi C.
      • Poggesi A.
      • Giorgio A.
      • De Stefano N.
      • Mascalchi M.
      • et al.
      Fractal dimension of cerebral white matter: A consistent feature for prediction of the cognitive performance in patients with small vessel disease and mild cognitive impairment.
      Pantoni L., et al.2019MRIPythonRegressionFractal dimension//LassoPearson R,nested 10-fold CVSmall vessel disease, cognitive impairment
      • Spera G.
      • Retico A.
      • Bosco P.
      • Ferrari E.
      • Palumbo L.
      • Oliva P.
      • et al.
      Evaluation of altered functional connections in male children with autism spectrum disorders on multiple-site data optimized with machine learning.
      Spera G., et al.2019MRIMatlab fitsvm, RapidminerClassificationFunctional Connectivity Measures//L-SVMAUCLOOCVAutism spectrum disorders

      Retico A, Giuliano A, Tancredi R, Cosenza A, Apicella F, Narzisi A, et al. The effect of gender on the neuroanatomy of children with autism spectrum disorders: a support vector machine case-control study. Mol Autism 2016;7:5-015-0067-3. eCollection 2016. https://doi.org/10.1186/s13229-015-0067-3.

      Retico A., et al.2018MRISVM-Light software packageClassificationIntensity values in gray matter/RFESVMAUCLPOCVAutism spectrum disorders

      Battineni G, Chintalapudi N, Amenta F, Traini E. A Comprehensive Machine-Learning Model Applied to Magnetic Resonance Imaging (MRI) to Predict Alzheimer’s Disease (AD) in Older Subjects. J Clin Med 2020;9:10.3390/jcm9072146.

      Battineni G., et al2020MRI/ClassificationMRI-based featuresDemographicWrappingANN, SVM, NB, KNNAUC10-fold CVAlzheimer diagnosis
      • Inglese P.
      • Amoroso N.
      • Boccardi M.
      • Bocchetta M.
      • Bruno S.
      • Chincarini A.
      • et al.
      Multiple RF classifier for the hippocampus segmentation: method and validation on EADC-ADNI Harmonized Hippocampal Protocol.
      Inglese P., et al.2015MRIMatlabSegmentationVoxel by voxel//RFError, precision, Recall, DSC10-fold CV + external validationHyppocampus segmentation
      • Tangaro S.
      • Amoroso N.
      • Brescia M.
      • Cavuoti S.
      • Chincarini A.
      • Errico R.
      • et al.
      Feature selection based on machine learning in MRIs for hippocampal segmentation.
      Tangaro2015MRI/SegmentationRadiomic voxel-wise/Sequential, Kolmogorov-Smirnof, RFNBDSCCVHippocampal segmentation
      • Ferraro P.M.
      • Agosta F.
      • Riva N.
      • Copetti M.
      • Spinelli E.G.
      • Falzone Y.
      • et al.
      Multimodal structural MRI in the diagnosis of motor neuron diseases.
      Ferraro PM., et al2017MRISASClassificationDigital tractography measures/RFACCvalidation colorMotor neuron disease
      • Vai B.
      • Parenti L.
      • Bollettini I.
      • Cara C.
      • Verga C.
      • Melloni E.
      • et al.
      Predicting differential diagnosis between bipolar and unipolar depression with multiple kernel learning on multimodal structural neuroimaging.
      Vai B., et al.2020MRI T1W DTIPRoNTo softwareClassificationRadiomicTract-basedl Statistics,Voxel-based morphometry/MKL, SVMROC AUC, sensitivity, specificity, confusion matrix10-folds nested CVsDiagnosis of depression
      Neurological
      • Retico A.
      • Bosco P.
      • Cerello P.
      • Fiorina E.
      • Chincarini A.
      • Fantacci M.E.
      Predictive models based on support vector machines: whole-brain versus regional analysis of structural MRI in the Alzheimer’s disease.
      Retico2015MRISVM-LightClassificationVoxels/RFESVMAUC20-fold CVAlzheimer
      • Amoroso N.
      • La Rocca M.
      • Monaco A.
      • Bellotti R.
      • Tangaro S.
      Complex networks reveal early MRI markers of Parkinson’s disease.
      Amoroso N., et al.2018MRIR package randomForestClassificationVoxel-wise/RFSVMSensitivity, Specificity, AUC10-fold CVParkinson’s disease
      • Maggipinto T.
      • Bellotti R.
      • Amoroso N.
      • Diacono D.
      • Donvito G.
      • Lella E.
      • et al.
      DTI measurements for Alzheimer’s classification.
      Maggipinto T, et al.2017MRIMatlabClassificationVoxel-wise/RelieffRFACC, AUC5-fold CVAlzhaimer
      • Morisi R.
      • Manners D.N.
      • Gnecco G.
      • Lanconelli N.
      • Testa C.
      • Evangelisti S.
      • et al.
      Multi-class parkinsonian disorders classification with quantitative MR markers and graph-based features using support vector machines.
      Morisi2018MRIMatlabClassificationRadiomic/RankingSVMAUCLOOCVParkinson disorder
      • Bandini A.
      • Orlandi S.
      • Escalante H.J.
      • Giovannelli F.
      • Cincotta M.
      • Reyes-Garcia C.A.
      • et al.
      Analysis of facial expressions in parkinson’s disease through video-based automatic methods.
      Bandini2016RGB Videos/ClassificationEuclidean distances between facial features//SVMConfusion matrixCVParkinson
      • Peruzzo D.
      • Arrigoni F.
      • Triulzi F.
      • Righini A.
      • Parazzini C.
      • Castellani U.
      A framework for the automatic detection and characterization of brain malformations: validation on the corpus callosum.
      Peruzzo D., et al.2016MRI/ClassificationRadiomic/regularized discriminative directionMultiple Kernel SVM (rule-based method)AUC ROC, ACC, Sensitivity, Specificity, PrecisionLOOCVBrain malformations
      • Nanni L.
      • Brahnam S.
      • Salvatore C.
      • Castiglioni I.
      Alzheimer’s Disease Neuroimaging Initiative. Texture descriptors and voxels for the early diagnosis of Alzheimer’s disease.
      Nanni2019MRIMatlabClassificationRadiomic/Different methodsSVMAccuracy, AUCCVAlzheimer disease diagnosis
      • Nanni L.
      • Lumini A.
      • Zaffonato N.
      Ensemble based on static classifier selection for automated diagnosis of mild cognitive impairment.
      Nanni2018MRIMatlabClassificationRadiomic/Mutual information, othersSVMAccuracyCV on public datasetAlzheimer disease
      • Bertacchini F.
      • Rizzo R.
      • Bilotta E.
      • Pantano P.
      • Luca A.
      • Mazzuca A.
      • et al.
      Mid-sagittal plane detection for advanced physiological measurements in brain scans.
      Bertacchini2019MRI/////K-means clustering//Identification of mid-plane for neurooncological diseases
      • Lombardi A.
      • Guaragnella C.
      • Amoroso N.
      • Monaco A.
      • Fazio L.
      • Taurisano P.
      • et al.
      Modelling cognitive loads in schizophrenia by means of new functional dynamic indexes.
      Lombardi A., et al.2019MRIC++ LibSVMClassificationMultilayer Graph, Connectivity matrix/SVM-RFESVMACC, TPR, TNR10 times repeated 10-fold CVSchizophrenia
      • Fasano F.
      • Mitolo M.
      • Gardini S.
      • Venneri A.
      • Caffarra P.
      • Pazzaglia F.
      Combining structural magnetic resonance imaging and visuospatial tests to classify mild cognitive impairment.
      Fasano F., et al.2018MRIMatlabClassificationVoxel-wise//SVMSensitivity, Specificity, ACCLOOCVMild cognitive impairment
      • Squarcina L.
      • Castellani U.
      • Bellani M.
      • Perlini C.
      • Lasalvia A.
      • Dusi N.
      • et al.
      Classification of first-episode psychosis in a large cohort of patients using support vector machine and multiple kernel learning techniques.
      Squarcina L., et al2017MRI/Classificationmean ROI thicknesses//SVM, KNNACCLOOCVPsychosis
      • Vasta R.
      • Cerasa A.
      • Sarica A.
      • Bartolini E.
      • Martino I.
      • Mari F.
      • et al.
      The application of artificial intelligence to understand the pathophysiological basis of psychogenic nonepileptic seizures.
      Vasta2018MRIRClassificationMorphologicalClinicalRFRFAccuracyBootstrapPsychogenic nonepilectic seizures
      • Cerasa A.
      • Castiglioni I.
      • Salvatore C.
      • Funaro A.
      • Martino I.
      • Alfano S.
      • et al.
      Biomarkers of eating disorders using support vector machine analysis of structural neuroimaging data: preliminary results.
      Cerasa2015MRIMAtlabClassificationVoxel values/PCASVMAccuracy20-fold CVEating disorders
      • Nanni L.
      • Brahnam S.
      • Salvatore C.
      • Castiglioni I.
      Alzheimer’s Disease Neuroimaging Initiative. Texture descriptors and voxels for the early diagnosis of Alzheimer’s disease.
      Nanni2019MRIMatlabClassificationRadiomic/Different methodsSVMAccuracy, AUCCVAlzheimer disease diagnosis
      • Salvatore C.
      • Cerasa A.
      • Castiglioni I.
      MRI characterizes the progressive course of AD and predicts conversion to Alzheimer’s Dementia 24 months before probable diagnosis.
      Salvatore2018MRIMatlabClassificationRadiomic/PCASVMAccuracy5-fold CVAlzheimer
      • Nigro S.
      • Barbagallo G.
      • Bianco M.G.
      • Morelli M.
      • Arabia G.
      • Quattrone A.
      • et al.
      Track density imaging: a reliable method to assess white matter changes in Progressive Supranuclear Palsy with predominant parkinsonism.
      Nigro2019MRI/ClassificationVoxel-wise//SVM//White matter changes in Parkinson
      • Kia S.M.
      • Pedregosa F.
      • Blumenthal A.
      • Passerini A.
      Group-level spatio-temporal pattern recovery in MEG decoding using multi-task joint feature learning.
      Kia2017Magnetoencephalography (MEG)Matlab/Voxel values//Multi-feature learning/bootstrapBrain maps
      • Tangaro S.
      • Fanizzi A.
      • Amoroso N.
      • Bellotti R.
      Alzheimer’s disease neuroimaging initiative. A fuzzy-based system reveals Alzheimer’s disease onset in subjects with Mild cognitive impairment.
      Tangaro2017MRILibSVMClassificationMRI measurements/StatisticalFLA, SVMAUCCVAlzheimer
      • Pagani M.
      • Nobili F.
      • Morbelli S.
      • Arnaldi D.
      • Giuliani A.
      • Öberg J.
      • et al.
      Early identification of MCI converting to AD: a FDG PET study.
      Pagani2017PETMatlabClassificationIntensities of Volumes of interest/StepwiseSVMAccuracyCVAlzheimer
      • Salvatore C.
      • Castiglioni I.
      A wrapped multi-label classifier for the automatic diagnosis and prognosis of Alzheimer’s disease.
      Salvatore C2018MRIMatlabClassificationMRI featuresClinicalFisherSVMAccuracy5-fold CVAlzheimer
      • Previtali F.
      • Bertolazzi P.
      • Felici G.
      • Weitschek E.
      A novel method and software for automatically classifying Alzheimer’s disease patients by magnetic resonance imaging analysis.
      Previtali2017MRIWekaClassificationImage space and intensity features/Sparse regressionSVMAccuracy10-fold CVAlzheimer disease
      • Castellazzi G.
      • Cuzzoni M.G.
      • Cotta Ramusino M.
      • Martinelli D.
      • Denaro F.
      • Ricciardi A.
      • et al.
      A machine learning approach for the differential diagnosis of alzheimer and vascular dementia fed by MRI selected features.
      Castellazzi G., et al.2020MRIMatlabclassificationMachine Learning/ReliefFANN, SVM, ANFISACC, sensitivity, specificity, precision, NPV, AUC10-fold CV, 100 bootstrapsAlzheimer and vascular dementia diagnosis
      • Salvatore C.
      • Cerasa A.
      • Battista P.
      • Gilardi M.C.
      • Quattrone A.
      • Castiglioni I.
      • et al.
      Magnetic resonance imaging biomarkers for the early diagnosis of Alzheimer’s disease: a machine learning approach.
      Salvatore2015MRIMatlabClassificationRadiomic/PCA, FisherSVMAccuracyCVEarly diagnosis of Alzheimer
      • Romeo V.
      • Maurea S.
      • Cuocolo R.
      • Petretta M.
      • Mainenti P.P.
      • Verde F.
      • et al.
      Characterization of adrenal lesions on unenhanced MRI using texture analysis: a machine-learning approach.
      Romeo2017MRIWekaClassificationRadiomic/SequentialSVMAccuracyLOOCVDiagnosis of adrenal benign lesions
      • Lombardi A.
      • Amoroso N.
      • Diacono D.
      • Monaco A.
      • Logroscino G.
      • De Blasi R.
      • et al.
      Association between structural connectivity and generalized cognitive spectrum in Alzheimer’s disease.
      Lombardi A et al.2020DTIMatlabRegressionConnectivity features/PCALASSOMutual correlation coefficient10-fold CVCognitive spectrum in Alzheimer
      • De Carli F.
      • Nobili F.
      • Pagani M.
      • Bauckneht M.
      • Massa F.
      • Grazzini M.
      • et al.
      Accuracy and generalization capability of an automatic method for the detection of typical brain hypometabolism in prodromal Alzheimer disease.
      De Carli F., et al.2019PETMatlabClassificationSUV/StepwiseSVMAUC, ACC, Younden index, TPR, TNR, positive and negative likelihoodratio25-fold CV, training and independent test setAlzheimer disease detection
      • Sacca V.
      • Sarica A.
      • Novellino F.
      • Barone S.
      • Tallarico T.
      • Filippelli E.
      • et al.
      Evaluation of machine learning algorithms performance for the prediction of early multiple sclerosis from resting-state FMRI connectivity data.
      Sacca V., et al.2019MRIMatlab, RClassificationfMRI components/RF, recursivefeature eliminationRF, SVM, NB, KNN, ANNACC5-fold CVMultiple sclerosis
      • Retico A.
      • Gori I.
      • Giuliano A.
      • Muratori F.
      • Calderoni S.
      One-class support vector machines identify the language and default mode regions as common patterns of structural alterations in young children with autism spectrum disorders.
      Retico A., et al.2016MRISVM-Light software packageClassificationVoxel values//SVMACC, AUCLOOCVAutism spectrum disorders
      • Castaldi E.
      • Aagten-Murphy D.
      • Tosetti M.
      • Burr D.
      • Morrone M.C.
      Effects of adaptation on numerosity decoding in the human brain.
      Castaldi2016fMRIMatlab, LibSVMClassificationBOLD response voxels/RegularizationSVMAccuracyBootstrap CVBrain activity
      • Marzi C.
      • Giannelli M.
      • Tessa C.
      • Mascalchi M.
      • Diciotti S.
      Toward a more reliable characterization of fractal properties of the cerebral cortex of healthy subjects during the lifespan.
      C., et al.2020MRIMatlab, PythonRegressionFractal dimension//Regression modelsPearson coefficientlinear regression CV loopCerebral cortex in healthy subjects
      • Rundo L.
      • Militello C.
      • Tangherloni A.
      • Russo G.
      • Vitabile S.
      • Gilardi M.C.
      • et al.
      NeXt for neuro-radiosurgery: a fully automatic approach for necrosis extraction in brain tumor MRI using an unsupervised machine learning technique.
      Rundo L. et al.2018MRIMatlabSegmentationIntensity values//Cellular AutomataDSC, JI, Sensitivity, Specificity, FPR, FNR, MAD, MAxD, HDValidation setBrain Necrosis
      • Amoroso N.
      • Errico R.
      • Bruno S.
      • Chincarini A.
      • Garuccio E.
      • Sensi F.
      • et al.
      Hippocampal unified multi-atlas network (HUMAN): protocol and scale validation of a novel segmentation tool.
      Amoroso2015MRIMatlabSegmentationRadiomic features, voxel values//ANNDSCADNI dataset multicentricHippocampus segmentation, Alzheimer disease
      • Garau N.
      • Paganelli C.
      • Summers P.
      • Choi W.
      • Alam S.
      • Lu W.
      • et al.
      External validation of radiomics-based predictive models in low-dose CT screening for early lung cancer diagnosis.
      Garau N., et al.2020CTMatlabClassificationRadiomic/Correlation-based ierarchical clustering, ReliefFSVM LASSO, ANNAUC10-fold CVLung cancer diagnosis
      • Fanizzi A.
      • Losurdo L.
      • Basile T.M.A.
      • Bellotti R.
      • Bottigli U.
      • Delogu P.
      • et al.
      Fully automated support system for diagnosis of breast cancer in contrast-enhanced spectral mammography images.
      Fanizzi A., et al.2019CESMMatlabClassificationRadiomic/Embedded/filterRFAUC, ACC, Sensitivity, Specificity, MCC10-fold CVCancer diagnosis
      • Chauvie S.
      • De Maggi A.
      • Baralis I.
      • Dalmasso F.
      • Berchialla P.
      • Priotto R.
      • et al.
      Artificial intelligence and radiomics enhance the positive predictive value of digital chest tomosynthesis for lung cancer detection within SOS clinical trial.
      Chauvie S., et al.2020Chest Tomosynthesis/ClassificationRadiomicSemanticBackward with GLMRF, NNET, LRACC, Sensitivity10-fold cross valdidationCancer diagnosis
      • Crisi G.
      • Filice S.
      Predicting MGMT Promoter methylation of glioblastoma from dynamic susceptibility contrast perfusion: A radiomic approach.
      Crisi G., et al.2020MRIWekaClassificationRadiomic//ANNSensitivity, Specificity, AUC10-fold cross validationGlioblastoma methylation
      • Brunese L.
      • Mercaldo F.
      • Reginelli A.
      • Santone A.
      An ensemble learning approach for brain cancer detection exploiting radiomic features.
      Brunese L., et al.2020MRIPythonClassificationRadiomic/Hypothesis testingRFACCTesting and validation datasetCancer diagnosis
      • Basile T.M.A.
      • Fanizzi A.
      • Losurdo L.
      • Bellotti R.
      • Bottigli U.
      • Dentamaro R.
      • et al.
      Microcalcification detection in full-field digital mammograms: a fully automated computer-aided system.
      Basile TMA., et al2019Mammograms/ClassificationHough Transform//ClusteringSensitivity, Specificity10-fold CVMicrocalcification detection
      • Gallivanone F.
      • Interlenghi M.
      • Canervari C.
      • Castiglioni I.
      A fully automatic, threshold-based segmentation method for the estimation of the Metabolic Tumor Volume from PET images: validation on 3D printed anthropomorphic oncological lesions.
      Gallivanone F., et al.2016PETMatlabSegmentationVoxel values//k-meansMean percentage differences in phantom3D phantomMetabolic volume

      Fanizzi A, Basile TMA, Losurdo L, Bellotti R, Bottigli U, Dentamaro R, et al. A machine learning approach on multiscale texture analysis for breast microcalcification diagnosis. BMC Bioinformatics 2020;21:91-020-3358–4. https://doi.org/10.1186/s12859-020-3358-4.

      Fanizzi A., et al.2020MammogramsMatlabSegmentationRadiomic/Embedded, wrappingRFAUC, ACC10 fold CVMicrocalcifications segmentation
      • Militello C.
      • Rundo L.
      • Vitabile S.
      • Russo G.
      • Pisciotta P.
      • Marletta F.
      • et al.
      Gamma Knife treatment planning: MR brain tumor segmentation and volume measurement based on unsupervised Fuzzy C-Means clustering.
      Militello C. et al.2015MRIMatlabSegmentationIntensity values//FCMDSC, JI, Sensitivity, Specificity, FPR, FNR, FRR, MAD, MaxD, HDunsupervised, no trainingBrain tumor segmentation
      • Rundo L.
      • Militello C.
      • Russo G.
      • Garufi A.
      • Vitabile S.
      • Gilardi M.C.
      • et al.
      Automated prostate gland segmentation based on an unsupervised fuzzy C-means clustering technique using multispectral T1w and T2w MR imaging.
      Rundo L. et al.2017MRIMatlabSegmentationIntensity value//multi-spectral FCMDSC, JI, Sensitivity, Specificity, FPR, FNR, MAD, MaxD, HDLOOCVProstate segmentation
      • Rundo L.
      • Stefano A.
      • Militello C.
      • Russo G.
      • Sabini M.G.
      • D’Arrigo C.
      • et al.
      A fully automatic approach for multimodal PET and MR image segmentation in gamma knife treatment planning.
      Rundo L. et al.2017MRI, PETMatlabSegmentationIntensity valuesPET/MRI fusion/FCM, Random-walksDSC, HD, MHDExpert radiologistBrain tumor segmentation
      • Comelli A.
      • Stefano A.
      • Russo G.
      • Bignardi S.
      • Sabini M.G.
      • Petrucci G.
      • et al.
      K-nearest neighbor driving active contours to delineate biological tumor volumes.
      Comelli A., et al.2019PETMatlabSegmentationBTV//LAC-KNNTPR, TNR, Precision, ACC, error, DSC, HD5-Fold CVDifferent cancer types

      Giannini V, Rosati S, Regge D, Balestra G. Texture Features and Artificial Neural Networks: A Way to Improve the Specificity of a CAD System for Multiparametric MR Prostate Cancer. In: Kyriacou E, Christofides S, Pattichis CS, editors., Cham: Springer International Publishing; 2016, p. 296–301.

      Giannini V., et al.2016MRI/Classification, SegmentationRadiomic//ANNConfusion MatrixValidation setProstate cancer detection
      • Bevilacqua V.
      • Brunetti A.
      • Guerriero A.
      • Trotta G.F.
      • Telegrafo M.
      • Moschetta M.
      A performance comparison between shallow and deeper neural networks supervised classification of tomosynthesis breast lesions images.
      Bevilacqua V., et al.2019TomosynthesisMatlabClassificationDeep Learning//SVM, KNN, NBA, DT, LDAACC, Specificity, SensitivityValidation setBreast lesions
      • Gitto S.
      • Cuocolo R.
      • Albano D.
      • Chianca V.
      • Messina C.
      • Gambino A.
      • et al.
      MRI radiomics-based machine-learning classification of bone chondrosarcoma.
      Gitto S., et al.2020MRIPyRadiomics, WekaclassificationRadiomic/RFAdaBoostAUC, F-score, Matthews corr func,10-fold CV, test setBone chondrosarcoma diagnosis
      • Gallivanone F.
      • Cava C.
      • Corsi F.
      • Bertoli G.
      • Castiglioni I.
      In silico approach for the Definition of radiomiRNomic signatures for breast cancer differential diagnosis.
      Gallivanone F., et al.2019CE-MRIR-package e1071ClassificationRadiomic, miRNomicmicro RNA/SVMAUCvalidation seBreast Cancer diagnosis
      • Ugga L.
      • Cuocolo R.
      • Solari D.
      • Guadagno E.
      • D’Amico A.
      • Somma T.
      • et al.
      Prediction of high proliferative index in pituitary macroadenomas using MRI-based radiomics and machine learning.
      Ugga2019MRIWekaClassificationRadiomic/Relieff, ranking, othersKNNSensitivity, specificity, precisionTest setProliferation of pituitary tumor
      • Lippi M.
      • Gianotti S.
      • Fama A.
      • Casali M.
      • Barbolini E.
      • Ferrari A.
      • et al.
      Texture analysis and multiple-instance learning for the classification of malignant lymphomas.
      Lippi M., et al.2020PETMatlab, PythonClassificationRadiomic/RFSVM, RFPrecision, Sensitivity, ACCLOOCVLymphoma diagnosis
      • Losurdo L.
      • Fanizzi A.
      • Basile T.M.A.
      • Bellotti R.
      • Bottigli U.
      • Dentamaro R.
      • et al.
      Radiomics analysis on contrast-enhanced spectral mammography images for breast cancer diagnosis: A pilot study.
      Losurdo L., et al.2019CESMMatlabClassificanRadioomics/Principal Component AnalysisSVMACC, Sensibility, Specificity100-fold CVBreast Cancer diagnosis
      • D’Amico N.C.
      • Grossi E.
      • Valbusa G.
      • Rigiroli F.
      • Colombo B.
      • Buscema M.
      • et al.
      A machine learning approach for differentiating malignant from benign enhancing foci on breast MRI.
      D’Amico et al2020MRIC++, ITKClassificationRadiomic/Training with input selection and testing (TWIST)KNNSens, spec, accCVBreast cancer benign/malignant
      • Taroni P.
      • Paganoni A.M.
      • Ieva F.
      • Pifferi A.
      • Quarto G.
      • Abbate F.
      • et al.
      Non-invasive optical estimate of tissue composition to differentiate malignant from benign breast lesions: A pilot study.
      Taroni2017Optical imaging/ClassificationTissue absorption map//EMLSensitivity, specificity, AUC/Breast cancer diagnosis
      Cancer diagnosis and characterization
      • Astorino A.
      • Fuduli A.
      • Veltri P.
      • Vocaturo E.
      Melanoma detection by means of multiple instance learning.
      Astorino A., et al.2020Dermosco-pic imagesMatlabClassificationmelanomas//Multiplenstance LearningACC, Sensitivity, Specificity5-fold CV, 10-fold CV, LOOCVMelanoma detection
      • Castaldo R.
      • Pane K.
      • Nicolai E.
      • Salvatore M.
      • Franzese M.
      The impact of normalization approaches to automatically detect radiogenomic phenotypes characterizing breast cancer receptors status.
      Castaldo2016DCE-MRIMatlabClassificationRadiomic//NB, SVM, RFSens, spec, acc, AUCBootstrapBreast cancer receptor status
      • Chiappa V.
      • Bogani G.
      • Interlenghi M.
      • Salvatore C.
      • Bertolina F.
      • Sarpietro G.
      • et al.
      The Adoption of Radiomics and machine learning improves the diagnostic processes of women with Ovarian MAsses (the AROMA pilot study).
      Chiappa V., et al.2020USTRACE4ClassificationRadiomic//SVMACC, Sensitivity, Specificity, AUC, Standard Deviation10-fold CVMalignancy of ovarian masses
      • Granata V.
      • Fusco R.
      • Risi C.
      • Ottaiano A.
      • Avallone A.
      • De Stefano A.
      • et al.
      Diffusion-weighted MRI and diffusion kurtosis imaging to detect RAS mutation in colorectal liver metastasis.
      Granata V, et al2020MRIMatlabClassificationDWI- and DKI-derived parametersNeighbourhood component analysis (NCA)Linear classifier, DTACCTesting setMutation in liver metastasis
      • Stanzione A.
      • Cuocolo R.
      • Cocozza S.
      • Romeo V.
      • Persico F.
      • Fusco F.
      • et al.
      Detection of extraprostatic extension of cancer on biparametric MRI combining texture analysis and machine learning: preliminary results.
      Stanzione2019MRIWekaClassificationRadiomic/Different feature selection methodsNB, SVM, othersAccuracy, AUC10-fold CVProstate ca extracapsular
      • Stanzione A.
      • Cuocolo R.
      • Del Grosso R.
      • Nardiello A.
      • Romeo V.
      • Travaglino A.
      • et al.
      Deep myometrial infiltration of endometrial cancer on MRI: A radiomics-powered machine learning pilot study.
      Stanzione2020MRIPython, WekaClassificationRadiomic/RFEMLSens, spec, acc, AUC10–fold CVDiagnosis of endometrial cancer infiltration
      • Giambelluca D.
      • Cannella R.
      • Vernuccio F.
      • Comelli A.
      • Pavone A.
      • Salvaggio L.
      • et al.
      PI-RADS 3 lesions: role of prostate MRI texture analysis in the identification of prostate cancer.
      Giambelluca D., et al2019MRIMatlabClassificationRadiomicT2WI, ADC/GLM, DAROC, AUC10-fold CVProstate cancer identification
      • Gugliandolo S.G.
      • Pepa M.
      • Isaksson L.J.
      • Marvaso G.
      • Raimondi S.
      • Botta F.
      • et al.
      MRI-based radiomics signature for localized prostate cancer: a new clinical tool for cancer aggressiveness prediction? Sub-study of prospective phase II trial on ultra-hypofractionated radiotherapy (AIRC IG-13218).
      Gugliandolo S.G., et al.2020MRIMatlab, SASClassificationRadiomic/Stepwise FSLRAUCLOOCVProstate cancer aggressiveness
      • Stefano A.
      • Comelli A.
      • Bravatà V.
      • Barone S.
      • Daskalovski I.
      • Savoca G.
      • et al.
      A preliminary PET radiomics study of brain metastases using a fully automatic segmentation method.
      Stefano A., et al.2020PETMatlab CGITAClassificationRadiomic/correlation matrix,point-biserial correlation coefficientDATPR, TNR,NPV, Error, ACC, AUCSensitivity, Specificity5-fold CVBrain metastasis segmentation
      • Lopez Torres E.
      • Fiorina E.
      • Pennazio F.
      • Peroni C.
      • Saletta M.
      • Camarlinghi N.
      • et al.
      Large scale validation of the M5L lung CAD on heterogeneous CT datasets.
      Lopez Torres2015CTCommercial CADClassificationRadiomic//ANNSensitivityMulticentric public databasesLung cancer diagnosis
      • Romeo V.
      • Cuocolo R.
      • Ricciardi C.
      • Ugga L.
      • Cocozza S.
      • Verde F.
      • et al.
      Prediction of tumor grade and nodal status in oropharyngeal and oral cavity squamous-cell carcinoma using a radiomic approach.
      Romeo2020CTKNIMEClassificationRadiomic//KNN, NB, ANNAccuracy10-fold CVHead and neck grading, nodal status
      • Stanzione A.
      • Ricciardi C.
      • Cuocolo R.
      • Romeo V.
      • Petrone J.
      • Sarnataro M.
      • et al.
      MRI radiomics for the prediction of fuhrman grade in clear cell renal cell carcinoma: a machine learning exploratory study.
      Stanzione A., et al2020MRIKNIMEClassificationRadiomic/correlation, wrapperDT, EMLACC10-fold CVGrading renal cell carcinoma
      Other
      • Nero C.
      • Ciccarone F.
      • Boldrini L.
      • Lenkowicz J.
      • Paris I.
      • Capoluongo E.D.
      • et al.
      Germline BRCA 1–2 status prediction through ovarian ultrasound images radiogenomics: a hypothesis generating study (PROBE study).
      Nero C., et al2020USR, PythonClassificationRadiomics/pair-wise Pearson correlation test, forward features selectionSVM, EML,LRACCTesting and validation dataset, 5-fold CVBreast cancer susceptibility genes from ovarian US
      • Moccia S.
      • Mattos L.S.
      • Patrini I.
      • Ruperti M.
      • Poté N.
      • Dondero F.
      • et al.
      Computer-assisted liver graft steatosis assessment via learning-based texture analysis.
      Moccia2018RGB cameraMatlabClassificationRadiomic/PCASVMAccuracyLOOCVLiver transplant steatosis
      • Militello C.
      • Vitabile S.
      • Rundo L.
      • Russo G.
      • Midiri M.
      • Gilardi M.C.
      A fully automatic 2D segmentation method for uterine fibroid in MRgFUS treatment evaluation.
      Militello C. et al.2015MRIMatlabSegmentation and classificationIntensity values//FCMDSC, JI, Sensitivity, Specificityunsupervised, no validationUterine Fibroids

      Recenti M, Ricciardi C, Gìslason M, Edmunds K, Carraro U, Gargiulo P. Machine Learning Algorithms Predict Body Mass Index Using Nonlinear Trimodal Regression Analysis from Computed Tomography Scans. In: Henriques J, Neves N, de Carvalho P, editors. XV Mediterr. Conf. Med. Biol. Eng. Comput. – MEDICON 2019, Cham: Springer International Publishing; 2020, p. 839–46.

      Recenti M., et al2019CTPython ML library Scikit-LearnRegression, ClassificationImage descriptors of muscleClinicalRF, DT, EMLCoefficient of Determination (R2), JI8,12, 16, 18-fold CVBody mass index
      • Romeo V.
      • Ricciardi C.
      • Cuocolo R.
      • Stanzione A.
      • Verde F.
      • Sarno L.
      • et al.
      Machine learning analysis of MRI-derived texture features to predict placenta accreta spectrum in patients with placenta previa.
      Romeo V., et al.2019USKNIMEClassificationRadiomic//RF ,KNN, NB, ANNPrecision, Sensitivity, Specificity10-fold CVPlacenta accrete
      • Rosati S.
      • Franco P.
      • Fiandra C.
      • Arcadipane F.
      • Silvetti P.
      • Gallio E.
      • et al.
      Comparison of different classifiers to recognize active bone marrow from CT images.
      Rosati S., et al.2020CTSegmentationRadiomicGAKNN, DT, SVMDSCValidation setActive bone marrow
      • Rundo L.
      • Tangherloni A.
      • Cazzaniga P.
      • Nobile M.S.
      • Russo G.
      • Gilardi M.C.
      • et al.
      A novel framework for MR image segmentation and quantification by using MedGA.
      Rundo L. et al.2019MRIPythonSegmentation///Genetic AlgorithmMaximum absolute Distance, Hausdorff Distance (HD)/Uterine fibroids
      • Matos J.
      • Paparo F.
      • Mussetto I.
      • Bacigalupo L.
      • Veneziano A.
      • Perugin Bernardi S.
      • et al.
      Evaluation of novel coronavirus disease (COVID-19) using quantitative lung CT and clinical data: prediction of short-term outcome.
      Matos J., et al.2020CTRClassificationCT descriptorsClinicalSingle-feature AUCLR, SVM, DTACC, Sensibility, Specificity, PPV, NPV, AUCTesting and validation dataset,10-times repeated 5-fold CVCOVID-19 outcome prediction
      • Ulivieri F.M.
      • Piodi L.P.
      • Grossi E.
      • Rinaudo L.
      • Messina C.
      • Tassi A.P.
      • et al.
      The role of carboxy-terminal cross-linking telopeptide of type I collagen, dual x-ray absorptiometry bone strain and Romberg test in a new osteoporotic fracture risk evaluation: A proposal from an observational study.
      Ulivieri F.M., et al.2018dual x-ray absorptiometry/ClassificationQuantitative, qualitative bone markers//ANNSemantic Map/Osteoporosis
      • Cordelli E.
      • Maulucci G.
      • De Spirito M.
      • Rizzi A.
      • Pitocco D.
      • Soda P.
      A decision support system for type 1 diabetes mellitus diagnostics based on dual channel analysis of red blood cell membrane fluidity.
      Cordelli E., et al.2018dual channel red blood cells microscopy imagesMatlab + LIBSVMClassificationRadiomic/PCASVM, ANN, KNNACCLOOCVDiabetes
      Cardiovascular
      • Ricciardi C.
      • Cantoni V.
      • Improta G.
      • Iuppariello L.
      • Latessa I.
      • Cesarelli M.
      • et al.
      Application of data mining in a cohort of Italian subjects undergoing myocardial perfusion imaging at an academic medical center.
      Ricciardi C., et al.2020SPECTKnimeClassificationRadiomicClinical/NB,KNN,RF,GBACC, error, precision, recall, specificity, sensitivityK-fold CVCoronary arterial disease diagnosis

      Ricciardi C, Edmunds KJ, Recenti M, Sigurdsson S, Gudnason V, Carraro U, et al. Assessing cardiovascular risks from a mid-thigh CT image: a tree-based machine learning approach using radiodensitometric distributions. Sci Rep 2020;10:2863-020-59873–9. https://doi.org/10.1038/s41598-020-59873-9.

      Ricciardi C., et al.2020CTKNIMEClassificationRadiomic//RF, ADA-Boost, GBACC, Sensitivity, Specificity, Precision, AUC12-fold CVCardiovascular risk
      • Piras P.
      • Torromeo C.
      • Re F.
      • Evangelista A.
      • Gabriele S.
      • Esposito G.
      • et al.
      Left atrial trajectory impairment in hypertrophic cardiomyopathy disclosed by geometric morphometrics and parallel transport.
      Piras et al.2016Echocardiography/ClassificationShape, landmark coordinates, trajectories/FilteringSVMAccuracy, AUCLOOCVLeft Atrial trajectory impairment
      • Bellavia D.
      • Iacovoni A.
      • Agnese V.
      • Falletta C.
      • Coronnello C.
      • Pasta S.
      • et al.
      Usefulness of regional right ventricular and right atrial strain for prediction of early and late right ventricular failure following a left ventricular assist device implant: a machine learning approach.
      Bellavia et al.2020USR (CARET, GLMNET, E1071)ClassificationImage measurementsClinical, biochemical/NBAccuracy, AUCLOOCV, two italian centersRight ventricular failure
      • Cantoni V.
      • Green R.
      • Ricciardi C.
      • Assante R.
      • Zampella E.
      • Nappi C.
      • et al.
      A machine learning-based approach to directly compare the diagnostic accuracy of myocardial perfusion imaging by conventional and cadmium-zinc telluride SPECT.
      Cantoni et al2020SPECTKNIMEClassificationImage measurements//RF, KNNAccuracy10-fold CVCoronary artery disease
      • Maffei N.
      • Fiorini L.
      • Aluisio G.
      • D’Angelo E.
      • Ferrazza P.
      • Vanoni V.
      • et al.
      Hierarchical clustering applied to automatic atlas based segmentation of 25 cardiac sub-structures.
      Maffei N., et al.2020CTPythonSegmentation///HCDSC, Average HDValidaition setCardiac sub structures
      Pathology and microscopy
      • Barricelli B.R.
      • Casiraghi E.
      • Gliozzo J.
      • Huber V.
      • Leone B.E.
      • Rizzi A.
      • et al.
      ki67 nuclei detection and ki67-index estimation: a novel automatic approach based on human vision modeling.
      Barricelli et al.2019Histochemical imagesMAtlabClassificationPixel colors//Bayes, decision trees//Tumor protein ki67
      • Galli M.
      • Zoppis I.
      • De Sio G.
      • Chinello C.
      • Pagni F.
      • Magni F.
      • et al.
      A support vector machine classification of thyroid bioptic specimens using MALDI-MSI data.
      Galli et al.2016Cytological smear/ClassificationPixel values//SVMAccuracy, AUCTest set
      • Salvi M.
      • Molinari F.
      • Dogliani N.
      • Bosco M.
      Automatic discrimination of neoplastic epithelium and stromal response in breast carcinoma.
      Salvi M., et al.2019Histopathology imagesMatlabClassificationRadiomic//ANNPrecision, Recall, F Scorevalidation setCancer characterisation
      • Bizzego A.
      • Bussola N.
      • Chierici M.
      • Maggio V.
      • Francescatto M.
      • Cima L.
      • et al.
      Evaluating reproducibility of AI algorithms in digital pathology with DAPPER.
      Bizzego A., et al.2019Digital PathologyPhytonClassificationDeep Learning//SVM, RFconfusion matrix,ACC, MCC10 times repeated 5fold CVPathology
      • Militello C.
      • Rundo L.
      • Minafra L.
      • Cammarata F.P.
      • Calvaruso M.
      • Conti V.
      • et al.
      MF2C3: multi-feature fuzzy clustering to enhance cell colony detection in automated clonogenic assay evaluation.
      Militello C. et al2020cells culture imageMatlabSegmentationRadiomic//spatial FCMPearson’s correlation coefficientsurviving fractionCell colony detection
      Cancer prognosis and outcome prediction

      Alongi P, Laudicella R, Stefano A, Caobelli F, Comelli A, Vento A, et al. Choline PET/CT features to predict survival outcome in high risk prostate cancer restaging: a preliminary machine-learning radiomics study. Q J Nucl Med Mol Imaging Off Publ Ital Assoc Nucl Med AIMN Int Assoc Radiopharmacol IAR Sect Soc Of 2020. https://doi.org/10.23736/S1824-4785.20.03227-6.

      Alongi P., et al.2020PETMatlab CGITAClassificationRadiomic/NCADASensitivity, Specificity, ACC5-fold CVProstate cancer survival prediction
      • Avanzo M.
      • Pirrone G.
      • Vinante L.
      • Caroli A.
      • Stancanello J.
      • Drigo A.
      • et al.
      Electron density and biologically effective dose (BED) radiomics-based machine learning models to predict late radiation-induced subcutaneous fibrosis.
      Avanzo M., et al.2020CTMatlabClassificationRadiomicDosomicStepwise with GLMSVM, EML, NBSensitivity, specificity, AUC5-foldCVRT late fibrosis prediction

      Botta F, Raimondi S, Rinaldi L, Bellerba F, Corso F, Bagnardi V, et al. Association of a CT-based clinical and radiomics score of non-small cell lung cancer (NSCLC) with lymph node status and overall survival. Cancers 2020;12. https://doi.org/10.3390/cancers12061432.

      Botta F., et al.2020CTR, SASClassification, RegressionRadiomicClinicalOverall Concordance Correation Coefficient ANOVALASSOAUCValidation setLung cancer survival
      • D’Amico N.C.
      • Sicilia R.
      • Cordelli E.
      • Valbusa G.
      • Grossi E.
      • Zanetti I.B.
      • et al.
      Early radiomics experiences in predicting cyberknife response in acoustic neuroma.
      D’Amico N.C., et al.2019MRIMatlabIBEXClassificationRadiomicClinicalMann Whitney U testRFACC,precision, recall and AUC10-fold crossValidationCancer outcome prediction
      • Ferrari R.
      • Mancini-Terracciano C.
      • Voena C.
      • Rengo M.
      • Zerunian M.
      • Ciardiello A.
      • et al.
      MR-based artificial intelligence model to assess response to therapy in locally advanced rectal cancer.
      Ferrari R., et al.2019MRIPythonClassificationRadiomic/2-tail t-StudentRFAUC, ACC, Sensitivity, Specificyvalidation setCancer outcome prediction

      Schenone D, Lai R, Cea M, Rossi F, Torri L, Bignotti B, et al. Radiomics and artificial intelligence analysis of CT data for the identification of prognostic features in multiple myeloma. Proc.SPIE, vol. 11314, 2020. https://doi.org/10.1117/12.2548983.

      Schenone D., et al.2020CT3DSlicer + Slicer radiomics (pyradiomics library)UnsupervisedRadiomic//hard C-means clustering////Myeloma prognosis

      Talamonti C, Piffer S, Greto D, Mangoni M, Ciccarone A, Dicarolo P, et al. Radiomic and Dosiomic Profiling of Paediatric Medulloblastoma Tumours Treated with Intensity Modulated Radiation Therapy. In: Vento M, Percannella G, Colantonio S, Giorgi D, Matuszewski BJ, Kerdegari H, et al., editors., Cham: Springer International Publishing; 2019, p. 56–64.

      Talamonti C., et al.2019MRI, CT, Dose distribution imagesPyRadiomicsClassificationRadiomic, Dosiomic/RFESVM, ANNSensitivity, Specificity, ACC, AUCLOOCVMedulloblastoma prognosis
      • Pota M.
      • Scalco E.
      • Sanguineti G.
      • Farneti A.
      • Cattaneo G.M.
      • Rizzo G.
      • et al.
      Early prediction of radiotherapy-induced parotid shrinkage and toxicity based on CT radiomics and fuzzy classification.
      Pota M., et al.2017CTPythonClassificationRadiomic//FLA, NBACC, Sensitivity, Specificity10-fold CV, LOOCVToxicity after RT
      • Leccisotti L.
      • Manfrida S.
      • Barone R.
      • Ripani D.
      • Tagliaferri L.
      • Masiello V.
      • et al.
      The prognostic role of FDG PET/CT before combined radio-chemotherapy in anal cancer patients.
      Leccisotti L., et al.2020PETRRegressionPET SUV parametersClinicalBackward eliminationLRAUCTesting and validation dataset, 5-fold CVPrognosis of anal cancer
      Radiotherapy planning
      • Seregni M.
      • Paganelli C.
      • Lee D.
      • Greer P.B.
      • Baroni G.
      • Keall P.J.
      • et al.
      Motion prediction in MRI-guided radiotherapy based on interleaved orthogonal cine-MRI.
      Seregni M., et al.2016MRIMatlabRegressionTumor trajectories//Autoregressive linear prediction (AR), SVMTracking error RMSTesting and validation datasetMotion prediction in RT
      • Ermacora D.
      • Pesente S.
      • Pascoli F.
      • Raducci S.
      • Mauro R.
      • Rumeileh I.A.
      • et al.
      Automated computed tomography-ultrasound cross-modality 3-D contouring algorithm for prostate.
      Ermacora D., et al.2015CT-USCSegmentationShape, texture/GaborIn-house supervised MLmean distance to conformityExpert radiologist contoursProstate contouring for RT
      • Guidi G.
      • Maffei N.
      • Meduri B.
      • D’Angelo E.
      • Mistretta G.M.
      • Ceroni P.
      • et al.
      A machine learning tool for re-planning and adaptive RT: A multicenter cohort investigation.
      Guidi G., et al.2016CBCT, MVCTIron-Python, MatlabClassificationDose and volumes of ROIsART/SVMclinical validation of timepoint suggestioncross-correlation, independent setRT adaptive planning
      • Guidi G.
      • Maffei N.
      • Vecchi C.
      • Gottardi G.
      • Ciarmatori A.
      • Mistretta G.M.
      • et al.
      Expert system classifier for adaptive radiation therapy in prostate cancer.
      Guidi G., et al.2015CT, MVCTMatlabClassificationOAR Volume, DVH//SVM, K-MeansDSC, JIRT adaptivelanning
      • Guidi G.
      • Maffei N.
      • Vecchi C.
      • Ciarmatori A.
      • Mistretta G.M.
      • Gottardi G.
      • et al.
      A support vector machine tool for adaptive tomotherapy treatments: prediction of head and neck patients criticalities.
      Guidi G., et al.2016CT, MVCTMatlab, Python scripting in RayStationClassificationDose and volumes of ROIs//SVM, cluster analysisAUCexternal validationRT adaptivelanning
      (RF = Random Forest, EML = Ensemble Machine Learning, RR = Ridge Regression, NCA = Neighborhood Component Analysis, DA = Discriminant Analysis, LDA = Linear Discriminant Analysis, RFE = Recursive Feature Elimination, LR = Logistic Regression, GLM = Generalized Linear Model, FLA = Fuzzy Logic Analysis, NB = Naïve Bayes, HC = Hierarchical Clustering, LAC = Local Active Contour, KNN = k-Nearest Neighbor , DT = Decision Tree, FCM = Fuzzy C-means Clustering, LASSO = Least Absolute Shrinkage and Selection Operator, SVM = Support Vector Machine, SVR = Support Vector Regression, L-SVM = Linear Kernel SVM, ANN = Artificial Neural Network, GB = Gradient Boosting, MKL = Multiple Kernel Machine Learning, ANFIS = Adaptive Neuro-Fuzzy Inference System, GAN = Geneerative Adversarial Network.
      BTV = Biological Tumor Volume, SUV = Standardized Uptake Value.
      CT = Computed Tomography, CBCT = Cone Beam CT, MVCT = Mega-voltage CT, PET = Positron Emission Tomography, MRI = Magnetic Resonance Imaging, CESM = Contrast Enhanced Spectral Mammography, XR = X-ray radiography.
      MAE = Mean Absolute Error, RMSE = Root Mean Squared Error, ACC = accuracy, PPV = Positive Predictive Value, NPV = Negative Predictive Value, TPR = True Positive Rate, TNR = True Negative Rate, FPR = False Positive Rate, FNR = False Negative Rate, FRR = False Region Rate, AUC = Area Under Curve, DSC = Dice’s Similarity Coefficient, JI = Jaccard Index, MAD = Mean Absolute distance, MaxD = Maximum Distance, HD = Hausdorff Distance, MHD = Mahalanobis Distance, MCC = Matthews Correlation Coefficient.
      CV = Cross Validation, LOOCV = Leave-One-Out CV, LPOCV = Leave-Pair-Out CV).
      Artificial neural networks (ANN) are often used in radiomics [
      • Avanzo M.
      • Stancanello J.
      • El Naqa I.
      Beyond imaging: the promise of radiomics.
      ] for classification [

      Battineni G, Chintalapudi N, Amenta F, Traini E. A Comprehensive Machine-Learning Model Applied to Magnetic Resonance Imaging (MRI) to Predict Alzheimer’s Disease (AD) in Older Subjects. J Clin Med 2020;9:10.3390/jcm9072146.

      ,
      • Crisi G.
      • Filice S.
      Predicting MGMT Promoter methylation of glioblastoma from dynamic susceptibility contrast perfusion: A radiomic approach.
      ,
      • Salvi M.
      • Molinari F.
      • Dogliani N.
      • Bosco M.
      Automatic discrimination of neoplastic epithelium and stromal response in breast carcinoma.
      ]. Random forest (RF), a popular concept in ML, are based on a large set of randomly generated decision trees which are trained individually. After training, the prediction is made for all the individual trees and the most frequently selected class is taken as a final result [
      • D’Amico N.C.
      • Sicilia R.
      • Cordelli E.
      • Valbusa G.
      • Grossi E.
      • Zanetti I.B.
      • et al.
      Early radiomics experiences in predicting cyberknife response in acoustic neuroma.
      ,
      • Chauvie S.
      • De Maggi A.
      • Baralis I.
      • Dalmasso F.
      • Berchialla P.
      • Priotto R.
      • et al.
      Artificial intelligence and radiomics enhance the positive predictive value of digital chest tomosynthesis for lung cancer detection within SOS clinical trial.
      ,
      • Ferrari R.
      • Mancini-Terracciano C.
      • Voena C.
      • Rengo M.
      • Zerunian M.
      • Ciardiello A.
      • et al.
      MR-based artificial intelligence model to assess response to therapy in locally advanced rectal cancer.
      ,
      • Inglese P.
      • Amoroso N.
      • Boccardi M.
      • Bocchetta M.
      • Bruno S.
      • Chincarini A.
      • et al.
      Multiple RF classifier for the hippocampus segmentation: method and validation on EADC-ADNI Harmonized Hippocampal Protocol.
      ,
      • Brunese L.
      • Mercaldo F.
      • Reginelli A.
      • Santone A.
      An ensemble learning approach for brain cancer detection exploiting radiomic features.
      ].
      Unsupervised ML techniques can determine patterns in the data which can be used for cathegorization, without the need of user-provided labels. Examples are clustering methods such as k-means [
      • Basile T.M.A.
      • Fanizzi A.
      • Losurdo L.
      • Bellotti R.
      • Bottigli U.
      • Dentamaro R.
      • et al.
      Microcalcification detection in full-field digital mammograms: a fully automated computer-aided system.
      ,
      • Gallivanone F.
      • Interlenghi M.
      • Canervari C.
      • Castiglioni I.
      A fully automatic, threshold-based segmentation method for the estimation of the Metabolic Tumor Volume from PET images: validation on 3D printed anthropomorphic oncological lesions.
      ], hard C-means[

      Schenone D, Lai R, Cea M, Rossi F, Torri L, Bignotti B, et al. Radiomics and artificial intelligence analysis of CT data for the identification of prognostic features in multiple myeloma. Proc.SPIE, vol. 11314, 2020. https://doi.org/10.1117/12.2548983.

      ], hierarchical [

      Fanizzi A, Basile TMA, Losurdo L, Bellotti R, Bottigli U, Dentamaro R, et al. A machine learning approach on multiscale texture analysis for breast microcalcification diagnosis. BMC Bioinformatics 2020;21:91-020-3358–4. https://doi.org/10.1186/s12859-020-3358-4.

      ], and Fuzzy C-Means [
      • Militello C.
      • Rundo L.
      • Vitabile S.
      • Russo G.
      • Pisciotta P.
      • Marletta F.
      • et al.
      Gamma Knife treatment planning: MR brain tumor segmentation and volume measurement based on unsupervised Fuzzy C-Means clustering.
      ,
      • Rundo L.
      • Militello C.
      • Russo G.
      • Garufi A.
      • Vitabile S.
      • Gilardi M.C.
      • et al.
      Automated prostate gland segmentation based on an unsupervised fuzzy C-means clustering technique using multispectral T1w and T2w MR imaging.
      ,
      • Rundo L.
      • Stefano A.
      • Militello C.
      • Russo G.
      • Sabini M.G.
      • D’Arrigo C.
      • et al.
      A fully automatic approach for multimodal PET and MR image segmentation in gamma knife treatment planning.
      ,
      • Militello C.
      • Rundo L.
      • Minafra L.
      • Cammarata F.P.
      • Calvaruso M.
      • Conti V.
      • et al.
      MF2C3: multi-feature fuzzy clustering to enhance cell colony detection in automated clonogenic assay evaluation.
      ]. Less frequently, ML is employed for image segmentation using supervised [
      • Comelli A.
      • Stefano A.
      • Russo G.
      • Bignardi S.
      • Sabini M.G.
      • Petrucci G.
      • et al.
      K-nearest neighbor driving active contours to delineate biological tumor volumes.
      ,

      Giannini V, Rosati S, Regge D, Balestra G. Texture Features and Artificial Neural Networks: A Way to Improve the Specificity of a CAD System for Multiparametric MR Prostate Cancer. In: Kyriacou E, Christofides S, Pattichis CS, editors., Cham: Springer International Publishing; 2016, p. 296–301.

      ,
      • Tangaro S.
      • Amoroso N.
      • Brescia M.
      • Cavuoti S.
      • Chincarini A.
      • Errico R.
      • et al.
      Feature selection based on machine learning in MRIs for hippocampal segmentation.
      ] or unsupervised [
      • Militello C.
      • Vitabile S.
      • Rundo L.
      • Russo G.
      • Midiri M.
      • Gilardi M.C.
      A fully automatic 2D segmentation method for uterine fibroid in MRgFUS treatment evaluation.
      ,
      • Militello C.
      • Rundo L.
      • Vitabile S.
      • Russo G.
      • Pisciotta P.
      • Marletta F.
      • et al.
      Gamma Knife treatment planning: MR brain tumor segmentation and volume measurement based on unsupervised Fuzzy C-Means clustering.
      ,
      • Rundo L.
      • Militello C.
      • Russo G.
      • Garufi A.
      • Vitabile S.
      • Gilardi M.C.
      • et al.
      Automated prostate gland segmentation based on an unsupervised fuzzy C-means clustering technique using multispectral T1w and T2w MR imaging.
      ,
      • Rundo L.
      • Stefano A.
      • Militello C.
      • Russo G.
      • Sabini M.G.
      • D’Arrigo C.
      • et al.
      A fully automatic approach for multimodal PET and MR image segmentation in gamma knife treatment planning.
      ,
      • Militello C.
      • Rundo L.
      • Minafra L.
      • Cammarata F.P.
      • Calvaruso M.
      • Conti V.
      • et al.
      MF2C3: multi-feature fuzzy clustering to enhance cell colony detection in automated clonogenic assay evaluation.
      ] e.g. by first computing features in the neighborhood of pixels which are then tagged using ML.
      Deep learning (DL) [
      • Polsinelli M.
      • Cinque L.
      • Placidi G.
      A light CNN for detecting COVID-19 from CT scans of the chest.
      ,
      • Bria A.
      • Marrocco C.
      • Tortorella F.
      Addressing class imbalance in deep learning for small lesion detection on medical images.
      ,
      • Banzato T.
      • Causin F.
      • Della Puppa A.
      • Cester G.
      • Mazzai L.
      • Zotti A.
      Accuracy of deep learning to differentiate the histopathological grading of meningiomas on MR images: a preliminary study.
      ,
      • Bevilacqua V.
      • Brunetti A.
      • Guerriero A.
      • Trotta G.F.
      • Telegrafo M.
      • Moschetta M.
      A performance comparison between shallow and deeper neural networks supervised classification of tomosynthesis breast lesions images.
      ,
      • Bizzego A.
      • Bussola N.
      • Chierici M.
      • Maggio V.
      • Francescatto M.
      • Cima L.
      • et al.
      Evaluating reproducibility of AI algorithms in digital pathology with DAPPER.
      ,
      • Salvi M.
      • Molinaro L.
      • Metovic J.
      • Patrono D.
      • Romagnoli R.
      • Papotti M.
      • et al.
      Fully automated quantitative assessment of hepatic steatosis in liver transplants.
      ,

      Galbusera F, Niemeyer F, Bassani T, Sconfienza LM, Wilke H-J. Estimating the three-dimensional vertebral orientation from a planar radiograph: Is it feasible? 3rd Int Workshop Spine Load Deform 2020;102:109328. https://doi.org/10.1016/j.jbiomech.2019.109328.

      ,
      • Merone M.
      • Sansone C.
      • Soda P.
      A computer-aided diagnosis system for HEp-2 fluorescence intensity classification.
      ,
      • Mencattini A.
      • Di Giuseppe D.
      • Comes M.C.
      • Casti P.
      • Corsi F.
      • Bertani F.R.
      • et al.
      Discovering the hidden messages within cell trajectories using a deep learning approach for in vitro evaluation of cancer drug treatments.
      ,
      • Duggento A.
      • Guerrisi M.
      • Toschi N.
      • Scimeca M.
      • Urbano N.
      • Bonanno E.
      • et al.
      A random initialization deep neural network for discriminating malignant breast cancer lesions.
      ,

      Brunese L, Mercaldo F AUID- ORCID: 0000-0002-9425-1657, Reginelli A, Santone A. Radiomics for Gleason Score Detection through Deep Learning. LID - E5411 [pii] LID - 10.3390/s20185411 [doi]. Sens Basel Switz JID - 101204366 n.d.

      ,
      • Mushtaq J.
      • Pennella R.
      • Lavalle S.
      • Colarieti A.
      • Steidler S.
      • Martinenghi C.M.A.
      • et al.
      Initial chest radiographs and artificial intelligence (AI) predict clinical outcomes in COVID-19 patients: analysis of 697 Italian patients.
      ,

      Mendizabal A, Tagliabue E, Brunet J-N, Dall’Alba D, Fiorini P, Cotin S. Physics-Based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-Guided Breast Biopsy. In: Miller K, Wittek A, Joldes G, Nash MP, Nielsen PMF, editors. Comput. Biomech. Med., Cham: Springer International Publishing; 2020, p. 33–45.

      ,
      • Rundo L.
      • Han C.
      • Nagano Y.
      • Zhang J.
      • Hataya R.
      • Militello C.
      • et al.
      USE-Net: Incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets.
      ]is a class of powerful MLs based on multiple deep layers of neural networks, characterized by hundreds of layers, each of which learns to detect different features of increasing complexity from an image. In contrast to ML, DL doesn’t need to define a priori a set of hand-crafted features, instead constructing its own internal features which are able to describe rich and comprehensive information, thus performing data representation and prediction jointly. CNNs can be used for regression [
      • Spampinato C.
      • Palazzo S.
      • Giordano D.
      • Aldinucci M.
      • Leonardi R.
      Deep learning for automated skeletal bone age assessment in X-ray images.
      ,
      • Fiorentino M.C.
      • Moccia S.
      • Capparuccini M.
      • Giamberini S.
      • Frontoni E.
      A regression framework to head-circumference delineation from US fetal images.
      ,

      Famouri S, Morra L, Lamberti F. A Deep Learning Approach for Efficient Registration of Dual View Mammography. In: Schilling F-P, Stadelmann T, editors. Artif. Neural Netw. Pattern Recognit., Cham: Springer International Publishing; 2020, p. 162–72. https://doi.org/10.1007/978-3-030-58309-5_13.

      ], classification [

      Marco B, Leonardo B, Enrico G, Francesco S, Gastone C, Claudia T, et al. Circumventing the Curse of Dimensionality in Magnetic Resonance Fingerprinting through a Deep Learning Approach. ArXiv E-Prints 2018:ar:1811.11477.

      ,
      • Kirienko M.
      • Sollini M.
      • Silvestri G.
      • Mognetti S.
      • Voulaz E.
      • Antunovic L.
      • et al.
      Convolutional neural networks promising in lung cancer T-parameter assessment on baseline FDG-PET/CT.
      ,

      Castiglioni I, Ippolito D, Interlenghi M, Monti CB, Salvatore C, Schiaffino S, et al. Artificial intelligence applied on chest X-ray can aid in the diagnosis of COVID-19 infection: a first experience from Lombardy, Italy. MedRxiv 2020:2020.04.08.20040907. https://doi.org/10.1101/2020.04.08.20040907.

      ], segmentation [

      Brunetti A, Cascarano GD, De Feudis I, Moschetta M, Gesualdo L, Bevilacqua V. Detection and Segmentation of Kidneys from Magnetic Resonance Images in Patients with Autosomal Dominant Polycystic Kidney Disease. In: Huang D-S, Jo K-H, Huang Z-K, editors., Cham: Springer International Publishing; 2019, p. 639–50.

      ,
      • Bevilacqua V.
      • Brunetti A.
      • Cascarano G.D.
      • Guerriero A.
      • Pesce F.
      • Moschetta M.
      • et al.
      A comparison between two semantic deep learning frameworks for the autosomal dominant polycystic kidney disease segmentation based on magnetic resonance images.
      ,
      • Panic J.
      • Defeudis A.
      • Mazzetti S.
      • Rosati S.
      • Giannetto G.
      • Vassallo L.
      • et al.
      A convolutional neural network based system for colorectal cancer segmentation on MRI images.
      ,
      • Giannini V.
      • Defeudis A.
      • Rosati S.
      • Cappello G.
      • Vassallo L.
      • Mazzetti S.
      • et al.
      Deep learning to segment liver metastases on CT images: impact on a radiomics method to predict response to chemotherapy.
      ,
      • Piantadosi G.
      • Sansone M.
      • Fusco R.
      • Sansone C.
      Multi-planar 3D breast segmentation in MRI via deep convolutional neural networks.
      ,
      • Valvano G.
      • Santini G.
      • Martini N.
      • Ripoli A.
      • Iacconi C.
      • Chiappino D.
      • et al.
      Convolutional neural networks for the segmentation of microcalcification in mammography imaging.
      ,
      • Nanni L.
      • Lumini A.
      • Ghidoni S.
      • Maguolo G.
      Stochastic selection of activation layers for convolutional neural networks.
      ] or image registration [

      Famouri S, Morra L, Lamberti F. A Deep Learning Approach for Efficient Registration of Dual View Mammography. In: Schilling F-P, Stadelmann T, editors. Artif. Neural Netw. Pattern Recognit., Cham: Springer International Publishing; 2020, p. 162–72. https://doi.org/10.1007/978-3-030-58309-5_13.

      ] tasks. Alternatively, DL networks can be used to extract machine learnt from their layers, which can then be passed to ML algorithms for classification [
      • Amoroso N.
      • La Rocca M.
      • Bellantuono L.
      • Diacono D.
      • Fanizzi A.
      • Lella E.
      • et al.
      Deep learning and multiplex networks for accurate modeling of brain age.
      ,

      Lizzi F, Laruina F, Oliva P, Retico A, Fantacci ME. Residual Convolutional Neural Networks to Automatically Extract Significant Breast Density Features. In: Vento M, Percannella G, Colantonio S, Giorgi D, Matuszewski BJ, Kerdegari H, et al., editors., Cham: Springer International Publishing; 2019, p. 28–35.

      ], or for image reconstruction [

      Antonio F, Danilo C, Simone S, Michele S, Aurelio U. A multimodal deep network for the reconstruction of T2W MR images. ArXiv E-Prints 2019:ar:1908.03009.

      ].
      Different DL architectures, such as Convolutional Neural Networks (CNN), Residual Networks (RN) [
      • Rocca M.A.
      • Anzalone N.
      • Storelli L.
      • Del Poggio A.
      • Cacciaguerra L.
      • Manfredi A.A.
      • et al.
      Deep learning on conventional magnetic resonance imaging improves the diagnosis of multiple sclerosis mimics.
      ], Autoencoders [
      • Ferrari E.
      • Bosco P.
      • Calderoni S.
      • Oliva P.
      • Palumbo L.
      • Spera G.
      • et al.
      Dealing with confounders and outliers in classification medical studies: the autism spectrum disorders case study.
      ], Recurrent Neural Networks (RNN), allow for unlimited flexibility in extracting information (features and filters) outperforming humans in 2D/3D images analysis.
      Instead of training a DL from scratch, architectures can be adapted from already existing and trained architecture such as Googlenet [
      • Spampinato C.
      • Palazzo S.
      • Giordano D.
      • Aldinucci M.
      • Leonardi R.
      Deep learning for automated skeletal bone age assessment in X-ray images.
      ], Google Inception [
      • Walsh S.L.F.
      • Calandriello L.
      • Silva M.
      • Sverzellati N.
      Deep learning for classifying fibrotic lung disease on high-resolution computed tomography: a case-cohort study.
      ] or ENet [
      • Moccia S.
      • Banali R.
      • Martini C.
      • Muscogiuri G.
      • Pontone G.
      • Pepi M.
      • et al.
      Development and testing of a deep learning-based strategy for scar segmentation on CMR-LGE images.
      ]. The training an existing network by fine tuning its parameters and weights for a task which is unrelated to the present goal, called transfer learning, enables to develop efficiently accurate models [
      • Spampinato C.
      • Palazzo S.
      • Giordano D.
      • Aldinucci M.
      • Leonardi R.
      Deep learning for automated skeletal bone age assessment in X-ray images.
      ,
      • Brunese L.
      • Mercaldo F.
      • Reginelli A.
      • Santone A.
      Explainable deep learning for pulmonary disease and coronavirus COVID-19 Detection from X-rays.
      ,
      • Basaia S.
      • Agosta F.
      • Wagner L.
      • Canu E.
      • Magnani G.
      • Santangelo R.
      • et al.
      Automated classification of Alzheimer’s disease and mild cognitive impairment using a single MRI and deep neural networks.
      ]. RN are DL networks residual or skip connections, which jump some layers [

      Lizzi F, Laruina F, Oliva P, Retico A, Fantacci ME. Residual Convolutional Neural Networks to Automatically Extract Significant Breast Density Features. In: Vento M, Percannella G, Colantonio S, Giorgi D, Matuszewski BJ, Kerdegari H, et al., editors., Cham: Springer International Publishing; 2019, p. 28–35.

      ] making it possible to train deeper networks. Generative Adversarial Networks (GANs) transport the generative modeling approach in the context of DL. The idea behind this architecture is to learn to generate new data with the same statistics as the training set, using two neural networks contesting each other in a sort of a zero-sum game, where one agent's gain is another agent's loss. In the clinical context it is worth noting the use of GAN for Adversarial Learning, that is to build carefully targeted attacks to fool a model prediction [
      • Comes M.C.
      • Filippi J.
      • Mencattini A.
      • Casti P.
      • Cerrato G.
      • Sauvat A.
      • et al.
      Multi-scale generative adversarial network for improved evaluation of cell–cell interactions observed in organ-on-chip experiments.
      ,
      • Barucci A.
      Adversarial radiomics: the rising of potential risks in medical imaging from adversarial learning.
      ,
      • Andreini P.
      • Bonechi S.
      • Bianchini M.
      • Mecocci A.
      • Scarselli F.
      Image generation by GAN and style transfer for agar plate image segmentation.
      ].

      Systematic review

      A search for peer-reviewed manuscripts written in English was performed using the PubMed engine. The search was aimed at selecting studies performed in Italian institutions using machine learning applied to medical images. The search strategy was: “((((artificial intelligence) OR (machine learning)) OR (deep learning)) AND (imaging)) AND (italy[Affiliation])) NOT (review[Pubblication Type])”, limited to years 2015–2020.
      In a second phase, participants to the AI4FM task-group were asked to select the research papers, not covered in the first search, starting from those published from their institute on AI in imaging.
      The participants independently reviewed the selected manuscripts. Studies were considered eligible when a machine learning method was applied to images or to features extracted from images, acquired for medical purpose, and the research was performed by investigators affiliated to insitutes in Italy.
      For eligible studies, the AI method, type of AI task (classification, regression, segmentation, registration), algorithm, metric used for evaluation of AI performance, and type of validation, were collected. In order to study the type of institution were the research was conducted, the studies were categorized according to the affiliation of the first and last author, as hospital, university, public research institute (mainly National Institute for Nuclear Physics, INFN and National Research Council, CNR) or private company or foundation. The studies were considered as collaborations among different institutions of different types when the first and last authors came from different categories.

      Results of systematic review

      The search produced 793 papers. 120 studies were removed as not related to the medical field, 223 had not imaging or AI, 52 were editorials or reviews, 244 had not authors from Italy. 32 were removed as they had a significant overlap with other papers with higher impact factor by the same group. 122 were considered eligible, and 46 papers were manually retrieved by the participants in the group, for a total of 168 studies. These are shown in Table 1 (ML studies) and Table 2 (DL studies), were the clinical indication, algorithms and software platform used for each study are described.
      Table 2Summary of research papers published in Italy in years 2015–2020 using deep learning applied to imaging. The table shows for each study the first author and year of publication, the image modality, the software for implementing the deep learning, the type of neural network implemented, its training modality, the metric for model evaluation and validation and the purpose of use of the research.
      Clinical fieldRefAuthorYearImagePlatformPurpose of CNNCNN ArchitectureTraining ModalityEvaluation metricsValidationClinicClinical indication
      Neurological
      • Amoroso N.
      • Diacono D.
      • Fanizzi A.
      • La Rocca M.
      • Monaco A.
      • Lombardi A.
      • et al.
      Deep learning reveals Alzheimer’s disease onset in MCI subjects: results from an international challenge.
      Amoroso N., et al.2018MRIRClassificationAd hocFeature extractoroverall accuracy, recall and precision10-fold cross-validation, independent test setAlzheimer disease detection
      • Amoroso N.
      • La Rocca M.
      • Bellantuono L.
      • Diacono D.
      • Fanizzi A.
      • Lella E.
      • et al.
      Deep learning and multiplex networks for accurate modeling of brain age.
      Amoroso N., et al.2019MRI“h2o” R packageRegressionAd hocFeature Extractor + RR or LASSOMAE, RMSE, and Pearson’s correlation100 times repeated 10-fold CVBrain Age Modelling

      Marco B, Leonardo B, Enrico G, Francesco S, Gastone C, Claudia T, et al. Circumventing the Curse of Dimensionality in Magnetic Resonance Fingerprinting through a Deep Learning Approach. ArXiv E-Prints 2018:ar:1811.11477.

      Barbieri M., et al.2018MRPython package Keras with TensorFlowImage ReconstructionAd HocEnd to EndPredicted vs ground truth T1Test setMR fingerprinting

      Antonio F, Danilo C, Simone S, Michele S, Aurelio U. A multimodal deep network for the reconstruction of T2W MR images. ArXiv E-Prints 2019:ar:1908.03009.

      Falvo A., et al.2020MRIKerasImage ReconstructionU-NetEnd to EndSSIMValidation setMultiple sclerosis
      • Rocca M.A.
      • Anzalone N.
      • Storelli L.
      • Del Poggio A.
      • Cacciaguerra L.
      • Manfredi A.A.
      • et al.
      Deep learning on conventional magnetic resonance imaging improves the diagnosis of multiple sclerosis mimics.
      Rocca et al.2020MRIKeras with Theano backendClassificationCNNEnd to EndAccuracy, sensitivity, specificityTest setMultiple sclerosis
      • Ferrari E.
      • Bosco P.
      • Calderoni S.
      • Oliva P.
      • Palumbo L.
      • Spera G.
      • et al.
      Dealing with confounders and outliers in classification medical studies: the autism spectrum disorders case study.
      Ferrari et al.2020MRI/ClassificationAutoencoders///Autism
      • Basaia S.
      • Agosta F.
      • Wagner L.
      • Canu E.
      • Magnani G.
      • Santangelo R.
      • et al.
      Automated classification of Alzheimer’s disease and mild cognitive impairment using a single MRI and deep neural networks.
      Basaia S., et al.2019MRIPython TheanoClassificationAd HocEnd to End, transfer learningAccuracyValidation, testing setAlzheimer diseaseand mild cog impairment
      • Aslani S.
      • Dayan M.
      • Storelli L.
      • Filippi M.
      • Murino V.
      • Rocca M.A.
      • et al.
      Multi-branch convolutional neural network for multiple sclerosis lesion segmentation.
      Aslani et al.2019MRIKeras with TensorflowSegmentationResnet50 modifiedEnd to EndDSCTest setMultiple sclerosis
      Cancer detection and characterization
      • Banzato T.
      • Causin F.
      • Della Puppa A.
      • Cester G.
      • Mazzai L.
      • Zotti A.
      Accuracy of deep learning to differentiate the histopathological grading of meningiomas on MR images: a preliminary study.
      Banzato T., et al.2019MRIMatlabClassificationInception-V3, AlexnetTransfer LearningAUC, sensitivity, specificityLeave one outMeningioma histopathological grading
      • Bevilacqua V.
      • Brunetti A.
      • Guerriero A.
      • Trotta G.F.
      • Telegrafo M.
      • Moschetta M.
      A performance comparison between shallow and deeper neural networks supervised classification of tomosynthesis breast lesions images.
      Bevilacqua V., et al.2019Tomosynthesis/ClassificationAd hocEnd to EndACC, TPR, TNRtraining(and test set )Breast cancer diagnosis
      • Duggento A.
      • Guerrisi M.
      • Toschi N.
      • Scimeca M.
      • Urbano N.
      • Bonanno E.
      • et al.
      A random initialization deep neural network for discriminating malignant breast cancer lesions.
      Duggento et al.2019MammographyKeras TensorflowClassificationAd HocEnd to EndACC, PPV, TPR, TNR, FPR, FNR, AUC, F2 score, F1 scorevalidation and test setCancer diagnosis

      Brunese L, Mercaldo F AUID- ORCID: 0000-0002-9425-1657, Reginelli A, Santone A. Radiomics for Gleason Score Detection through Deep Learning. LID - E5411 [pii] LID - 10.3390/s20185411 [doi]. Sens Basel Switz JID - 101204366 n.d.

      Brunese L. et al2020MRIPyRadiomics, Keras TensorFlowRegressionAd HocEnd to EndSensitivity, specificity, ACC, PP,5-fold CVCancer characterization

      Mendizabal A, Tagliabue E, Brunet J-N, Dall’Alba D, Fiorini P, Cotin S. Physics-Based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-Guided Breast Biopsy. In: Miller K, Wittek A, Joldes G, Nash MP, Nielsen PMF, editors. Comput. Biomech. Med., Cham: Springer International Publishing; 2020, p. 33–45.

      Mendizabal et al.2020USPyTorchRegistrationU-NetEnd to Endregistration errorsin-phantomBiopsy guidance

      Famouri S, Morra L, Lamberti F. A Deep Learning Approach for Efficient Registration of Dual View Mammography. In: Schilling F-P, Stadelmann T, editors. Artif. Neural Netw. Pattern Recognit., Cham: Springer International Publishing; 2020, p. 162–72. https://doi.org/10.1007/978-3-030-58309-5_13.

      Famouri et a.2020MammographyKeras TensorflowRegistrationAd hoc based on ResNet50Transfer learningMean Squared Errorsplit in training and validationBreast cancer diagnosis
      • Kirienko M.
      • Sollini M.
      • Silvestri G.
      • Mognetti S.
      • Voulaz E.
      • Antunovic L.
      • et al.
      Convolutional neural networks promising in lung cancer T-parameter assessment on baseline FDG-PET/CT.
      Kirienko et a.2018CT-PETPython PyTorchClassificationAd HocFeature extractionAUC, ACC, Recall, SpecificityTest setCancer characterization
      • Piantadosi G.
      • Sansone M.
      • Fusco R.
      • Sansone C.
      Multi-planar 3D breast segmentation in MRI via deep convolutional neural networks.
      Piantadosi G., et al.2020MRIPythonSegmentationU-NetEnd to EndACC, Sensitivity, Specificity, DSC10-fold CVSegmentation of breast parenchyma
      • Panic J.
      • Defeudis A.
      • Mazzetti S.
      • Rosati S.
      • Giannetto G.
      • Vassallo L.
      • et al.
      A convolutional neural network based system for colorectal cancer segmentation on MRI images.
      Panic J., et al.2020MRIPython, Keras, TensorflowSegmentationAd HocEnd-to-EndDSC, PrecisionTesting setColorectal cancer segmentation
      • Valvano G.
      • Santini G.
      • Martini N.
      • Ripoli A.
      • Iacconi C.
      • Chiappino D.
      • et al.
      Convolutional neural networks for the segmentation of microcalcification in mammography imaging.
      Valvano G., et al.2019MammographyPython TensorflowClassification, SegmentationAd HocEnd to EndAccuracyValidation and Test setsMicrocalcification segmentation
      • Nanni L.
      • Lumini A.
      • Ghidoni S.
      • Maguolo G.
      Stochastic selection of activation layers for convolutional neural networks.
      Nanni et al.20,202VariousClassification, segmentationDeeplabv3+End to EndAccuracyCVVarious tasks incl. breast cancer
      • Soomro M.H.
      • Coppotelli M.
      • Conforto S.
      • Schmid M.
      • Giunta G.
      • Del Secco L.
      • et al.
      Automated segmentation of colorectal tumor in 3D MRI using 3D multiscale densely connected convolutional neural network.
      Soomro et al.2019MRICaffeSegmentationModified MSDNetEnd to EndDSCCVColorectal cancer segmentation
      • Sena P.
      • Fioresi R.
      • Faglioni F.
      • Losi L.
      • Faglioni G.
      • Roncucci L.
      Deep learning techniques for detecting preneoplastic and neoplastic lesions in human colorectal histological images.
      Sena et al.2019Histology imagesPytorchClassificationAd hocEnd to EndAccuracyTest setColorectal cancer detection
      • De Logu F.
      • Ugolini F.
      • Maio V.
      • Simi S.
      • Cossu A.
      • Massi D.
      • et al.
      Recognition of cutaneous melanoma on digitized histopathological slides via artificial intelligence algorithm.
      De Logu et al.2020Histopathological imagesMatlabClassificationInception-ResNet-v2Transfer learningAccuracy, F-score, Cohen’s kappaValidation setMelanoma detection
      • Ligabue G.
      • Pollastri F.
      • Fontana F.
      • Leonelli M.
      • Furci L.
      • Giovanella S.
      • et al.
      Evaluation of the classification accuracy of the kidney biopsy direct immunofluorescence through convolutional neural networks.
      Ligabue et al.2020Immunofluorescence/ClassificationResnet101End to EndAccuracyTest setCharacterization of kidney biopsy
      Other
      • Polsinelli M.
      • Cinque L.
      • Placidi G.
      A light CNN for detecting COVID-19 from CT scans of the chest.
      Polsinelli M., et al.2020XrMatlabClassificationSqueezeNetEnd to End, transfer learningACC, Sensitivity, Specificity, Precision , F1 scorevalidation and test se.t, 10-fold CVCOVID-19 detection
      • Bria A.
      • Marrocco C.
      • Tortorella F.
      Addressing class imbalance in deep learning for small lesion detection on medical images.
      Bria A., et al.2020Mammograms, fundus imagesC++ OpenCVClassificationVGGEnd to EndAUC, mean sensitivity, TP, TPF2-fold CVMicrocalcification in mammograms, microaneurysm in retinal images
      • Salvi M.
      • Molinaro L.
      • Metovic J.
      • Patrono D.
      • Romagnoli R.
      • Papotti M.
      • et al.
      Fully automated quantitative assessment of hepatic steatosis in liver transplants.
      Salvi M., et al.2020HistologyKerasSegmentationResNet34Feature ExtractionACC, MAETraining and test setHepatic steatosis

      Galbusera F, Niemeyer F, Bassani T, Sconfienza LM, Wilke H-J. Estimating the three-dimensional vertebral orientation from a planar radiograph: Is it feasible? 3rd Int Workshop Spine Load Deform 2020;102:109328. https://doi.org/10.1016/j.jbiomech.2019.109328.

      Galbusera F., et al.2020XRPython, KerasClassi-ficationResNet101End-to-EndACC, error analysisIndipendent datasetVertebre description
      • Mushtaq J.
      • Pennella R.
      • Lavalle S.
      • Colarieti A.
      • Steidler S.
      • Martinenghi C.M.A.
      • et al.
      Initial chest radiographs and artificial intelligence (AI) predict clinical outcomes in COVID-19 patients: analysis of 697 Italian patients.
      MushTaq et al.2020XRqXR v2.1 c2, Qure.ai TechnologiesClassificationAd hocEnd to EndAUC, sensitivity, Two-tailed testsvalidation setCOVID outcome prediction
      • Rundo L.
      • Han C.
      • Nagano Y.
      • Zhang J.
      • Hataya R.
      • Militello C.
      • et al.
      USE-Net: Incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets.
      Rundo L. et al.2019MRIKeras (TensorFlow backend)SegmentationModified U-netEnd to EndDSC, Sensitivity, Specificity, MAD, MaxD, HD4-fold CV (three datasets)Prostate segmentation
      • Spampinato C.
      • Palazzo S.
      • Giordano D.
      • Aldinucci M.
      • Leonardi R.
      Deep learning for automated skeletal bone age assessment in X-ray images.
      Spampinato C. et al.2017XRPythonRegressionAd hoc or OverFeat, GoogleNet, OxfordNetEnd to End, Transfer LearningMAE5-fold CVSkeletal bone age

      Castiglioni I, Ippolito D, Interlenghi M, Monti CB, Salvatore C, Schiaffino S, et al. Artificial intelligence applied on chest X-ray can aid in the diagnosis of COVID-19 infection: a first experience from Lombardy, Italy. MedRxiv 2020:2020.04.08.20040907. https://doi.org/10.1101/2020.04.08.20040907.

      Castiglioni I. et al.2020Chest XRTrace4ClassificationResNET50End to EndACC, sensitivity, specificity, PPV, NPV, AUC10-fold CV, independent test datasetCOVID-19 diagnosis

      Brunetti A, Cascarano GD, De Feudis I, Moschetta M, Gesualdo L, Bevilacqua V. Detection and Segmentation of Kidneys from Magnetic Resonance Images in Patients with Autosomal Dominant Polycystic Kidney Disease. In: Huang D-S, Jo K-H, Huang Z-K, editors., Cham: Springer International Publishing; 2019, p. 639–50.

      Brunetti A. et al.2019MRI/SegmentationAd HocEnt to EndAcc, TPR, TNR, Confusion matrix, Boundary F1 Score, Jaccard Similarity Coefficienttraining, validation and test setSegmentation of kidneys in polycystic disease
      • Bevilacqua V.
      • Brunetti A.
      • Cascarano G.D.
      • Guerriero A.
      • Pesce F.
      • Moschetta M.
      • et al.
      A comparison between two semantic deep learning frameworks for the autosomal dominant polycystic kidney disease segmentation based on magnetic resonance images.
      Bevilacqua V. et al.2019MRI/SegmentationVGG-16End to Endconfusion matrix, ACC, BF score, precision, recall, Jaccard sim coef5-fold CV, test setPolycystic kidney disease
      • Walsh S.L.F.
      • Calandriello L.
      • Silva M.
      • Sverzellati N.
      Deep learning for classifying fibrotic lung disease on high-resolution computed tomography: a case-cohort study.
      Walsh S.L.F. et al.2018CTTensorFlowClassificationBased on Google InceptionEnd to EndACC, AUC, TPR, TNR, weighted k coefficient of interobserver agreementvalidation and test set (test set A and test set B)Fibrotic lung disease
      • Brunese L.
      • Mercaldo F.
      • Reginelli A.
      • Santone A.
      Explainable deep learning for pulmonary disease and coronavirus COVID-19 Detection from X-rays.
      Brunese L. et al.2020Chest XRKerasClassificationVGG16 - Visual Geometry GroupTransfer learningTPR, TNR, F-score, ACCcross-validation, training and independent test setCOVID-19 detection
      • Cerveri P.
      • Belfatto A.
      • Baroni G.
      • Manzotti A.
      Stacked sparse autoencoder networks and statistical shape models for automatic staging of distal femur trochlear dysplasia.
      Cerveri et al.2018CT/ClassificationSparse stacked AutoencodersEnd to endAccuracy, sensitivity, specificityTest setFemur Dysplasia
      • Tartaglione E.
      • Barbano C.A.
      • Berzovini C.
      • Calandri M.
      • Grangetto M.
      Unveiling COVID-19 from CHEST X-Ray with deep learning: a hurdles race with small data.
      Tartaglione et al.2020RadiographsPytorch 1.4ClassificationResNet-18, Resnet-50, COVID-Net, DenseNet-121Transfer learningAccuracy, AUCTest on different public datasetsCOVID-19 diagnosis
      • Patrini I.
      • Ruperti M.
      • Moccia S.
      • Mattos L.S.
      • Frontoni E.
      • De Momi E.
      Transfer learning for informative-frame selection in laryngoscopic videos through learned features.
      Patrini et al.2020Laryngoscopic videosKeras with TensorFlow backendClassificationInception – ResNet V2, othersFeatures extraction+ SVMAccuracyCVSelection of informative frames
      • Galbusera F.
      • Bassani T.
      • Casaroli G.
      • Gitto S.
      • Zanchetta E.
      • Costa F.
      • et al.
      Generative models: an upcoming innovation in musculoskeletal radiology? A preliminary test in spine imaging.
      Galbusera et al.2018MRICaffe, Pytorch, TensorFlowResolution Enhancement, Image synthesisGANEnd to Endk-coefficient of agreement between imagesTest setGenerate image different MRI modality
      • Tanzi L.
      • Vezzetti E.
      • Moreno R.
      • Aprato A.
      • Audisio A.
      • Massè A.
      Hierarchical fracture classification of proximal femur X-Ray images using a multistage Deep Learning approach.
      Tanzi et al.2020RadiograophyKeras, TensorflowClassificationInceptionV3 , CNNEnd to EndAccuracy, AUCTest setClassification of fractures
      • Galbusera F.
      • Niemeyer F.
      • Wilke H.-J.
      • Bassani T.
      • Casaroli G.
      • Anania C.
      • et al.
      Fully automated radiological analysis of spinal disorders and deformities: a deep learning approach.
      Galbusera et al.2019RadiographsKeras with TensorFlow backendRegressionAD hocEnd to EndStandard errorTesting setSpine deformities
      • Scarpa F.
      • Colonna A.
      • Ruggeri A.
      Multiple-image deep learning analysis for neuropathy detection in corneal nerve images.
      Scarpa et al.2020Corneal Images/ClassificationAd hocEnd to endAccuractTest setDiabetes diagnosis
      • Lepore D.
      • Ji M.H.
      • Pagliara M.M.
      • Lenkowicz J.
      • Capocchiano N.D.
      • Tagliaferri L.
      • et al.
      Convolutional neural network based on fluorescein angiography images for retinopathy of prematurity management.
      Lepore et al.2020Fluorescein AngiographyKeras for R with TensorflowClassificationAd hocEnd to endAccuracyTest setRetinopathy
      • Casella A.
      • Moccia S.
      • Frontoni E.
      • Paladini D.
      • De Momi E.
      • Mattos L.S.
      Inter-foetus membrane segmentation for TTTS using adversarial networks.
      Casella et al.2019Fetoscopic imagesPytorchSegmentationAdversarial networkEnd to endDSCValidation and testing setsInter-foetal membrane
      Cardiovascular
      • Moccia S.
      • Banali R.
      • Martini C.
      • Muscogiuri G.
      • Pontone G.
      • Pepi M.
      • et al.
      Development and testing of a deep learning-based strategy for scar segmentation on CMR-LGE images.
      Moccia S. et al.2020CE – MRITensorFlowSegmentationModified ENetEnd to EndACC, TPR, TNR, DSCLeave one-patient-out, test setScar in left ventricle
      • Muscogiuri G.
      • Chiesa M.
      • Trotta M.
      • Gatti M.
      • Palmisano V.
      • Dell’Aversana S.
      • et al.
      Performance of a deep learning algorithm for the evaluation of CAD-RADS classification with CCTA.
      Muscogiuri G. et al.2020CCTAKeras R packageClassificationAd hocEnd to EndAUC, ACC, Accuracy, sensitivity, specificity4-fold CVCoronary artery disease
      • Martini N.
      • Aimo A.
      • Barison A.
      • Della Latta D.
      • Vergaro G.
      • Aquaro G.D.
      • et al.
      Deep learning to diagnose cardiac amyloidosis from cardiovascular magnetic resonance.
      Martini et al.2020MRIKeras, TensorflowClassificationAd hocEnd to EdnAUCTest setCardiac amyloidosis
      • Fantazzini A.
      • Esposito M.
      • Finotello A.
      • Auricchio F.
      • Pane B.
      • Basso C.
      • et al.
      3D automatic segmentation of aortic computed tomography angiography combining multi-view 2D convolutional neural networks.
      Fantazzini et al.2020CT AngiographyKeras with TensorflowSegmentation2D U-NetEnd to EndDSCTest setAortic lumen segmentation
      Pathology and microscopy
      • Bizzego A.
      • Bussola N.
      • Chierici M.
      • Maggio V.
      • Francescatto M.
      • Cima L.
      • et al.
      Evaluating reproducibility of AI algorithms in digital pathology with DAPPER.
      Bizzego A., et al.2019Digital PathologyPythonClassificationVGG, ResNet, InceptionFeature Extractionconfusion matrix, MCC ACC10 × 5 fold-cross validationDigital pathology
      • Merone M.
      • Sansone C.
      • Soda P.
      A computer-aided diagnosis system for HEp-2 fluorescence intensity classification.
      Merone M., et al.2019Fluorence intensity/ClassificationScatNetFeature Extractionconfusion and cost matrix, ACC, recal, precision, F1 score.Fluorescence
      • Mencattini A.
      • Di Giuseppe D.
      • Comes M.C.
      • Casti P.
      • Corsi F.
      • Bertani F.R.
      • et al.
      Discovering the hidden messages within cell trajectories using a deep learning approach for in vitro evaluation of cancer drug treatments.
      Mencattini A., et al.2020time-lapse microscopy imagingPythonImage ReconstructionAlexNetTransfer LearningACCTest setCell trajectory
      • Comes M.C.
      • Filippi J.
      • Mencattini A.
      • Casti P.
      • Cerrato G.
      • Sauvat A.
      • et al.
      Multi-scale generative adversarial network for improved evaluation of cell–cell interactions observed in organ-on-chip experiments.
      Comes M.C., et al.2020Organs on a chip, TL Microscopy/Image ReconstructionGANend to endconcordance correlation coefficient, K-S test/Cell-interactions in organ on chip experiments
      • Andreini P.
      • Bonechi S.
      • Bianchini M.
      • Mecocci A.
      • Scarselli F.
      Image generation by GAN and style transfer for agar plate image segmentation.
      Andreini et al.2020Agar platesTensorflowSegmentationGANEnd to EndJIBacteria segmentation
      • Dimauro G.
      • Ciprandi G.
      • Deperte F.
      • Girardi F.
      • Ladisa E.
      • Latrofa S.
      • et al.
      Nasal cytology with deep learning techniques.
      Dimauro G., et al.2019Nasal CytologyPython KerasClassificationAd hocEnd to EndSensitivity, Specificity, ACCValidation setCells in nasal cytology
      Cancer prognosisand outcome prediction
      • Giannini V.
      • Defeudis A.
      • Rosati S.
      • Cappello G.
      • Vassallo L.
      • Mazzetti S.
      • et al.
      Deep learning to segment liver metastases on CT images: impact on a radiomics method to predict response to chemotherapy.
      Giannini V., et al.2020CTMatlabSegmentationU-NetEnd-to-EndPrecisionValidation setLiver metastis segmentation and outcome prediction
      • Bizzego A.
      • Bussola N.
      • Salvalai D.
      • Chierici M.
      • Maggio V.
      • Jurman G.
      • et al.
      Integrating deep and radiomics features in cancer bioimaging.
      Bizzego A., et al.2019CT-PETPython TheanoClassificationAd hocFeature ExtractorMCC, TPR, TNR, ACC5 fold cross validation (10 repetitions), independent test setHead and neck cancer prognosis
      Radiotherapy planning
      • Zaffino P.
      • Pernelle G.
      • Mastmeyer A.
      • Mehrtash A.
      • Zhang H.
      • Kikinis R.
      • et al.
      Fully automatic catheter segmentation in MRI with 3D convolutional neural networks: application to MRI-guided gynecologic brachytherapy.
      Zaffino et al.2019MRITheano, LasagneSegmentation3D-U-netEnd to EndDSC4-fold CVCathether segmentation brachytherapy
      • Cusumano D.
      • Lenkowicz J.
      • Votta C.
      • Boldrini L.
      • Placidi L.
      • Catucci F.
      • et al.
      A deep learning approach to generate synthetic CT in low field MR-guided adaptive radiotherapy for abdominal and pelvic cases.
      Cusumano et al.2020MRIKerasImage synthesiscGANEnd to EndMean absolute error of Hounsfield UnitsTest setSynthesis of CT for RT planning
      We have observed a rapid increase of research in Italy on artificial intelligence in the last 5 years (Fig. 2). The increase in papers peaked at 155% from 2018 to 2019.
      Figure thumbnail gr2
      Fig. 2histogram of studies on AI applied to imaging published in Italy included in the systematic review versus years of publication.
      Fig. 3a-c show the distribution of studies according to image modality, clinical indication, and institution were the research was undertaken.
      Figure thumbnail gr3
      Fig. 3Pie charts showing the distribution of clinical indication (a), imaging technique (b) and type of institution (c) for studies on AI applied to imaging in Italy.
      Results show that most studies (36%) were from universities followed by hospitals (16%). The most used imaging modality was MRI (44%), followed by CT(11%) ad radiography/mammography (10%). and the most common clinical indications were neurological diseases (29%) and diagnosis of cancer (25%).
      Classification was the most common task for AI (57%) followed by segmentation (16%). Among ML algorithms, SVM was by far the most used ML (40.2%). Most of the works retrieved use ML techniques (65% of works).

      Used software tools

      Among the platforms used for ML (Table 1), the most used are Python ML specific libraries, like the Scikit-Learn library [

      Recenti M, Ricciardi C, Gìslason M, Edmunds K, Carraro U, Gargiulo P. Machine Learning Algorithms Predict Body Mass Index Using Nonlinear Trimodal Regression Analysis from Computed Tomography Scans. In: Henriques J, Neves N, de Carvalho P, editors. XV Mediterr. Conf. Med. Biol. Eng. Comput. – MEDICON 2019, Cham: Springer International Publishing; 2020, p. 839–46.

      ]. The Waikato Environment for Knowledge Analysis (Weka), is a free software developed at the University of Weikato, New Zealand, since 1993 [
      • Crisi G.
      • Filice S.
      Predicting MGMT Promoter methylation of glioblastoma from dynamic susceptibility contrast perfusion: A radiomic approach.
      ,
      • Gitto S.
      • Cuocolo R.
      • Albano D.
      • Chianca V.
      • Messina C.
      • Gambino A.
      • et al.
      MRI radiomics-based machine-learning classification of bone chondrosarcoma.
      ]. Many ML algorithms from Weka are included in the Konstanz Information Miner (Knime) analytics platform [
      • Ricciardi C.
      • Cantoni V.
      • Improta G.
      • Iuppariello L.
      • Latessa I.
      • Cesarelli M.
      • et al.
      Application of data mining in a cohort of Italian subjects undergoing myocardial perfusion imaging at an academic medical center.
      ,
      • Romeo V.
      • Ricciardi C.
      • Cuocolo R.
      • Stanzione A.
      • Verde F.
      • Sarno L.
      • et al.
      Machine learning analysis of MRI-derived texture features to predict placenta accreta spectrum in patients with placenta previa.
      ,

      Ricciardi C, Edmunds KJ, Recenti M, Sigurdsson S, Gudnason V, Carraro U, et al. Assessing cardiovascular risks from a mid-thigh CT image: a tree-based machine learning approach using radiodensitometric distributions. Sci Rep 2020;10:2863-020-59873–9. https://doi.org/10.1038/s41598-020-59873-9.

      ], where ML analyses are made intuitive by a graphical interface in which the tasks in the workflow (e.g. data reading, ML training and ML prediction) are visualised as nodes which can be configured and connected to each other. Commercial packages Matlab (Mathworks, Natick, MA), and SAS (SAS Institute, Cary, NC, USA) include ML and DL packages and graphical interfaces [
      • Ferraro P.M.
      • Agosta F.
      • Riva N.
      • Copetti M.
      • Spinelli E.G.
      • Falzone Y.
      • et al.
      Multimodal structural MRI in the diagnosis of motor neuron diseases.
      ].
      R also offers many R-package implementing most of the machine learning algorithms as glmnet for regularized generalized linear models, e1071 for SVM [
      • Amoroso N.
      • Diacono D.
      • Fanizzi A.
      • La Rocca M.
      • Monaco A.
      • Lombardi A.
      • et al.
      Deep learning reveals Alzheimer’s disease onset in MCI subjects: results from an international challenge.
      ,

      Lombardi A, Amoroso N, Diacono D, Monaco A, Tangaro S, Bellotti R. Extensive evaluation of morphological statistical harmonization for brain age prediction. Brain Sci 2020;10:10.3390/brainsci10060364.

      ,
      • Gallivanone F.
      • Cava C.
      • Corsi F.
      • Bertoli G.
      • Castiglioni I.
      In silico approach for the Definition of radiomiRNomic signatures for breast cancer differential diagnosis.
      ], neuralnet for ANN, stats and kohonen for k-means/hierarchical clustering and self-organizing maps. The caret package includes many algorithms implemented in different packages by specifying them as methods of a single training function·H2O is a package for AI in R that offers parallelized implementations of ML [
      • Amoroso N.
      • La Rocca M.
      • Bellantuono L.
      • Diacono D.
      • Fanizzi A.
      • Lella E.
      • et al.
      Deep learning and multiplex networks for accurate modeling of brain age.
      ]. PRoNTo (Pattern Recognition for Neuroimaging Toolbox) is a software toolbox providing analysis tools for neuroimaging including ML [
      • Vai B.
      • Parenti L.
      • Bollettini I.
      • Cara C.
      • Verga C.
      • Melloni E.
      • et al.
      Predicting differential diagnosis between bipolar and unipolar depression with multiple kernel learning on multimodal structural neuroimaging.
      ], while SVM-Light software is a package in C for SVM [

      Retico A, Giuliano A, Tancredi R, Cosenza A, Apicella F, Narzisi A, et al. The effect of gender on the neuroanatomy of children with autism spectrum disorders: a support vector machine case-control study. Mol Autism 2016;7:5-015-0067-3. eCollection 2016. https://doi.org/10.1186/s13229-015-0067-3.

      ,
      • Retico A.
      • Bosco P.
      • Cerello P.
      • Fiorina E.
      • Chincarini A.
      • Fantacci M.E.
      Predictive models based on support vector machines: whole-brain versus regional analysis of structural MRI in the Alzheimer’s disease.
      ].
      Deep neural networks have been used to develop classification models on a variety of modalities including MRI [
      • Amoroso N.
      • La Rocca M.
      • Monaco A.
      • Bellotti R.
      • Tangaro S.
      Complex networks reveal early MRI markers of Parkinson’s disease.
      ,
      • Aslani S.
      • Dayan M.
      • Storelli L.
      • Filippi M.
      • Murino V.
      • Rocca M.A.
      • et al.
      Multi-branch convolutional neural network for multiple sclerosis lesion segmentation.
      ], DTI [
      • Maggipinto T.
      • Bellotti R.
      • Amoroso N.
      • Diacono D.
      • Donvito G.
      • Lella E.
      • et al.
      DTI measurements for Alzheimer’s classification.
      ], CT [
      • Cerveri P.
      • Belfatto A.
      • Baroni G.
      • Manzotti A.
      Stacked sparse autoencoder networks and statistical shape models for automatic staging of distal femur trochlear dysplasia.
      ], PET[
      • Kirienko M.
      • Sollini M.
      • Silvestri G.
      • Mognetti S.
      • Voulaz E.
      • Antunovic L.
      • et al.
      Convolutional neural networks promising in lung cancer T-parameter assessment on baseline FDG-PET/CT.
      ], radiographs [
      • Tartaglione E.
      • Barbano C.A.
      • Berzovini C.
      • Calandri M.
      • Grangetto M.
      Unveiling COVID-19 from CHEST X-Ray with deep learning: a hurdles race with small data.
      ] as well as on videos [
      • Patrini I.
      • Ruperti M.
      • Moccia S.
      • Mattos L.S.
      • Frontoni E.
      • De Momi E.
      Transfer learning for informative-frame selection in laryngoscopic videos through learned features.
      ] as shown in Table 2. Complex networks are graphs described by pairs of nodes and links that represent the elements of the system to be modelled and the iterations between the same, respectively, and allow to measure particularly informative topological features.
      Mostly, DL (Table 2) have been used for End-to-End learning, i.e. networks learn how to do tasks automatically from raw data provided to only interested task, whereas 7 articles developed a DL model as features extractor for classification [
      • Bevilacqua V.
      • Brunetti A.
      • Guerriero A.
      • Trotta G.F.
      • Telegrafo M.
      • Moschetta M.
      A performance comparison between shallow and deeper neural networks supervised classification of tomosynthesis breast lesions images.
      ,
      • Bizzego A.
      • Bussola N.
      • Chierici M.
      • Maggio V.
      • Francescatto M.
      • Cima L.
      • et al.
      Evaluating reproducibility of AI algorithms in digital pathology with DAPPER.
      ,

      Marco B, Leonardo B, Enrico G, Francesco S, Gastone C, Claudia T, et al. Circumventing the Curse of Dimensionality in Magnetic Resonance Fingerprinting through a Deep Learning Approach. ArXiv E-Prints 2018:ar:1811.11477.

      ,

      Castiglioni I, Ippolito D, Interlenghi M, Monti CB, Salvatore C, Schiaffino S, et al. Artificial intelligence applied on chest X-ray can aid in the diagnosis of COVID-19 infection: a first experience from Lombardy, Italy. MedRxiv 2020:2020.04.08.20040907. https://doi.org/10.1101/2020.04.08.20040907.

      ], regression [
      • Amoroso N.
      • La Rocca M.
      • Bellantuono L.
      • Diacono D.
      • Fanizzi A.
      • Lella E.
      • et al.
      Deep learning and multiplex networks for accurate modeling of brain age.
      ,
      • Spampinato C.
      • Palazzo S.
      • Giordano D.
      • Aldinucci M.
      • Leonardi R.
      Deep learning for automated skeletal bone age assessment in X-ray images.
      ,
      • Fiorentino M.C.
      • Moccia S.
      • Capparuccini M.
      • Giamberini S.
      • Frontoni E.
      A regression framework to head-circumference delineation from US fetal images.
      ,

      Famouri S, Morra L, Lamberti F. A Deep Learning Approach for Efficient Registration of Dual View Mammography. In: Schilling F-P, Stadelmann T, editors. Artif. Neural Netw. Pattern Recognit., Cham: Springer International Publishing; 2020, p. 162–72. https://doi.org/10.1007/978-3-030-58309-5_13.

      ], segmentation [
      • Moccia S.
      • Banali R.
      • Martini C.
      • Muscogiuri G.
      • Pontone G.
      • Pepi M.
      • et al.
      Development and testing of a deep learning-based strategy for scar segmentation on CMR-LGE images.
      ,
      • Comes M.C.
      • Filippi J.
      • Mencattini A.
      • Casti P.
      • Cerrato G.
      • Sauvat A.
      • et al.
      Multi-scale generative adversarial network for improved evaluation of cell–cell interactions observed in organ-on-chip experiments.
      ,

      Brunetti A, Cascarano GD, De Feudis I, Moschetta M, Gesualdo L, Bevilacqua V. Detection and Segmentation of Kidneys from Magnetic Resonance Images in Patients with Autosomal Dominant Polycystic Kidney Disease. In: Huang D-S, Jo K-H, Huang Z-K, editors., Cham: Springer International Publishing; 2019, p. 639–50.

      ,
      • Bevilacqua V.
      • Brunetti A.
      • Cascarano G.D.
      • Guerriero A.
      • Pesce F.
      • Moschetta M.
      • et al.
      A comparison between two semantic deep learning frameworks for the autosomal dominant polycystic kidney disease segmentation based on magnetic resonance images.
      ,
      • Panic J.
      • Defeudis A.
      • Mazzetti S.
      • Rosati S.
      • Giannetto G.
      • Vassallo L.
      • et al.
      A convolutional neural network based system for colorectal cancer segmentation on MRI images.
      ,
      • Giannini V.
      • Defeudis A.
      • Rosati S.
      • Cappello G.
      • Vassallo L.
      • Mazzetti S.
      • et al.
      Deep learning to segment liver metastases on CT images: impact on a radiomics method to predict response to chemotherapy.
      ,
      • Piantadosi G.
      • Sansone M.
      • Fusco R.
      • Sansone C.
      Multi-planar 3D breast segmentation in MRI via deep convolutional neural networks.
      ,
      • Valvano G.
      • Santini G.
      • Martini N.
      • Ripoli A.
      • Iacconi C.
      • Chiappino D.
      • et al.
      Convolutional neural networks for the segmentation of microcalcification in mammography imaging.
      ], or image registration [

      Famouri S, Morra L, Lamberti F. A Deep Learning Approach for Efficient Registration of Dual View Mammography. In: Schilling F-P, Stadelmann T, editors. Artif. Neural Netw. Pattern Recognit., Cham: Springer International Publishing; 2020, p. 162–72. https://doi.org/10.1007/978-3-030-58309-5_13.

      ]. Five works have used transfer learning to solve the task of interest [
      • Polsinelli M.
      • Cinque L.
      • Placidi G.
      A light CNN for detecting COVID-19 from CT scans of the chest.
      ,
      • Spampinato C.
      • Palazzo S.
      • Giordano D.
      • Aldinucci M.
      • Leonardi R.
      Deep learning for automated skeletal bone age assessment in X-ray images.
      ,
      • Brunese L.
      • Mercaldo F.
      • Reginelli A.
      • Santone A.
      Explainable deep learning for pulmonary disease and coronavirus COVID-19 Detection from X-rays.
      ,
      • Basaia S.
      • Agosta F.
      • Wagner L.
      • Canu E.
      • Magnani G.
      • Santangelo R.
      • et al.
      Automated classification of Alzheimer’s disease and mild cognitive impairment using a single MRI and deep neural networks.
      ], e.g. for The Inception V3 and Alexnet pretrained on ImageNet large-scale database were used for grading of meningioma [
      • Banzato T.
      • Causin F.
      • Della Puppa A.
      • Cester G.
      • Mazzai L.
      • Zotti A.
      Accuracy of deep learning to differentiate the histopathological grading of meningiomas on MR images: a preliminary study.
      ].
      DL is mostly performed using the Python programming language and the Keras library [

      Lizzi F, Laruina F, Oliva P, Retico A, Fantacci ME. Residual Convolutional Neural Networks to Automatically Extract Significant Breast Density Features. In: Vento M, Percannella G, Colantonio S, Giorgi D, Matuszewski BJ, Kerdegari H, et al., editors., Cham: Springer International Publishing; 2019, p. 28–35.

      ,
      • Fiorentino M.C.
      • Moccia S.
      • Capparuccini M.
      • Giamberini S.
      • Frontoni E.
      A regression framework to head-circumference delineation from US fetal images.
      ,

      Antonio F, Danilo C, Simone S, Michele S, Aurelio U. A multimodal deep network for the reconstruction of T2W MR images. ArXiv E-Prints 2019:ar:1908.03009.

      ,
      • Brunese L.
      • Mercaldo F.
      • Reginelli A.
      • Santone A.
      Explainable deep learning for pulmonary disease and coronavirus COVID-19 Detection from X-rays.
      ,
      • Dimauro G.
      • Ciprandi G.
      • Deperte F.
      • Girardi F.
      • Ladisa E.
      • Latrofa S.
      • et al.
      Nasal cytology with deep learning techniques.
      ,
      • Muscogiuri G.
      • Chiesa M.
      • Trotta M.
      • Gatti M.
      • Palmisano V.
      • Dell’Aversana S.
      • et al.
      Performance of a deep learning algorithm for the evaluation of CAD-RADS classification with CCTA.
      ]. Keras can be run as a stand-alone library or from TensorFlow (from tensorflow version greater than 2), an open-source software library for numeric computation [

      Brunese L, Mercaldo F AUID- ORCID: 0000-0002-9425-1657, Reginelli A, Santone A. Radiomics for Gleason Score Detection through Deep Learning. LID - E5411 [pii] LID - 10.3390/s20185411 [doi]. Sens Basel Switz JID - 101204366 n.d.

      ]. The Google Colaboratory interface provides free parallel GPU computing for Python Jupyter notebooks. Theano [
      • Zaffino P.
      • Pernelle G.
      • Mastmeyer A.
      • Mehrtash A.
      • Zhang H.
      • Kikinis R.
      • et al.
      Fully automatic catheter segmentation in MRI with 3D convolutional neural networks: application to MRI-guided gynecologic brachytherapy.
      ], Caffe[
      • Soomro M.H.
      • Coppotelli M.
      • Conforto S.
      • Schmid M.
      • Giunta G.
      • Del Secco L.
      • et al.
      Automated segmentation of colorectal tumor in 3D MRI using 3D multiscale densely connected convolutional neural network.
      ] and Pytorch [

      Mendizabal A, Tagliabue E, Brunet J-N, Dall’Alba D, Fiorini P, Cotin S. Physics-Based Deep Neural Network for Real-Time Lesion Tracking in Ultrasound-Guided Breast Biopsy. In: Miller K, Wittek A, Joldes G, Nash MP, Nielsen PMF, editors. Comput. Biomech. Med., Cham: Springer International Publishing; 2020, p. 33–45.

      ,
      • Kirienko M.
      • Sollini M.
      • Silvestri G.
      • Mognetti S.
      • Voulaz E.
      • Antunovic L.
      • et al.
      Convolutional neural networks promising in lung cancer T-parameter assessment on baseline FDG-PET/CT.
      ,
      • Tartaglione E.
      • Barbano C.A.
      • Berzovini C.
      • Calandri M.
      • Grangetto M.
      Unveiling COVID-19 from CHEST X-Ray with deep learning: a hurdles race with small data.
      ,
      • Sena P.
      • Fioresi R.
      • Faglioni F.
      • Losi L.
      • Faglioni G.
      • Roncucci L.
      Deep learning techniques for detecting preneoplastic and neoplastic lesions in human colorectal histological images.
      ] are other open source DL frameworks which can be run from Python.

      Clinical applications

      In the following sections we present the most representative studies on AI applied to imaging in Italy, grouped according to the clinical use of AI.

      Neurological applications

      The most common neuroimaging issues faced with AI tools are early diagnosis, biomarker identification and understanding the mechanism of development of neurodegenerative [
      • Morisi R.
      • Manners D.N.
      • Gnecco G.
      • Lanconelli N.
      • Testa C.
      • Evangelisti S.
      • et al.
      Multi-class parkinsonian disorders classification with quantitative MR markers and graph-based features using support vector machines.
      ,
      • Bandini A.
      • Orlandi S.
      • Escalante H.J.
      • Giovannelli F.
      • Cincotta M.
      • Reyes-Garcia C.A.
      • et al.
      Analysis of facial expressions in parkinson’s disease through video-based automatic methods.
      ,
      • Peruzzo D.
      • Arrigoni F.
      • Triulzi F.
      • Righini A.
      • Parazzini C.
      • Castellani U.
      A framework for the automatic detection and characterization of brain malformations: validation on the corpus callosum.
      ,
      • Nanni L.
      • Lumini A.
      • Zaffonato N.
      Ensemble based on static classifier selection for automated diagnosis of mild cognitive impairment.
      ] and oncological diseases [
      • Ugga L.
      • Cuocolo R.
      • Solari D.
      • Guadagno E.
      • D’Amico A.
      • Somma T.
      • et al.
      Prediction of high proliferative index in pituitary macroadenomas using MRI-based radiomics and machine learning.
      ,