- •An overview of AI tools applied to imaging for Healthcare is presented.
- •Code and available database for developing AI strategies are indicated.
- •The limitation and opportunity of AI application is discussed.
- •Explainable AI represents the new frontiers for penetrating the “black box” of ML algorithms.
Literature search strategy
Description of included studies and inclusion criteria
Training, validation and test set
|Image type and additional information||Available database|
|Digital Mammogram||The mini‑MIAS |
is a free scientific database for research and consists of 161 patients with 322 mammograms.
|The DDSM |
database consists of approximately 2500 patients with 10,239 multi‑view images including benign, malignant and normal cases.
|The Breast Cancer Wisconsin (BCW) dataset includes 683 cases (after removing 16 cases with missing values), 9 dimensions with integer values ranging from 0 to 10 (computed from a digitized image of fine needle aspirate of breast mass) and 2 classes (whether the diagnostic is benign or malignant) |
|The Mammographic Mass (MM) dataset includes 830 cases, 2 numeric dimensions (age and Breast Imaging Reporting And Data System value, BI-RADS), 3 categorical dimensions (shape, margin and density of the mass) and 2 classes (whether the diagnostic is benign or malignant) |
|The Breast Cancer (BC) dataset includes 286 cases, 4 numeric dimensions (age, tumor size, etc.), 4 categorical dimensions (breast quadrant, etc.) and 2 classes (whether the cancer is recurrent or not) |
|Dermatoscopic images||Biopsy verified skin lesion image set from the International Skin Imaging Collaboration (ISIC) archive |
|HAM10000 Dataset contains dermoscopic images of heterogeneous populations that are publicly accessible, anonymous and taken by different camera systems and thus have a high external validity |
|MRI||BRATS 2015 and BRATS 2016 brain tumor segmentations challenge datasets |
|The Autism Brain Imaging Data Exchange (ABIDE) repository contains pre-processed resting-state functional MRI (rs-fMRI) data from ASD (autism spectrum disorder) and healthy subjects across all independent data sites |
|Radiologic, molecular, genetic, pathological and clinical history||The Cancer Genome Atlas’ (TCGA) includes hepatocellular carcinoma (LIHC) and cholangiocarcinoma(CHOL) diagnostic FFPE WSI collections.|
|TCGA dataset metadata |
available through the TCGA’s Genomic Data Commons (GDC), including the original pathology reports (PDF files) containing the results of any additional immune-histochemical or special stains performed, as well as available radiologic, molecular diagnostic, and clinical history
|The Cancer Genome Atlas (TCGA) Lower-Grade Glioma|
(LGG) and Glioblastoma (GBM) projects
|TCGA’s dataset |
|Spectrometry||METASPACE datasets |
|Xray||GitHub covid-chest-X-ray dataset |
|EEG||ImageNet – EEG |
|Neuroimaging||the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database |
and the 273 regions designated in the Brainnetome Atlas
- •A DNN is composed of several hidden layers in which all neurons of a layer are connected to all the neurons of the following layer. The potential disadvantage is a slow learning process.
- •A recurrent neural network (RNN) is useful for the processing of temporally dependent information. The output of the network is a function not only of the input at a specific time but also of the inputs at previous times.
- •An auto-encoder (AE) is composed of a hidden layer that encodes layer for identifying a latent dominant structure of reduced size with respect to the input signal. The architecture is characterized by a full connection between neurons in input, hidden and output layers. Several variants have been developed; one of the advantages s that it does not require labeled data using unsupervised learning.
- •The Boltzmann machine (BM) is a model in which all the neurons are bidirectionally connected to each other; it can generate new input data during the unsupervised learning.
- •In Deep belief network (DBNs) only the two deepest layers have bidirectional connections. The training is unsupervised, except for the final adjustment of the network parameters performed in a supervised approach by adding a classification layer at the output of the network.
- •The generative adversarial networks (GANs) are based on two models: a generative model that produces synthetic data, and a discriminating model that estimates the probability that these data are part of the training data
- •CNN consists of several successive layers of data processing to find representative features of the input image, then more elaborate as the layers succeed each other. CNNs can be learned using both unsupervised or supervised approaches. CNNs are specifically designed to take advantage of spatially structured information such as medical images in which neighboring pixels corresponding to the same anatomical structure could share similar intensity characteristics; a specifically parameterized patch of neurons might detect pixels corresponding to the same anatomical region.
Application of AI tools
Declaration of Competing Interest
Appendix A. Supplementary data
- Supplementary data 1
- Supplementary data 2
- Supplementary data 3
- Supplementary data 4
Ranschaert E, Morozov S, Algra P. Artificial Intelligence in Medical Imaging Opportunities, Applications and Risks: Opportunities, Applications and Risks; 2019.
- Deep learning in medical imaging: general overview.Korean J Radiol. 2017; 18: 570-584
- The international MAQC Society launches to enhance reproducibility of high-throughput technologies.Nat Biotechnol. 2017; 35: 1127-1128
- Machine learning.McGraw-Hill, New York1997
- Survey on deep learning for radiotherapy.Comput Biol Med. 2018; 98: 126-146
- Deploying machine and deep learning models for efficient data-augmented detection of COVID-19 infections.Viruses. 2020; 12
- Deep learning in bioinformatics.Brief Bioinf. 2017; 18: 851-869
- Deep learning in neural networks: an overview.Neural Netw. 2015; 61: 85-117
- Optical coherence tomography-based deep-learning models for classifying normal and age-related macular degeneration and exudative and non-exudative age-related macular degeneration changes.Ophthalmol Ther. 2019; 8: 527-539
- Cognitive and system factors contributing to diagnostic errors in radiology.AJR Am J Roentgenol. 2013; 201: 611-617
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.BMJ. 2009; 339b2535
- Data division for developing neural networks applied to geotechnical engineering.J Comput Civil Eng. 2004; 18: 105-114
- A novel deep learning architecture outperforming 'off-the-shelf' transfer learning and feature-based methods in the automated assessment of mammographic breast density.Oncol Rep. 2019; 42: 2009-2015
- Explainable artificial intelligence for breast cancer: a visual case-based reasoning approach.Artif Intell Med. 2019; 94: 42-53
- A survey on case-based reasoning in medicine.Int J Adv Comput Sci Appl. 2016; 7
- The effectiveness of computerized clinical guidelines in the process of care: a systematic review.BMC Health Serv Res. 2010; 10: 2
- Emerging trends in geospatial artificial intelligence (geoAI): potential applications for environmental epidemiology.Environ Health. 2018; 17: 40
- The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions.Sci Data. 2018; 5180161
- Deep learning.Nature. 2015; 521: 436-444
- AdaptAhead optimization algorithm for learning deep CNN applied to MRI segmentation.J Digit Imaging. 2019; 32: 105-115
- A novel transfer learning approach to enhance deep neural network classification of brain functional connectomes.Front Neurosci. 2018; 12: 491
- Impact of a deep learning assistant on the histopathologic classification of liver cancer.NPJ Digit Med. 2020; 3: 23
- Predicting cancer outcomes from histology and genomics using convolutional networks.Proc Natl Acad Sci U S A. 2018; 115: E2970-E2979
- Toward a shared vision for cancer genomic data.N Engl J Med. 2016; 375: 1109-1112
- OffsampleAI: artificial intelligence approach to recognize off-sample mass spectrometry images.BMC Bioinf. 2020; 21: 129
- Using Artificial Intelligence for COVID-19 Chest X-ray Diagnosis.Fed Pract. 2020; 37: 398-404
Deng J, Dong W, Socher R, Li L, Kai L, Li F-F, editors. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009 20-25 June 2009.
Spampinato C, Palazzo S, Kavasidis I, Giordano D, Souly N, Shah M, editors. Deep Learning Human Mind for Automated Visual Classification. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 21-26 July 2017.
- EEG-based image classification via a region-level stacked bi-directional deep learning framework.BMC Med Inf Decis Making. 2019; 19: 268
- Diagnosis of Alzheimer's disease via multi-modality 3D convolutional neural network.Front Neurosci. 2019; 13: 509
- Generalizable, reproducible, and neuroscientifically interpretable imaging biomarkers for Alzheimer's disease.Adv Sci (Weinh). 2020; 7: 2000675
- AI-assisted forward modeling of biological structures.Front Cell Dev Biol. 2019; 7: 279
- Automated deep learning-based system to identify endothelial cells derived from induced pluripotent stem cells.Stem Cell Rep. 2018; 10: 1687-1695
- MitoSegNet: easy-to-use deep learning segmentation for analyzing mitochondrial morphology.iScience. 2020; 23 (101601)
- DeepM6ASeq: prediction and characterization of m6A-containing sequences using deep learning.BMC Bioinf. 2018; 19: 524
- De novo profile generation based on sequence context specificity with the long short-term memory network.BMC Bioinf. 2018; 19: 272
- Machine learning model for identifying antioxidant proteins using features calculated from primary sequences.Biology (Basel). 2020; 9
- CTRL - a label-free artificial intelligence method for dynamic measurement of single-cell volume.J Cell Sci. 2020; 133
- Comparing biological information contained in mRNA and non-coding RNAs for classification of lung cancer patients.BMC Cancer. 2019; 19: 1176
- Development and validation of deep-learning algorithm for electrocardiography-based heart failure identification.Korean Circ J. 2019; 49: 629-639
- Artificial intelligence algorithm for predicting mortality of patients with acute heart failure.PLoS ONE. 2019; 14e0219302
- An Algorithm based on deep learning for predicting in-hospital cardiac arrest.J Am Heart Assoc. 2018; 7
- Utility of machine learning algorithms in assessing patients with a systemic right ventricle.Eur Heart J Cardiovasc Imaging. 2019; 20: 925-931
- A Deep-Learning Algorithm (ECG12Net) for detecting hypokalemia and hyperkalemia by electrocardiography: algorithm development.JMIR Med Inf. 2020; 8e15931
- Noninvasive assessment of dofetilide plasma concentration using a deep learning (neural network) analysis of the surface electrocardiogram: a proof of concept study.PLoS ONE. 2018; 13e0201059
- Classification of melanocytic lesions in selected and whole-slide images via convolutional neural networks.J Pathol Inf. 2019; 10: 5
- Superior skin cancer classification by the combination of human and artificial intelligence.Eur J Cancer. 2019; 120: 114-121
- Dermoscopy diagnosis of cancerous lesions utilizing dual deep learning algorithms via visual and audio (sonification) outputs: laboratory and prospective observational studies.EBioMedicine. 2019; 40: 176-183
- Artificial intelligence and its effect on dermatologists' accuracy in dermoscopic melanoma image classification: web-based survey study.J Med Internet Res. 2020; 22e18091
- Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: automatic construction of onychomycosis datasets by region-based convolutional deep neural network.PLoS ONE. 2018; 13e0191493
- Evaluating reproducibility of AI algorithms in digital pathology with DAPPER.PLoS Comput Biol. 2019; 15e1006269
- Impact of pre-analytical variables on deep learning accuracy in histopathology.Histopathology. 2019; 75: 39-53
- Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.JAMA. 2017; 318: 2199-2210
- Local interpretable model-agnostic explanations for classification of lymph node metastases.Sensors (Basel). 2019; 19
- Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer.Sci Rep. 2017; 7: 45938
- Deep learning enables robust assessment and selection of human blastocysts after in vitro fertilization.NPJ Digit Med. 2019; 2: 21