If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
An overview of AI tools applied to imaging for Healthcare is presented.
•
Code and available database for developing AI strategies are indicated.
•
The limitation and opportunity of AI application is discussed.
•
Explainable AI represents the new frontiers for penetrating the “black box” of ML algorithms.
Abstract
The manuscript aims at providing an overview of the published algorithms/automation tool for artificial intelligence applied to imaging for Healthcare. A PubMed search was performed using the query string to identify the proposed approaches (algorithms/automation tools) for artificial intelligence (machine and deep learning) in a 5-year period. The distribution of manuscript in the various disciplines and the investigated image types according to the AI approaches are presented. The limitation and opportunity of AI application in the clinical practice or in the next future research is discussed.
The trend in medical imaging is an ever-increasing amount of information that can be viewed as a derived demand in each discipline. Improvements in computer technology and performance have made information processing faster, its speed has churned more information for physician to process.
Artificial intelligence (AI) is a branch of computer sciences [
Ranschaert E, Morozov S, Algra P. Artificial Intelligence in Medical Imaging Opportunities, Applications and Risks: Opportunities, Applications and Risks; 2019.
]. Its objective is the development of machines whose cognitive functions related to mimic the perception, learning, problem-solving and decision-making exceed that of humans. AI is the entire universe of computer technology that exhibits anything remotely resembling human intelligence.
] in which, based on the training dataset that are first provided, the computer develops its own logic for answering future questions. Given the large volume of data generated in many domains of science, AI appears uniquely advantageous for analysis of OMIC studies [
]. The key concept of machine learning is to produce accurate predictions on new unseen data after being trained on a finite learning dataset. To concretely define what learning means, Mitchell et al. proposed to specify three parameters, namely T, P, and E [
]: “a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance on tasks in T, as measured by P, improves with experience E.” [
Advances in ML techniques may facilitate processing of large amounts of medical image data. Notwithstanding their utility, ML methods are known to have limitations [
] related to: manual extraction and selection of features, this is a fundamental task in order to find a group of significant variables to predict and correlate with outcome; Poor performance when dealing with imbalanced datasets; Over-fitting generating a model that corresponds exactly to the input training set of data and may therefore fail to fit new data, when this occurs, the algorithm will be accurate (low error rate between predicted and actual results) on the training data set, but highly inaccurate on the testing data set; Complexity and time consumption, it is important to remind that the AI model can aid the human decision process for example in routine medical application only if their results are robust and obtainable in a reasonable computational time.
Deep learning (DL) is the newest class of ML and has been found to be advantageous to other forms ofML [
]. DL employs multiple layers of neural networks, leading to expanded ‘neuronal’ complexity, to significantly enhance computational power. DL has been recently applied to bioinformatics.
Furthermore, deep learning has ability to find out irrelevant and particular minute variations, which allows these methods to reach higher accuracy than other machine learning methods [
Optical coherence tomography-based deep-learning models for classifying normal and age-related macular degeneration and exudative and non-exudative age-related macular degeneration changes.
Deep neural networks (DNNs) are forms of machine learning methods. In machine learning, it is often necessary to reduce the complexity of the input data and make relevant patterns more visible forthe learning algorithms to function. Indeed, their performance greatly depends on how accurately these features have been identified and extracted. Given that this feature engineering process is based on domain knowledge and is specific to the data type, it is difficult and expensive in terms of time and expertise to apply. In contrast, DNNs independently learn a hierarchical representation of the input data adapted to the task at hand; this eliminates the task of developing new features extractors for every problem. The main drawback is that DNNs require a large amount of input data to be effective. The idea of a computer program that could find itself representing a model from a dataset is not new. Perceptron isone of the first approaches to conceptualize data directly from the environment
As discussed by Meyer et al the DNN’s performance greatly depends on how accurately the features have been identified and extracted. Using DNNs the task of developing new features extractors for specific problem is not necessary because the algorithm independently learn a hierarchical representation of the input data. To perform these task, the DNN based algorithms require a large number of input data [
The predictive power on artificial neural networks (ANNs) depends on the number of multiple layers of information allowing the analysis of datasets with more complex patterns. These algorithms are enabling pivotal advances in text/speech and image recognition.
ML and DL algorithms use different learning approaches, as discussed by Meyer et al.[
] the most important methods are supervised, unsupervised, reinforcement, and transfer learning. In supervised learning, the algorithm uses training inputs and their corresponding outputs in order to learn a rule that connects inputs to outputs. In unsupervised learning, no referenced outputs are used by the algorithm to find a structure in the inputs. Reinforcement learning is based on communication and exploration; a feedback is used to adjust the learning process. In cases in which the available input data are insufficient to train a network (i.e. medical applications), transfer learning can be used to adapt a pre-trained network to perform a new task; the fine-tune of the network weights can be performed using a small amount of coherent labeled data [
Supervised learning techniques are undoubtedly the most widely used methods in ML and those with the best results. These procedures rely on a dataset from which the response variable to be predicted (eg. diagnosis, parameter, segmentation) by two or more classes using a series of variables for prediction and classification algorithms or continuous values for regressive algorithms.
Unfortunately, AI tools suffer from an inability to correctly detect and classify cases that they have not previously analyzed. Thus, the reliability and quality of the data source are essential for an algorithm to be realistic, correct, not biased, and applicable into realistic clinical contexts. The images are generally labeled with two or more conditions (e.g. healthy/sick) by two or more experts or through radiology/specialists reports.
With the advancement of artificial intelligence in medical diagnostics, many studies have been devoted to compare whether AI or diagnostician which is expected to be better than diagnostician alone. Thanks to these approaches often several hundred measurable features which are beyond the search limits of the human retina and require computational power for both detection and analysis are available for each image. Thus, the specialists are overwhelmed not just with what they can see but with what is exceeding their cognitive bandwidth [
Another trend is the evolution from visual subjectivity to quantitative objectivity, which reduces the inter-observer variation with this subjective assessment, while improving the consistency of parameters which are objectively quantified.
The impact of AI on the various disciplines depends on the balance between how fast it automates old tasks and how many new information it generates. In the short-term, AI may create more work for experts because it may increase information at a faster rate than it removes work by automating tasks.
To evaluate the state of the art of ML and DL methods applied to medical imaging in Healthcare, a PubMed search has been conducted with the purpose of discussing the actually implemented methods, the investigated images, highlighting the efforts in multicentric initiative and the limitation of studies as strategy to reinforce the technology pillar.
Methods
Literature search strategy
A PubMed search was performed using the query string to identify the proposed approaches (algorithms/automation tools) for artificial intelligence (machine and deep learning) applied to medical imaging for Healthcare. Query search included the following keywords/string:
Search: (algorithm OR automation) AND “deep learning” AND “machine learning” AND “artificial intelligence” NOT dental NOT review Filters: from 2015/9/27–2020/9/28. The research was restricted to the last five years to include only the most recently published studies. The search was done on 28thSeptember 2020.
Study selection
Titles and abstracts were independently reviewed by three authors to decide study inclusion. Full articles were retrieved when the abstract was considered relevant and only papers or abstracts published in English were considered. Papers were considered eligible when 1) the main and/or secondary study aim included the application of algorithms/automation tools for artificial intelligence applied to medical imaging or Healthcare; 2) involved a number of patients of training/test set >30 or investigated at least one type of image modality with a number of images >50. The data were summarized in a database with the following issues: first author, journal, year, title, discipline/area, exclusion/issues, AI area, number of images, number of patients, training set size, test set size, validation set size, algorithm, main aim (classification, detection, pattern recognition, predict outcome, segmentations), image type, specific image type, code algorithm, secondary study aim (improve accuracy/knowledge/performances/efficiency, comparison, other), for the subsequent data analysis.
Results
Description of included studies and inclusion criteria
Based on reported PubMed/Medline search, 169 paper and abstracts reporting or proposing algorithms/automation tool for artificial intelligence (machine and deep learning) applied to medical imaging for Healthcare were identified. The number of all identified papers per year increased as shown in Fig. 1 according to the inclusion/exclusion judgment. An exponential growth can be observed looking at the included papers trough years. The number of papers related to the 2020 was not complete due to the inclusion criteria limited to the 2020/9/28.
Fig. 1The number of identified papers per year according to inclusion (yes) and exclusion (no) judgment identified from 2015 to 09-27 to 2020-09-28.
Optical coherence tomography-based deep-learning models for classifying normal and age-related macular degeneration and exudative and non-exudative age-related macular degeneration changes.
A novel deep learning architecture outperforming 'off-the-shelf' transfer learning and feature-based methods in the automated assessment of mammographic breast density.
Noninvasive assessment of dofetilide plasma concentration using a deep learning (neural network) analysis of the surface electrocardiogram: a proof of concept study.
Dermoscopy diagnosis of cancerous lesions utilizing dual deep learning algorithms via visual and audio (sonification) outputs: laboratory and prospective observational studies.
Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: automatic construction of onychomycosis datasets by region-based convolutional deep neural network.
Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF.
Validation of Deep Convolutional Neural Network-based algorithm for detection of diabetic retinopathy - Artificial intelligence versus clinician for screening.
Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations Using IBM Watson's natural language processing algorithm.
Automated detection of macular diseases by optical coherence tomography and artificial intelligence machine learning of optical coherence tomography images.
Comparative effectiveness of convolutional neural network (CNN) and recurrent neural network (RNN) architectures for radiology text report classification.
Application of explainable ensemble artificial intelligence model to categorization of hemodialysis-patient and treatment using nationwide-real-world data in Japan.
A deep artificial neural network-based model for prediction of underlying cause of death from death certificates: algorithm development and validation.
Artificial intelligence-based differential diagnosis: development and validation of a probabilistic model to address lack of large-scale clinical datasets.
Beginnings of artificial intelligence in medicine (AIM): computational artifice assisting scientific inquiry and clinical art - with reflections on present AIM challenges.
Breathing signature as vitality score index created by exercises of qigong: implications of artificial intelligence tools used in traditional Chinese medicine.
ASN-ASAS SYMPOSIUM: FUTURE OF DATA ANALYTICS IN NUTRITION: Mathematical modeling in ruminant nutrition: approaches and paradigms, extant models, and thoughts for upcoming predictive analytics1,2.
Improving classification of pollen grain images of the POLEN23E dataset through three different applications of deep learning convolutional neural networks.
Developing intelligent algorithm as a machine learning overview over the big data generated by Euler-Euler method to simulate bubble column reactor hydrodynamics.
An extensible big data software architecture managing a research resource of real-world clinical radiology data linked to other health data from the whole Scottish population.
] screened were excluded for the following reasons: Not inherent (# 55) and Overview/Summary (# 27). Out of the 87 full-text articles assessed for eligibility, 5full-text articles were excluded with reasons(i.e. few patients or not images). Out of 82 included papers (reported in supplementary materialTable S1), the following subgroups methods were obtained: Artificial Intelligence (# 6), Deep Learning (# 65) or Machine learning (# 11). The excluded papers are reported in supplementary materialTable S2.
Fig. 2PRISMA Flow-Diagram of study selection and inclusion in the systematic review.
The distribution of papers according to the discipline and type or algorithm is reported in Fig. 3. Out of 65 paper focusing on DL, the majority regarded the radiology (# 8), oncology (# 12), neurology (# 7), cardiology (# 6) and biology (# 6) field. Out 7 papers on AI, the radiology and oncology field were investigated in 2 studies. Out of 10 papers using ML, # 3 focused on the field of biology (for more details see Fig. 3).
Fig. 3The distribution of investigated papers according the type or algorithm and the discipline.
The main aim of the included manuscripts can be summarized as follows: classification (# 40), detection (# 37), segmentation (# 9), predict outcomes (# 12). A combination of them, to the used algorithm type, was reported in Fig. 4.
Fig. 4The distribution of investigated papers according to the main aim of the paper.
Forty-six of the 82 included papers are related to imaging analysis belonging to several fields of medicine. The medical image types investigated can be clustered in different area groups: AI (# 5), DL (# 37), ML (# 4). The types of investigated images are reported in supplementary materialTable S3.
Training, validation and test set
A robust application of AI strategies in medical studies requires a training, validation and test phases. The training dataset is a set of data used to fit the parameters of the model such as weights of connections between neurons. The training dataset often consists of pairs of an input vector (or scalar) and the corresponding output vector (or scalar). The model result is compared with the expected one to adjust the model parameters. Each iteration calculates the difference between the predicted and actual outcomes and refines the algorithms weightings accordingly while reducing this difference. The produced algorithm is tailored specifically to the training data set. To optimize the performance of the algorithm, assess its generalizability of the final algorithm and its learnt parameters, the model is applied on the test set, also used for tuning hyperparameters. The validation dataset is data used to evaluate the model on a “new” external set of data.
The number of hidden units, layers or layer width can for example be iteratively tuned stopping training when the error on the validation dataset increases, as this is a sign of over fitting to the training dataset. The test dataset provides finally an unbiased evaluation of a final model fit using a completely new set of data.
], the training dataset often consists of pairs of an input vector (or scalar) and the corresponding output vector (or scalar) and it is a set of data used to fit the parameters of the model such as weights of connections between neurons, for example. It is well-understood that the training set performance tends to overestimate the test set accuracy, therefore It is advantageous to use a test set to evaluate the trained model's performance during hyper-parameter search and model optimization. These are commonly called “Resampling methods” and one of the most popular method is “k-fold cross validation” or “leave-x-out cross validation” where the training set is divided in k parts and each of the n original cases has been left out exactly once (x = n/k). The resulting algorithm will then be tailored specifically to the training data. There are many possible ways to split the original dataset and this task could be dependent on data availability (e.g., 20% of dataset can be used for training). A split providing 20% validation and 30% test from the remaining 80% has been shown to produce good generalization from the test to validation set accuracy [
The mean value of percentage of patients/images used as training, test and validation dataset according to the different approach is shown in Fig. 5. Artificial Intelligence area includes studies in which algorithm is no better specified to belong at ML or DP area. The training set were 72%, 73%, 78% for AI, DP and ML, respectively. The test set were 7%, 20%, 17% for AI, DP and ML, respectively. Finally, 21%,7% and 5% were the percentage for validation test, respectively.
Fig. 5The mean value of percentage of patients/images used as training, test and validation dataset according to the different approach.
The percentage of them, considering all the included articles, are: 71%, 6% and 23% for training, validation and test purposes respectively.
Several tools were implemented for improving the accuracy, the efficiency and the performance of the models for detection/classification and outcome prediction. The input images ranged from CT and radiographs to dermoscopy, microscopy or spectroscopy. Our overview indicates that the amount of research on image-based AI models is not balanced across districts, as some of them, such as brain, breast, liver and lungs have been receiving more attention than other sites. Images were extracted from mono-institutional or multicentric database or free available datasets (e.g., Table 1).
Table 1Examples of free available datasets used in the reviewed AI studies.
A novel deep learning architecture outperforming 'off-the-shelf' transfer learning and feature-based methods in the automated assessment of mammographic breast density.
A novel deep learning architecture outperforming 'off-the-shelf' transfer learning and feature-based methods in the automated assessment of mammographic breast density.
database consists of approximately 2500 patients with 10,239 multi‑view images including benign, malignant and normal cases.
The Breast Cancer Wisconsin (BCW) dataset includes 683 cases (after removing 16 cases with missing values), 9 dimensions with integer values ranging from 0 to 10 (computed from a digitized image of fine needle aspirate of breast mass) and 2 classes (whether the diagnostic is benign or malignant)
The Mammographic Mass (MM) dataset includes 830 cases, 2 numeric dimensions (age and Breast Imaging Reporting And Data System value, BI-RADS), 3 categorical dimensions (shape, margin and density of the mass) and 2 classes (whether the diagnostic is benign or malignant)
The Breast Cancer (BC) dataset includes 286 cases, 4 numeric dimensions (age, tumor size, etc.), 4 categorical dimensions (breast quadrant, etc.) and 2 classes (whether the cancer is recurrent or not)
HAM10000 Dataset contains dermoscopic images of heterogeneous populations that are publicly accessible, anonymous and taken by different camera systems and thus have a high external validity
The Autism Brain Imaging Data Exchange (ABIDE) repository contains pre-processed resting-state functional MRI (rs-fMRI) data from ASD (autism spectrum disorder) and healthy subjects across all independent data sites
available through the TCGA’s Genomic Data Commons (GDC), including the original pathology reports (PDF files) containing the results of any additional immune-histochemical or special stains performed, as well as available radiologic, molecular diagnostic, and clinical history
The Cancer Genome Atlas (TCGA) Lower-Grade Glioma (LGG) and Glioblastoma (GBM) projects
Deng J, Dong W, Socher R, Li L, Kai L, Li F-F, editors. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009 20-25 June 2009.
Spampinato C, Palazzo S, Kavasidis I, Giordano D, Souly N, Shah M, editors. Deep Learning Human Mind for Automated Visual Classification. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 21-26 July 2017.
Several codes according to the main aim have been implemented/used as summarized in Table S3 (supplementary data) according to the main aim of papers. Of note, only a limited part of the reported software has been compared one each other within the indicated studies.
The most frequently used AI Network architectures [
A DNN is composed of several hidden layers in which all neurons of a layer are connected to all the neurons of the following layer. The potential disadvantage is a slow learning process.
•
A recurrent neural network (RNN) is useful for the processing of temporally dependent information. The output of the network is a function not only of the input at a specific time but also of the inputs at previous times.
•
An auto-encoder (AE) is composed of a hidden layer that encodes layer for identifying a latent dominant structure of reduced size with respect to the input signal. The architecture is characterized by a full connection between neurons in input, hidden and output layers. Several variants have been developed; one of the advantages s that it does not require labeled data using unsupervised learning.
•
The Boltzmann machine (BM) is a model in which all the neurons are bidirectionally connected to each other; it can generate new input data during the unsupervised learning.
•
In Deep belief network (DBNs) only the two deepest layers have bidirectional connections. The training is unsupervised, except for the final adjustment of the network parameters performed in a supervised approach by adding a classification layer at the output of the network.
•
The generative adversarial networks (GANs) are based on two models: a generative model that produces synthetic data, and a discriminating model that estimates the probability that these data are part of the training data
•
CNN consists of several successive layers of data processing to find representative features of the input image, then more elaborate as the layers succeed each other. CNNs can be learned using both unsupervised or supervised approaches. CNNs are specifically designed to take advantage of spatially structured information such as medical images in which neighboring pixels corresponding to the same anatomical structure could share similar intensity characteristics; a specifically parameterized patch of neurons might detect pixels corresponding to the same anatomical region.
Application of AI tools
This section presents the evolution of the contributions of AI according to the different application areas. The supplementary material for this article includes a list of the contributions of the AI techniques applied on disciplines listed in Fig. 3 considering various typologies of images.
Biology
AI tools have been used in Classification/Detection or in outcomes prediction and among other for modeling of biological structures [
Noninvasive assessment of dofetilide plasma concentration using a deep learning (neural network) analysis of the surface electrocardiogram: a proof of concept study.
In dermatology, the recognition of visual patterns is fundamental for skin diagnosis. AI tools are expected increasing the sensitivity and specificity in classifying images when combined with dermatologist’ expertise. Recently, whole-slide imaging now enables dermatology to storage a vast amounts of images [
Dermoscopy diagnosis of cancerous lesions utilizing dual deep learning algorithms via visual and audio (sonification) outputs: laboratory and prospective observational studies.
Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: automatic construction of onychomycosis datasets by region-based convolutional deep neural network.
Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF.
Validation of Deep Convolutional Neural Network-based algorithm for detection of diabetic retinopathy - Artificial intelligence versus clinician for screening.
Regarding the field of radiology, several type of images were investigated (including radiographs, screening mammograms, CT images, MRI, Computed tomography pulmonary angiography) for the automatic determination of the need for intravenous contrast in musculoskeletal MRI [
Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations Using IBM Watson's natural language processing algorithm.
Diabetic eye disease is amongst the most common conditions seen in routine ophthalmology practice and constitutes a significant and growing public health issue. Diabetic retinopathy is the commonest cause of vision loss in working age adults. The outcomes of these patients improve dramatically with early detection using digital imaging (high-resolution color retinal photography and optical coherence tomography). The utility of the AI algorithms to supplement clinician grading has been explored for diabetic retinal screening, diagnosis and appropriate referral of other sight threatening disorders, or replace current investigations. More in details, AI tools have been applied for identification of macular diseases or its degeneration [
Optical coherence tomography-based deep-learning models for classifying normal and age-related macular degeneration and exudative and non-exudative age-related macular degeneration changes.
Automated detection of macular diseases by optical coherence tomography and artificial intelligence machine learning of optical coherence tomography images.
The application of AI to imaging has generated significant interest within the medical disciplines such as radiology, pathology, ophthalmology, and dermatology. The selected papers, identified from the medical literature using the described search strategy, although there was some heterogeneity across included studies, were quite similar in terms of study design, methodology, analyses, and reporting, but present several limitations recognized by the Authors. In several studies [
] the data selection induced several biases. When only biopsy-verified images are considered, a certain bias is introduced, as these lesions are difficult to diagnose likely because during clinical examination of the patient, more information is available for the dermatologist for the diagnosis than just the visual impression of the examined skin area.
], the test data was enriched with cases containing metastases and, specifically, micro metastases and, thus, is not directly comparable with the mix of cases pathologists encounter in clinical practice. In another, low-quality images and patients who had other concomitant diseases were excluded, so additional studies are necessary to ensure the utility of these methods in future clinical practice, such as telemedicine [
Optical coherence tomography-based deep-learning models for classifying normal and age-related macular degeneration and exudative and non-exudative age-related macular degeneration changes.
], a relatively small portion of each slide was used for training and prediction, and the selection of ROIs within each slide required expert guidance. if a certain proportion of images is manually excluded, this could be problematic when trying to incorporate this process into the clinical workflow using images from PACS in the hospital. Thus, future studies should explore more advanced methods for automatic selection of regions and for incorporating a higher proportion of each images in training and prediction to better account for image heterogeneity.
The use of models based on training data which is not representative of the population, case mix, modalities, and acquisition protocols can compromise performance and confidence in its use, particularly if over-fitting has occurred [
]. Re-evaluations of algorithm performance in the clinical practice is a mandatory task and requires a practical understanding of these potential pitfalls to ensure the most responsible deployment of AI tools in practice.
In addition, most studies were carried out in a single center with limited data availability, e.g. [
]. To avoid/reduce the over-fitting, the image set was augmented by randomly rotating the image several times and then flipping the rotated image horizontally [
] or covered a period of >10 years so the quality of data is unclear/questionable, and image quality varied between patients and across the study period [
AI tools are known as a “black box” because it is used to find the relationship between the input data and a result, not to create a rule based on knowledge and/or support the decision for a clinical endpoint/decision making [
]. Thus, interpretable DL needs to be implemented/developed.
Of note, the Dialogue on Reverse Engineering Assessment and Methods (DREAM) Consortium has run dozens of biomedical challenges, establishing robust and objective computational benchmarks in multiple disease areas and across multiple data modalities [
], including the development and validation of breast cancer detection algorithms and the assessment of whether machine learning methods applied to mammography data can improve screening accuracy.
Segmentation plays an important role in traditional classification methods and is also known to be time-consuming, so the possibility to extract useful features directly from raw images is challenging and of great interest, but currently limited by the calculation resources.
To improve the replication of methods, several Authors freely disseminated the source codeor implemented websites (see Table S3) for demonstrating the potential of AI tools. At this stage only a limited part of the reported software/tools has been compared one each other. This can be considered an important improvement of the field. Open-source allows fostering future investigations on a massive scale. The most used open-source programming languages are currently Python and R, which can avail of libraries with implementations of the most widely used techniques and algorithms.
Moreover, the performance of the algorithm was somewhat limited by heterogeneity in the training data [
Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations Using IBM Watson's natural language processing algorithm.
Within certain limitations and considering a purely image-based setting, artificial intelligence can achieve on par or superior performance to physician, thereby highlighting its potential as a decision-support system with immediate clinical implications [
]. In terms of data, AI efforts are expected to shift from processed medical images to raw acquisition data and unsupervised learning techniques to fully utilize all the available information.
At the state-of-the-art AI is still in its infancy, and it is also evident that AI is unlikely to replace radiologists or other specialists but is expected to evolve into a valuable educational resource useful for discover hidden information that might have been overlooked. Initiatives are underway to standardize the requirements that must be met by an AI algorithm.
Regarding the privacy, healthcare entities have been able to use and share de-identified records for research and development, either with patient consent or with waivers, and ensure the true de-identification or anonymization. For example, successful de-identification of medical imaging data stored in the DICOM format requires complete deletion or overwriting metadata. Another important limitation is the opacity and interpretability of these techniques used as ‘‘black boxes’’, without any biological explanation. Explainable AI represents the new frontiers for penetrating the “black box” of ML algorithms and build the basis of paradigm shifts in future image-based analysis in the daily clinical practice exceeding the cognitive bandwidth of human perception. The synergy of the human interpreter with the results of the artificial intelligence algorithm has not been fully addressed, nor how AI would affect the final assessments of radiologists [
]. This is an area that requires more research efforts.
For these reasons, the application of AI on 2D/3D images continues to be an active research area for the application of precision medicine or research progresses of complex patterns from clinical databases.
Future prospective
In recent years, AI has been successfully used to solve real problems in a wide range of applications. A potential opportunity for AI tools is its capability of combine the information from the multiple image modalities (acquired at the baseline and during the patient follow-up) and clinical/genomic data to interpret the data or predict the outcome for a given task. This task could support the physicians in a multidisciplinary context producing a synergic support. In addition, AI tools might be used for building predictive models from the data, understanding of Radiomics, biological and genetics pathways to domain knowledge of a specific type of diseases and how they manifest in a given or multiple medical imaging modalities.
Advanced AI efforts are focused on the complex tasks currently done by “human” experts/physicians. Surprisingly, despite a large portion of its daily routine is devoted to high-tech diagnostic imaging, there have still been fewer efforts of AI in diagnostic imaging [
A potential role for AI-assisted diagnosis is the qualitative/quantitative image processing with extraction of radiomic features and their time-dependent changes. A strategic role for AI techniques would be the quality assurance of image device by reviewing the rawdata, detecting the motion artifacts and noise and correcting them by applying relevant correction tools without human supervision and intervention. The introduction of AI might change approval criteria and new quality control parameters and bring further challenges of validation standards.
Another potential role for AI is the triage of requests by screening the available institutional digital archive (previous images, reports, tests) and distinguishing the patients with high probability of disease/recurrence. For follow-up purposes or in repetitive screening tests (such as mammography, chest X-rays, PETC/CT or CT imaging) AI could support physicians probably in a shorter time, with less interobserver variability and greater accuracy than “human” radiologists, more effort and time are required for implementing and validating AI tools. In addition, once the patients are scanned, the AI tools would be potentially coded to screen the scans and decide if additional scans in different positions/regions or at a different time point is needed. This feature would reduce the patient waiting time and physician workload. In this context, AI may potentially open the way to a better individualized patient care and precision medicine. AI would be of further identify the most recently published papers and guidelines that are relevant to the key findings/disease detected on the current scan aiding to prepare a most up-to-date customized patient report. This time-consuming focused literature search is already a part of routine clinical reporting for selected examinations in some academic departments.
Dosimetry is another potential area of work for AI tools by performing precise segmentation of organs and lesions and calculations of absorbed dose using data obtained from AI improved three-dimensional imaging. This would be potentially useful for both diagnostic and therapeutic radionuclide procedures. Also, optimization of imaging equipment technology, criteria, and parameters through AI tools integrated to imaging devices can possibly bring the opportunity to reduce unnecessary patient absorbed dose (e.g. administered activity in nuclear medicine procedures). As another future direction is the potential applications of AI approaches in the radiotherapy workflow [
]. A decision support tool can be used for patient consultation purposes. In the field of image processing, metal artifact reduction, synthetic CT from MRI and image quality improvements can be applied to the planning acquisition process (CT, MRI, PET). For target and structure segmentation, AI algorithms can be used for auto-detection/auto-segmentation of OARs and target volumes and for multi-modality image registration. Dose prediction tool and on-line adaptive radiotherapy application could be used to support the treatment planning process. During the treatment delivery, AI approaches could be used for the image-based motion managementand patient setup (e.g., object recognition, collision avoidance, intra-fraction motion prediction, surface guide radiotherapy).
Once they are validated, AI tools has the potential to represent a standard in real-life clinical applications and to reduce repetitive, time-consuming, and monotonous tasks. This may even result in improved patient care and increased life quality of physicians in whom “burnout syndrome” due to heavy workload has been getting a serious problem [
Finally, the implementation of AI programs containing sensitive health information leads to cyber security risk and their reduction/prevention represents another challenge of these tools in clinical practice [
Although the number of AI algorithms, studies and applications to medical imaging will increase rapidly in the coming years, the main obstacle to this development could be related to the lack of homogeneous training data. Multicentric studies will have an important role in this research area. The use of inter operable standards and homogeneous protocols will also be needed before such study can be performed. AI methods can provide useful models for quality assurance, personalized and predictive medicine. For this purpose, the contribute of clinicians and researchers in the interpretation of models and their application has a crucial role in the daily clinical practice.
Funding
This study was partially supported by Associazione Italiana per la Ricerca sul Cancro (AIRC) to LS (IG 2017 ID. 20809).
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
Ranschaert E, Morozov S, Algra P. Artificial Intelligence in Medical Imaging Opportunities, Applications and Risks: Opportunities, Applications and Risks; 2019.
Optical coherence tomography-based deep-learning models for classifying normal and age-related macular degeneration and exudative and non-exudative age-related macular degeneration changes.
A novel deep learning architecture outperforming 'off-the-shelf' transfer learning and feature-based methods in the automated assessment of mammographic breast density.
Deng J, Dong W, Socher R, Li L, Kai L, Li F-F, editors. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009 20-25 June 2009.
Spampinato C, Palazzo S, Kavasidis I, Giordano D, Souly N, Shah M, editors. Deep Learning Human Mind for Automated Visual Classification. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 21-26 July 2017.
Noninvasive assessment of dofetilide plasma concentration using a deep learning (neural network) analysis of the surface electrocardiogram: a proof of concept study.
Dermoscopy diagnosis of cancerous lesions utilizing dual deep learning algorithms via visual and audio (sonification) outputs: laboratory and prospective observational studies.
Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: automatic construction of onychomycosis datasets by region-based convolutional deep neural network.
00143-5&id=gr1.jpg){kind=link}
00143-5&id=gr2.jpg){kind=link}
00143-5&id=gr3.jpg){kind=link}
00143-5&id=gr4.jpg){kind=link}
00143-5&id=gr5.jpg){kind=link}