Current applications of deep-learning in neuro-oncological MRI

Purpose: Magnetic Resonance Imaging (MRI) provides an essential contribution in the screening, detection, diagnosis, staging, treatment and follow-up in patients with a neurological neoplasm. Deep learning (DL), a subdomain of artificial intelligence has the potential to enhance the characterization, processing and interpretation of MRI images. The aim of this review paper is to give an overview of the current state-of-art usage of DL in MRI for neuro-oncology. Methods: We reviewed the Pubmed database by applying a specific search strategy including the combination of MRI, DL, neuro-oncology and its corresponding search terminologies, by focussing on Medical Subject Headings (Mesh) or title/abstract appearance. The original research papers were classified based on its application, into three categories: technological innovation, diagnosis and follow-up. Results: Forty-one publications were eligible for review, all were published after the year 2016. The majority (N = 22) was assigned to technological innovation, twelve had a focus on diagnosis and seven were related to patient follow-up. Applications ranged from improving the acquisition, synthetic CT generation, auto-segmentation, tumor classification, outcome prediction and response assessment. The majority of publications made use of standard (T1w, cT1w, T2w and FLAIR imaging), with only a few exceptions using more advanced MRI technologies. The majority of studies used a variation on convolution neural network (CNN) architectures. Conclusion: Deep learning in MRI for neuro-oncology is a novel field of research; it has potential in a broad range of applications. Remaining challenges include the accessibility of large imaging datasets, the applicability across institutes/vendors and the validation and implementation of these technologies in clinical practise.


Introduction
The field of Artificial Intelligence (AI) is evolving at a rapid speed. The exponential growth of computational algorithms, like artificial intelligence methods are expected to improve diagnosis, therapy and follow-up in medicine [1]. Especially imaging related studies in health care are emerging in the subdomain of AI called deep-learning (DL) [2,3]. In contrast to traditional machine learning (ML), where careful engineering is necessary to define and extract elements (features) to detect or classify patterns in the image, deep-learning allows the use of raw imaging data and can automatically discover the representations needed for detection or classification [3]. Deep-learning based AI technology therefore provides unprecedented enhancements in terms of (automated) image analysis in many fields of medicine [4]. For oncological investigations, typical applications are in the area of diagnosis and staging of cancer, treatment decision and individual treatment optimization including prognosis modelling, and follow-up imaging. Many imaging modalities have opportunities to aid in the care path of cancer patients. In this review we will focus on the most frequently used imaging method applied to neuro-oncology, namely magnetic resonance imaging (MRI). Given the large amount of data currently generated on MRI scanners applying different image acquisition sequences and postprocessing steps, deep-learning technology is ideally suited for analysis of these large scale, multi-dimensionalimage sets. MRI and (automated) analysis of MRI data is one of the cornerstones for these previous mentioned applications in the neuro-oncology domain [5].
Data scientist have an increasing role in the image analysis and interpretation of advanced MRI images, due to the generation of a large amount of data. For example, diffusion tensor imaging (DTI) is used to measure the directionally of proton motion, which is often altered in the presence of brain tumors [6]. For this application physicists, radiologists and computer scientists needed to collaborate to extract the maximum amount of information captured in these imaging sequences. With the increase in computing power and advanced programming algorithms, the use of deep-learning algorithms to extract relevant information from MRI imaging is expected to increase even further.
The aim of this review paper is to give an overview of the current state-of-art of applications of deep-learning in MRI with a specific focus on neuro-oncology. For this investigation, we categorized the available literature into various domains following the clinical care path from technology to diagnosis and follow-up.

Search strategy
To provide an overview of the available literature combining Deep Learning and MRI in neuro-oncology we used the Pubmed database and defined a specific search strategy including a combination of MRI, DL, neuro-oncology and corresponding search terminologies. We applied a search terminology by focussing on Mesh terms or title/abstract appearance. The specific search string was: "Magnetic Resonance Imaging" [Mesh] AND "Deep Learning"[Mesh] AND (Neuro-oncology [tiab] OR "Brain Neoplasms" [Mesh] OR "Central Nervous System Neoplasms" [Mesh] OR "Neoplasms, Neuroepithelial" [Mesh] OR "Meningeal Neoplasms" [Mesh])

Data extraction
From the original publications, we extracted title, first author, journal, year of publication, study type, goal of study, patient population, sample size, DL technology and MRI technology used. In addition, we classified the original research papers into three categories: 1) technological innovations 2) diagnosis and 3) follow-up.

Results
The search strategy resulted in 45 publications (date: 1st November 2020). We excluded publications written in Chinese [N = 1], reviews or editorial publications [N = 4], publications not specifically focussing on DL applications using MRI images [N = 1] and publications without an available full text [N = 1]. From reference searching in these papers, we identified three additional publications, which were included in the scope of this review (Fig. 1). A total of 41 publications were reviewed. All publications were recently published, after the year 2016 indicating the recent and timely evolution of this field (Fig. 2).

Technological innovations
We defined the category technological innovations for research with a focus on improving the image acquisition, image analysis or automation in the treatment process. 22 of the 41 publications were assigned to 'technological innovations' (Table 1), with a large variation in approaches. This ranged from tasks regarding the reduction of contrast agents in the image acquisition [7], the filtering of artefacts and spectral fitting in MRSI [8,9], the separation of brain from non-brain tissue [10], the generation of synthetic Computed Tomography (CT) images for radiotherapy treatment [11][12][13][14], the auto-segmentation tasks of brain tumors [15][16][17][18][19][20][21][22][23][24][25][26] and solutions to generate, or cope with, limited annotated data [26,27]. The patient populations was diverse, containing healthy subjects, not specified brain tumors, as well as glioma, glioblastoma, meningeomas and brain metastasis. Sample sizes ranged from 10 MRI scans [9] to 1107 patients with 2925 MRI scans [10]. The most frequently used MRI sequences were T1 weighted (T1-w) or contrast enhanced T1-w MRI (cT1-w), followed by a frequent addition of T2 weighted (T2-w) and fluid-attenuated inversion recovery (FLAIR). The use of Magnetic resonance spectroscopic imaging (MRSI) was only presented in two publications (see Table 1).

Image acquisition and pre-processing
During image acquisition, MRI contrast agents can be used to enhance the visibility of pathology on images. Gadolinium-based contrast is the most frequently used in clinical practice and has vital importance in neuro-oncological MRI (e.g. T1-w, T2-w and FLAIR sequences). There is, however convincing evidence that a deposition of gadolinium in the deep nuclei of the brain can occur, especially after repeated exposure to Gadolinium-based contrast. At the moment, the clinical or biological significance is still unknown, nevertheless the International Society of Magnetic Resonance in Medicine (ISMRM) urges caution in the use of gadolinium and a reduction of the frequency and amount of contrast agent is preferred [29]. A deep-learning framework was used to generate full-dose gadolinium images from low-dose, using only 10% of the gadolinium dose, images. The study shown that the DL method showed no significant differences with regard to overall image quality, clarity of the contrast enhancement or artifact suppression and therefore has the potential to reduce the gadolinium contrast agent while preserving the image quality, [7].
Specifically in magnetic resonance spectroscopic imaging (MRSI), the removal of spectral artefacts is an essential pre-processing step. Deep learning models have been developed to identify and filter these poor quality spectral data, that otherwise could lead to incorrect classification of voxel pathology. Gurbani et al. [8] trained a CNN to analyze the MRSI frequency domain spectra to detect artifacts, and compared the performance of the model to experts and achieved a high sensitivity and specificity with an AUC of 0.95. In addition, the same authors used a convolution neural network to process the MRSI data and perform rapid spectral fitting, were they were able to perform a sub-minute calculation of the relative metabolite concentration of the brain [9].
In the processing of images for neuroimaging studies, the identification of the brain tissue is an important pre-processing step. The accuracy can have impact on the quality of further image analysis like image registration, segmentation of tumor lesions, measurement of brain volume, cortical thickness and planning for interventions [10,30,31]. Challenges in brain segmentation are the labour and time intensiveness of manual segmentation and when segmentation is automated the diversity in MRI pulse sequences, MRI vendors and neurological pathologies, which can impact the automatic segmentation. Isensee et al. [10] trained and independently validated an artificial neural network (ANN), for brain identification on four different datasets including a large dataset from a prospective randomized neuro-oncology trial (EORTC-26101) and three independent public datasets [32][33][34]. The ANN algorithm outperformed six public brain identification methods, by the comparison of comparing DICE coefficient and Hausdorff distances and enabled a robust brain identification in the presence of pathology. The brain extraction algorithm was applicable to a broad range of MRI sequence types (T1-w, cT1-2, T2-w and FLAIR) and was not influenced by MRI acquisition parameters or hardware.

Synthetic CT generation
For radiotherapy treatment planning, CT scans are used to determine the electron or mass density of the tissues. This density is necessary to estimate the absorbed dose of the radiotherapy treatment. In neurooncology the use of MRI images provides significant additional value regarding soft tissue contrast in comparison to CT imaging. Therefore, in current clinical practice, most patients receive both MRI and CT imaging in the workup for radiotherapy treatment (Fig. 3). The radiotherapy domain is however moving towards the use of MRI only treatments, by generating a synthetic CT from MRI images. A challenge in this use case is the accurate separation of air and bone, which is essential to limit the dose calculation error. Kazemifar et al. [11] used a generative adversarial network to generate synthetic CT images from cT1-w MRI images and assessed the dosimetric accuracy. Overall, they observed no significant difference in dose parameters evaluated for the target volume and healthy structures. In addition, Neppl et al. [12] evaluated the use of a 2D and 3D U-shaped Convolutional Neural Network (CNN) to generate synthetic CT images from T1-w MRI and assessed the effect of photon and proton dose distribution. The dose evaluation for photons yielded a good pass rate for both 2D and 3D U-Nets, while the gamma passing rate (2%, 2 mm) for the proton plans were all above 89.3%. Liu et al. [13] utilized the deep convolutional encoder-decoder network to develop an automated approach to generate pseudo CT images from T1-w MR images and observed no significant difference in dose-distribution compared to the use of standard kvCT imaging. Last, Dinkla et al. [14] showed that dose calculations performed on synthetic CT images generated by a dilated CNN, by using an additional dilation parameter to the standard convolutional kernels, were accurate and can be used for MRI-only intracranial radiation therapy treatment planning.

Auto-segmentation
For characterization of individual brain structures, the segmentation of relevant structures (both tumor and functional regions of interest) within the brain is an important process for many applications including radiation treatment planning (Fig. 3). The outlining of regions of interest is a time-intensive procedure and it is known to be prone to interobserver variability. [35][36][37] The use of automated segmentation has therefore the potential to improve the delineation quality and treatment workflow.
As part of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) a yearly challenge has been set for Brain Tumor Segmentation Challenge (BRATS) from 2012 to 2020, which focusses on the evaluation state-of-the-art methods for the segmentation of brain tumors in multimodal magnetic resonance imaging (MRI) scans. The available dataset is modified for each yearly challenge. In our search 8 publications used data made available from this BRATS MICCAI challenge from several years and propose a diversity of DL methodsto improve tumor segmentation [15][16][17][18][19][20][21][22].
All studies using BRATS data report at least the DICE similarity coefficient for their proposed methods, which ranged for the whole tumor from 0.84 to 0.91, for the tumor core 0.72-0.86 and for the enhanced from 0.62 − 0.82 (Fig. 4). In detail, on the MICCAI 2015 dataset; Iqbal et   Additional studies on independent datasets are 1) The use of preoperative MRI scans (T1, T2, FLAIR and cT1) of glioblastoma patients to detect and segment tumor lesion based on a multilayer, multiscale convolutional neural network. Perkuhn et al. [23] showed in a dataset of 64 patients from 15 different institutes a high lesion detection rate and segmentation accuracy (DICE for whole tumor segmentation: 0.86), which was comparable to the interrater variability. 2) A multi-sequence MRI guided convolution neural network showed to accurately delineate post-operative gliomas on CT images for radiotherapy, to assist image  segmentation and reduce workload [24]. Challenges in the generation of deep learning applications for image analysis is the presence of a sufficient amount of annotated datasets. An approach in the development of annotated data for tumor segmentation is the use of eye-tracking technology. Segmentation masks were generated by eye-tracking technology and compared to hand annotation and the authors demonstrated that eye-tracking can be used to generate segmentation masks suitable for deep learning segmentation tasks [27]. In addition, strategies are developed to circumvent the issue of limited annotated data. For example by building image classifiers using features from pre-trained segmentation networks. By using these segmentation networks, the machine can learn first from the simpler shapes and structures before tackling the actual classification problem. Using this methodology a high classification performance can be obtained with limited training data [28].
To summarize; deep learning applications are in development for a broad range of technological or (pre-) processing steps of MRI. The additional value has been shown for the increase of MRI image quality (e.g. by reduction of contrast or artefacts), the generation of synthetic CT imaging from MRI allowing less imaging interventions for the individual patients and for the generation of automatic tissue and auto-contouring task with the potential to reduce workload within current processes.

Diagnosis
Twelve papers investigated the role of AI in the diagnosis of neurological neoplasms ( Table 2). The main objectives of the papers involved: pathological or molecular classification of the tumors (N = 7) [38][39][40][41][42][43][44] solely detection of tumor in a Computer Aided Diagnosis (CAD) fashion (N = 1) [45] and the combination of detection and segmentation of the lesions (N = 4) [46][47][48][49]. The number of patients used in the studies span from a minimum of 33 patients [47] to a maximum of 266 patients [49]. Sert et al. [48] used a publicly available dataset from the cancer imaging archive (TCIA; TCGA-GBM), which includes more than 500 samples; however the authors selected 100 positives (including at least a tumor) and 100 negative samples (healthy subjects) to train the Convolutional Neural network (CNN; ResNet architecture) .
All of the studies were retrospective. Half of the studies used data collected from a single institution and the other half collected data from at least two different institutions (minimum 2, maximum 37 centres). Four studies were focused on meningioma [38,42,43,46], two studies on glioblastoma [47,48]. Two studies used glioma patients [44,45]. Two studies used different solid brain tumors [39,41]. One study used brain metastases arising from other primary tumors [40].

Brain tumor classification
Accurate diagnosis of brain lesions is essential for selecting an effective treatment. The classification of brain tumor into subtypes is a challenging research problem. Deepak et al. [39] proposed an automatic classification system based on GoogleNet CNN architecture and used it to identify three pathological subtypes of brain tumors (e.g. glioma, meningioma and pituitary tumor) on cT1w imaging. A mean classification accuracy of 98% was achieved. Swati et al. [41] approached the same problem by using a pre-trained deep CNN model and a block-wise fine-tuning strategy based on transfer learning, and achieved an average accuracy of 95%. To differentiate meningeomas and spinal schwannomas, the most frequent tumors of the spinal cord, Maki et al. [43] used a convolutional neural network (CNN) and was able to make a discrimination with an accuracy of AUC = 0.88 and 0.87 for T2 weighted imaging and contrast enhanced T1w imaging respectively.

Meningioma
The detection and grading of meningioma is important to select the suitable treatment for the individual patient. Laukamp et al. [46] used a multiparametric deep-learning model on multiple MRI sequences including T1 (with or without contrast), T2 and FLAIR and was able to automatically detect meningiomas in 55 out of 56 cases. In addition, a deep CNN was able to discriminate between benign and atypical/ anaplastic meningiomas from the apparent diffusion coefficient (ADC) maps, with an accuracy of AUC: 0.94 [38].
Radiomics is an image analysis method that uses a series of qualitative and quantitative analyses of high-throughput image features to obtain predictive or prognostic information from medical images. Zhu et al. [42] used a deep learning radiomics model to pre-operatively  Glioma & glioblastoma In all neurological lesions the ability to classify lesions according to the WHO classification is essential, for which currently invasive methods like biopsy or resection are used [50,51]. Besides the risk of surgery related complication there is the risk of a sample error. For a public dataset of the Cancer Genome Atlas Glioblastoma Multiforme, Sert et al. [48] developed an approach to use super resolution (SR; converting low-resolution input images into high-resolution images), and maximum fuzzy entropy segmentation (MFES) in combination with a convolution neural network to increase the classification performance between benign/malignant lesions. Out of 200 samples (100 benign, 100 malignant), a total of 10 false positive/false negative cases were observed (AUC = 0.98). The automatic detection of lesions is an additional benefit of DL in the diagnostic phase of cancer care. Atici et al. [45] developed a CNN using 3580 images from 179 patients for the automatic detection of high-grade gliomas on T2w MRI images and report an acceptable performance with an accuracy between 0.85 and 0.94 and precision between 0.81 and 0.98.
In addition, deep learning based radiomics can be used in the diagnosis of low-grade gliomas to detect patients with the isocitrate dehydrogenase 1 (IDH1) mutation. Note that, the IDH1 mutation status accounts for a large proportion of the predictive value in low-grade glioma and the treatment regimen is currently defined according to IDH1 status. Li et al. [44] demonstrated that deep learning radiomics on FLAIR and cT1-w MRI imaging has potential to predict IDH1 mutation in low-grade gliomas (AUC 0.92).
Also, in the preparation of the treatment, accurate target definition is of utmost importance to increase the outcome from either radiotherapy or neurosurgery. Peeken et al. [47] used a fully connected neural network in combination with DTI images to define the infiltrative tumor areas of GBM, with the purpose to guide radiation treatment. This novel infiltrating tumor definition was in all patients related to the location of tumor recurrences and has potential to further individualize radiotherapy treatment.
To summarize, deep learning has the potential to contribute to the automatic classification of lesions with regard to disease type, grade or mutational status. tumorAll these developments can contribute to improved cancer care with regard to more efficient diagnostic workflow as well as a step towards more personalized treatment.

Follow-up
Seven publications were related to the use of DL and MRI for the follow-up of neuro-oncological patients. All publications used retrospective data of glioma or glioblastoma patients (Table 3).

Response assessment in Neuro-Oncology (RANO)
To assess the treatment response of neuro-oncology patients the Response Assessment in Neuro-Oncology (RANO) criteria are frequently used. These criteria are generally accepted to assess response in clinical trials and are increasingly used in clinical practise. The RANO criteria divides response in four types (complete response, partial response, stable disease or progression) based on MRI and clinical features. Chang et al. [52] developed an automatic pipeline for brain extraction, tumor segmentation and RANO measurements and applied it in two patient cohorts; low-or high-grade gliomas (843 patients with 843 MRI scans) and newly diagnosed glioblastomas (54 patients with 713 MRI scans). To develop the deep learning algorithm, they utilized the 3D U-Net architecture, and automatically segmented the FLAIR hyper intensity, contrast-enhancing tumor, tumor volumes and the product of maximum diameters, according to RANO criteria. The Automatic RANO measurement was reproducible when evaluating the change in tumor burden during treatment, with an intraclass correlation coefficient (ICC) between automatic and manual delta RANO measurements of 0.85 (P < 0.001).
A disadvantage of the current RANO criteria is that it relies on 2D measurements. The use of 3D measurements for assessing tumor response could provide a more reliable result, but due to the significant workload of 3D manual assessment, it has practical limitations. For this reason, Kickingereder et al. [5] developed an infrastructure to enable full-automated analysis of MRI and investigated its performance for tumor response assessment. The assessment of tumor response based on their neural network outperformed the RANO assessment as a predictor for overall survival in an EORTC dataset, in addition the automatic assessment of tumor response showed a higher agreement to radiologist assessment than the use of RANO criteria. Based on both studies artificial intelligence has shown additional value in the automation of the RANO assessment as well as improving it.

Deep features
In the assessment of overall survival deep radiomics features have the potential to provide additional value as well. Transfer learning can be used within the current radiomics models for the extraction of a large number of deep features from the hidden layers of CNN. Deep features contain more abstract information of the MRI images and potentially  [53]. The proposed radiomics signature showed a higher performance (C-index 0.710) compared to general risk factors (age and KPS) and the combination of deep-learning based radiomics and general risk factors improved the predictive performance to a C-index of 0.739 [53]. In addition, the combination of standard radiomics features in combination with deep features was successful in an initial validation in cT1-w MRI images [54].

Advanced MRI
Most studies use standard cT1-w, T1-w, T2-w or FLAIR MRI acquisitions, however the use of additional MRI sequences, like DWI, DTI or fMRI in the follow-up were described in three articles [55][56][57].
Bacchi et al. [55] aimed to distinguish high grade glioma progression from treatment related changes like pseudoprogression or radionecrosis. For this purpose, the authors performed classification experiments using a CNN on DWI, ADC, FAIR and c-T1-w images. The DWI sequence had the best performance (AUC 0.63, Accuracy 0.73) and these DWI images were used in combination with other sequences. The combination of DWI with FLAIR sequences showed the highest performance (AUC: 0.80, Accuracy: 0.82), which shows that DL may be useful in distinguishing progression from treatment induced normal brain tissue changes. To predict overall survival in patients with high-grade glioma Nie et al. proposed a CNN architecture on multi-model pre-operative MRI images (T1w, fMRI and DTI) to train a survival time prediction model. This method was used to extract features from the image modalities in a supervised manner and train a Support Vector Machine to predict overall survival time. The experimental results showed that both fMRI and DTI played a more significant role compared to conventional T1 MRI, in building a successful prediction model [56]. In addition, in a subsequent study the authors show in 68 high-grade glioma patients, that a combination of the features extracted by their multi-modality, multi-channel deep survival prediction framework in combination with demographic and tumor-related features have an accuracy of 91% to predict overall survival [57]. In brief, deep learning applications have shown their potential in the automation of response assessment criteria (RANO), improve the response assessment by including 3D information as well as improve the prediction of overall survival by the inclusion of deep features or advanced MRI sequences and to distinguish between progression and treatment related changes on MRI. None of these methods though were extensively implemented or validated in current clinical practice but do show promising avenues for further investigation.

Discussion & Conclusion
The use of DL methods in the analysis of MRI data in neuro-oncology is rising. This study provides an overview of the current use of DL in the field of neuro-oncological MRI. Forty-one publications were reviewed and covered a broad range of applications, from technological innovations to improving diagnosis and follow-up.
We observed that the majority of publications were in the category of technological innovations. This is evident since these technological developments precede the clinical applications in diagnosis and follow-up. The technological innovations have a variety of impact on the clinical applicability of MRI technology or on its efficiency, e.g. by reduction of workload. In addition, the availability of open data to develop DL technology for brain tumor image segmentation, directly results in a significant number of publications. These Brain Tumor Segmentation challenges (BRATS) are repeated on a yearly basis. The challenge of 2020 had on top of the tumor segmentation challenge, also the focus on the prediction of patient overall survival and had initially a task planned to differentiate between pseudoprogression and true tumor recurrence. Based on these (proposed) challenges we can only expect the field to move rapidly in development of additional DL applications to support patient follow-up as well.
Challenges in the use of deep learning are amongst others, the generalisability of the models to different institutes and MRI scanners, as well as the accessibility to a large amount of annotated data to develop, train and externally validate the model. By using data from several institutes, a DL model is exposed to a larger range of data variations, which will generate a more robust and broader applicable model. There are, however, several barriers when trying to share clinical imaging data, considering technical, ethical, political and administrative issues [58]. Therefore, the ability to train a deep learning model without sharing the data by using distributed or federated learning is a promising approach. Using federated learning, models can be developed on data of different institutes and therefore a larger and more diverse dataset. Czeitzler et al. [59] recently showed the ability to train deep neural network model for organ segmentation in a distributed manner with similar performance to a centralized approach. Distributed learning is an active field of research as well as the use of deep learning on imaging data. The combination of both research fields could be a promising next step. Furthermore, clinical introduction of these AI methods requires a careful consideration in training, commissioning and acceptance of such models, recently published guidelines could be followed for this [60].
In our review, all studies related to 'follow-up' were focused on its applicability to evaluate tumor response or treatment outcome (e.g. overall survival). This can be related to the fact that most studies used data of patients with high grade glioma or glioblastoma with a relative poor survival. Nevertheless, for patients with a brain lesion with a relative long life expectancy, like meningioma patients, the prediction of treatment induced side effects could have additional value as well. We expect additional value from cross-disciplinary research combining the knowledge from general neuro-imaging, neuropsychology and specific research fields in dementia, Parkinson's disease or epilepsy. For example, after high dose radiotherapy for primary or metastatic brain tumors 50-90% of greater than 6 months' survivors develop irreversible disabling cognitive decline leading to premature loss of independence, reduced Quality of Life (QOL) as well as significant economic burden both at the individual as societal level [61]. Therefore, evaluation methods to assess and predict side effects after treatment would be beneficial to allow the development of optimized treatment strategies.
To allow the evaluation of treatment induced side effects, and to target oncological treatment, one should take into account the healthy brain structures and their susceptibility for treatment induced damage. The auto segmentation of structures at risk can have additional benefits on top of automatic tumor segmentation. For the neuro-oncological domain there is a consensus-based atlas available for CT-and MRI based contouring from the European Particle Therapy Network [37]. Auto segmentation of these brain structures could provide advantages by improving quality, due to the reduction of inter observer variation and improve on efficiency by reduction of delineation time.
The majority of publications made use of the standard MRI sequences available like T1w, cT1w, T2 or FLAIR imaging. The most straightforward reason for this is the availability and quantity of these imaging data. Nevertheless, in a few studies used advanced MRI imaging techniques (e.g. fMRI, DTI, DWI) and presented favourable results in comparison to standard imaging techniques [55,57].Furthermore, perfusion MRI imaging is another area that requires some elaborate postprocessing and the some initial investigations of the use of deep learning for perfusion MRI in neuro-oncology are ongoing [62][63][64].
The publications in this review all originate from the past four years, which shows that the development of deep learning technology is evolving rapidly for the application within neuro-oncological MRI. Nevertheless, the clinical use of these models is still limited. Shortlife et al. [65] presents six challenges in the implementation of AI in clinical support systems: black boxes are unacceptable, time is a scarce resource, intuitive and simple, relevance and insight are essential, inform and assist not replace clinician and the scientific foundation must be strong. Where the first one (black boxes are unacceptable) could be the biggest challenges in the clinical application of DL in clinical practice an emphasis should be given to develop explainable AI. Also, ethical dilemmas, like balancing the advantages and risks of using AI technology, as well as the role of AI in the medical education (e.g. how do we prepare future clinicians for the use of AI) and potential legal conflicts while using AI (who is responsible when using a black-box AI?) can slow down implementation of these novel technologies [66].
To conclude, Deep learning in MRI for neuro-oncology is a novel field of research, it has shown to have potential in a broad range of applications. Nevertheless, challenges remain the accessibility of large representative imaging datasets, the applicability of the models across institutes and MRI vendors and the potential barriers to implement these AI technologies in clinical practise.