If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, CH-1211 Geneva 4, SwitzerlandGeneva University Neurocenter, Geneva University, CH-1205 Geneva, SwitzerlandDepartment of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, 9700 RB Groningen, NetherlandsDepartment of Nuclear Medicine, University of Southern Denmark, 500 Odense, Denmark
Artificial intelligence and deep learning-based solutions emerged as promising approaches pertinent to PET and SPECT imaging.
•
Numerous applications in the field are taking advantage of these developments.
•
Successful commercial implementation of deep learning-based solutions is projected.
•
Clinical validation and adoption of these tools face many challenges.
Abstract
This review sets out to discuss the foremost applications of artificial intelligence (AI), particularly deep learning (DL) algorithms, in single-photon emission computed tomography (SPECT) and positron emission tomography (PET) imaging. To this end, the underlying limitations/challenges of these imaging modalities are briefly discussed followed by a description of AI-based solutions proposed to address these challenges. This review will focus on mainstream generic fields, including instrumentation, image acquisition/formation, image reconstruction and low-dose/fast scanning, quantitative imaging, image interpretation (computer-aided detection/diagnosis/prognosis), as well as internal radiation dosimetry. A brief description of deep learning algorithms and the fundamental architectures used for these applications is also provided. Finally, the challenges, opportunities, and barriers to full-scale validation and adoption of AI-based solutions for improvement of image quality and quantitative accuracy of PET and SPECT images in the clinic are discussed.
Artificial intelligence (AI) approaches, particularly deep learning (DL) techniques, have received tremendous attention during the last decade owing to their remarkable success in offering novel solutions to solve complex problems. Novel AI/DL-based solutions have created opportunities in clinical and research settings to automate a number of tasks deemed to depend on human cognition, and hence require his intervention to facilitate the decision-making process [
]. State-of-the-art AI/DL algorithms have exhibited exceptional learning capability from high dimensional and/or highly complex data, accomplishing daunting challenging tasks in image and data analysis/processing in general and multimodality medical imaging in particular.
In the context of medical imaging, challenging tasks, such as image segmentation/classification, data correction (such as noise or artifact reduction), image interpretation (prognosis, diagnosis, and monitoring of response to treatment), cross-modality image translation or synthesis, and replacing computationally demanding algorithms (such as Monte Carlo calculations) have been broadly revisited and evolved ever since the adoption of deep learning approaches [
]. AI-based solutions have been proposed to address the fundamental limitations/challenges faced by image acquisition and analysis procedures on modern molecular imaging technologies. Considering the superior performance of deep learning approaches compared to conventional techniques, a paradigm shift is expected to occur provided that task-specific pragmatic developments of these algorithms continue to evolve in the right direction.
Single-photon emission computed tomography (SPECT) and Positron emission tomography (PET) imaging provide the in vivo radiotracer activity distribution maps, representative of biochemical processes in humans and animal species. The introduction of hybrid imaging combining functional and anatomical imaging modalities in the form of combined PET/CT and PET/MRI systems has remarkably thrived the widespread adoption and proliferation of these modalities in clinical practice. In this light, AI-based algorithms/solutions are developed to overcome the major shortcomings or to enhance the current functionality of these modalities.
The applications of AI-based algorithms in PET and SPECT imaging ranges from low-level electronic signal formation/processing to high-level internal dosimetry and diagnostic/prognostic modeling. For developments in instrumentation, deep learning approaches have been mostly employed to improve the timing resolution and localization accuracy of the incident photons aiming at enhancing the overall spatial and time-of-flight (TOF) resolutions in PET. Image reconstruction algorithms are being revisited through the introduction of deep learning algorithms wherein the whole image reconstruction process or certain critical components (analytical models) are being replaced by machine learning models. A large body of literature is dedicated to quantitative SPECT and PET imaging aiming at reducing the impact of noise, artifact, and motion, or to correct for physical degrading factors, including attenuation, Compton scattering, and partial volume effects. The lack of straightforward techniques for generation of the attenuation map on organ-specific standalone PET scanners or hybrid PET/MRI systems inspired active scientists in the field to devise suitable strategies to enhance the quantitative potential of molecular imaging. High-level image processing tasks, such as segmentation, data interpretation, image-based diagnostic and prognostic models as well as internal dosimetry based on SPECT or PET imaging have substantially evolved owing to the formidable power and versatility of deep learning algorithms.
AI/DL-based solutions have been proposed to undertake certain tasks belonging to the long chain of processes involved in image formation, analysis, and extraction of quantitative features for the development of disease-specific diagnosis/prognosis models from SPECT and PET imaging. In this review, the applications of AI/DL in these imaging modalities are summarized in six key sections focusing on the major challenges/opportunities and seminal contributions in the field. A concise overview of machine learning methods, in particular deep learning approaches, is presented in section 2. The following section describes AI-based techniques employed in PET instrumentation, image acquisition and formation, image reconstruction and low-dose scanning, quantitative imaging (attenuation and scatter corrections), image analysis and computer-aided detection/diagnosis/prognosis, as well as internal radiation dosimetry. The last section provides in perspective the major challenges and opportunities for AI/DL-based solutions in PET and SPECT imaging.
Principles of machine learning and deep learning
Machine learning algorithms are considered as a subset of non-symbolic artificial intelligence, which tends to automatically recognize a pattern and create/extract a desirable representation from raw data [
]. In machine learning algorithms, the system attempts to learn certain patterns from the extracted features. Likewise, in deep learning algorithms, a subtype of machine learning techniques, feature extraction, feature selection, and ultimate tasks of classification or regression are carried out automatically in one step [
]. Some applications of machine learning algorithms, such as classification, segmentation, and image-to-image translation, have attracted more attention [
A number of deep learning architectures became popular in the field of medical image analysis, including convolutional endcoders-decoders (CED) networks consisting of encoder and decoder parts designed to convert input images to feature vectors and feature vectors to target images, respectively [
]. In addition, GANs consist of two major components: a generator, mostly a CED network, and a discriminator, a classifier to differentiate the ground truth from the synthetic images/data [
]. Different architectures based on these models were developed and applied on medical images for different tasks, including image segmentation and image to image translation [
] is one the most popular architectures built upon the CED structure via adding some skip connections for context capturing and for creating a symmetric expanding path, which enables more efficient feature selection. Upgrading networks with different modules, such as attention blocks/components [
Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:180403999. 2018.
] to prevent gradient vanishing, are intended to improve the overall performance of the networks. Conventional GAN architectures have been upgraded in different ways, leading to conditional GAN (cGAN) [
Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition2017. p. 1125-34.
Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision2017. p. 2223-32.
Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision2017. p. 2223-32.
] is an unsupervised model for image-to-image transformation, which does not require paired (labeled) datasets. In the Cycle-GAN model, two generator and discriminator components are jointly involved in the training process, wherein images from two different domains are used as input and output within a cycle consistency scheme. In the cycle consistency scheme, the output of the generator component is used as input and vise versa with the calculated loss between the input and output acting as regularization of the generator model [
Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision2017. p. 2223-32.
]. AI-based approaches, especially deep learning algorithms, do not require handcraft features extraction, specific data preprocessing, or user intervention within the learning and inferring processes [
]. The major applications of deep learning approaches in SPECT and PET imaging are summarized in Fig. 1. Deep learning methods face many challenges, including the fact that they are data hungry, require high computation burden for the training process, and their black box nature (which hampers systematic analysis of their operation/performance) [
]. To reach peak performance, these algorithms require a large number of clean and cured datasets for the training process. However, data collection remains the main challenge owing to patients’ privacy and complexity of ethical issues. Moreover, task-specific deep learning algorithms (i.e. for a particular organ/body region or radiotracer) are able to exhibit superior performance compared to more general models which are commonly more sensitive to variability in image acquisition protocols and reconstruction settings. Another challenge faced by the application of deep learning algorithms in medical imaging is the high computational burden owing to the large size of clinical data in terms of number of subjects and individual images (large 3-dimensional images or sinograms) which might cause memory or data management issues.
Fig. 1Main applications of deep learning-based algorithms in PET and SPECT imaging.
Applications of deep learning in SPECT and PET imaging
Instrumentation and image acquisition/formation
Detector modules play a key role in the overall performance achieved by PET scanners. An ideal PET detector should have a good energy and timing resolution and capable of accurate event positioning. Energy resolution is a metric that determines how accurately a detector can identify the energy of incoming photons and as a result, distinguish scatter and random photons from true coincidences. These parameters affect the scanner’s sensitivity, spatial resolution, and signal-to-noise ratio (true coincidence versus scatters or randoms). Despite significant progress in PET instrumentation, there are a number of challenges that still need to be addressed and where machine learning approaches can offer alternative solutions to complex and multi-parametric problems.
Accurate localization of the interaction position inside the crystals improves the overall spatial resolution of PET scanners. Since optical photons distribution is stochastic, particularly near the edges of the crystal, and owing to multiple Compton scattering and reflection, accurate positioning of the interaction within the crystal is challenging. In comparison with other positioning algorithms, such as Anger logic and correlated signal enhancement, which rely on determination of the centre of gravity, machine learning algorithms led to a better position estimation particularly at the crystal edges [
]. In this regard, Peng et al. trained a CNN classifier that was fed with signals from each Silicon photomultiplier’s channel to the coordinates of the scintillation point for a quasi-monolithic crystal [
]. Another study applied a multi-layer perceptron to predict the 3D coordinates of the interaction position inside a monolithic crystal and compared the performance of this positioning algorithm with anger logic for a preclinical PET scanner based on NEMA NU4 2008 standards [
]. Fig. 2 depicts the adopted deep learning-based event-positioning scheme in monolithic detectors. To address the challenge of determining the depth of interaction, a gradient tree boosting supervised machine learning algorithm was used to extract the scintillation position, resulting in a spatial resolution of 1.4 mm full width half maximum (FWHM) for a 12 mm thick monolithic block [
]. Recently, Cherenkov-based detectors attracted much attention owing to their superb performance in terms of time and spatial resolution. Hashimoto et al. studied the performance of a deep learning model for 3D positioning in this type of detectors through a Monte Carlo simulation study [
]. They demonstrated that in comparison with conventional positioning methods, such as the centre of gravity determination and principal component analysis, the deep learning model led to significantly improved spatial resolution.
Fig. 2Deep learning-based event positioning in monolithic detectors.
Time resolution is another crucial factor in PET instrumentation which determines the achievable performance using TOF imaging as well as the efficiency of randoms and scatter rejection. This factor depends on the physical characteristics of the scintillator, photodetector quantum efficiency, and electronic circuits that convert the scintillation light to electrical signals. Considering the physics of photon interactions within a crystal, only a portion of produced scintillation photons reach the photodetector and contribute to positioning and timing. The consequence of this is noticeable statistical uncertainty and noise-induced bias. Straightforward approaches, such as feeding a CNN model with detector signals to estimate TOF information produced promising results. In a recent study, a training dataset (reference) obtained by scanning a 68Ga point source shifted repeatedly with steps of 5 mm across the field-of-view of the PET scanner was used to train a deep learning algorithm [
]. The authors reported a TOF resolution of about 185 ps, exhibiting significant improvement with respect to conventional methods with a resolution of 210 to 527 ps. Gladen et al. developed a machine learning method, referred to as self-organized map (SOM) algorithm, for estimating the arrival time of annihilation photons in a high purity germanium detector (HPGe). SOM was able to cluster the TOF bins based on the signal shape and its raising edge [
Gladen R, Chirayath V, Fairchild A, Manry M, Koymen A, Weiss A. Efficient Machine Learning Approach for Optimizing the Timing Resolution of a High Purity Germanium Detector. arXiv preprint arXiv:200400008. 2020.
Recent studies substantiated the applicability of deep learning techniques to reliably estimate the interaction position, energy, and arrival time of incident photons within the crystal with improved accuracy and robustness to noise. One of the major difficulties in developing such models is the creation of labelled data (used as reference), which require extensive experimental measurements. For example, preparing a training dataset for position estimation requires a precise and reproducible setup of a single pencil beam and several measurements at any possible spot within the field-of-view. A number of recent studies came up with novel ideas to perform these tasks for monolithic crystal through using uniform or fan-beam sources or applying clustering to the training dataset [
]. Likewise, for TOF training dataset, hundreds of point source positionings and data acquisitions are required to create a realistic range of TOF variations. In this regard, artificial ground-truth data creation was proposed through switching the PET detector waveforms forward and backward in the time domain [
Sophisticated machine learning-based algorithms for event positioning, timing, and/or calibration are envisioned on next generation SPECT and PET systems on the front-end electronics using dedicated application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs) [
]. Furthermore, developing a single model for extracting time, position, and energy simultaneously from photodetector outputs would be an interesting approach that can potentially improve the overall performance of the nuclear imaging systems.
Image reconstruction and low-dose/fast image acquisition
Deep learning algorithms have recognized capabilities in solving complex inverse problems, such as image reconstruction from projections. The process of image reconstruction for CT, PET, and SPECT using deep learning techniques entails roughly the same procedure. Overall, four strategies were adopted for image reconstruction using deep learning algorithms. The first approach consists of image-to-image translation in the image space, wherein a model is trained to convert reconstructed images to another representation to improve image quality through, for instance, noise removal, supper resolution modelling, motion correction, etc. [
]. The second approach implements the training of the deep learning model in the projection space prior to image reconstruction to avoid the sensitivity and dependence on reconstruction algorithms. In the third approach, a model learns to develop non-linear direct mapping between information in the sinogram and image domains [
]. The fourth approach, referred to as hybrid domain learning, relies simultaneously on analytical reconstruction and machine learning approaches to reach an optimal solution for the image reconstruction problem [
]. Likewise, FastPET, is a machine learning-based approach for direct PET image reconstruction using a simple memory-efficient architecture implemented to operate for any tracer and level of injected activity [
Decreasing the injected activities is often desired owing to potential hazards of ionizing radiation for pediatric patients or subjects undergoing multiple serial PET or SPECT scans over time for monitoring of disease progression or in longitudinal studies. Moreover, decreasing the acquisition/scanning time increases scanners throughput and enhances patients’ comfort, particularly elderly patients and those suffering from neurodegenerative diseases where the risk of involuntary motion during scanning is more common.
Reducing the injected activity amplifies Poisson noise, thus impacting image quality, lesion detectability, and quantitative accuracy of PET images. Devising optimized low-dose scanning protocols that preserve the critical information in the images is desirable. Although there is a fundamental difference between fast and low-dose scanning, both approaches have been interchangeably used in the literature. While both strategies produce noisy images, the content and information collected by these scanning modes are completely different. In a fast scan, the acquired data reflect the radiotracer kinetics in a short time course. For instance, if the scan starts right after injection, much information would be missing owing to insufficient and/or slow uptake in some organs. Fast acquisition protocols are also less sensitive to motion artifacts, though the patient’s effective dose is similar to standard protocols. Conversely, low-dose scanning is performed with standard acquisition time, with a much lower injected activity, which obviously decreases the effective dose.
There might be a need to redesign/optimize reconstruction algorithms for low-dose scanning to reach an optimal trade-off between noise level and signal convergence. In low-dose/fast imaging, much critical information would be buried under the increased noise level wherein an efficient denoising algorithm would be able to recover genuine signals [
To address the above-mentioned challenges, a number of denoising techniques to generate full-dose PET images from corresponding noisy/low-dose counterparts have been proposed. Conventional techniques include post-reconstruction processing/filtering algorithms [
]. Although these approaches attempted to minimize noise and quantitative bias, they still suffer from loss of spatial resolution and over-smoothing. By introducing image super-resolution techniques, such as sparse representation [
], effective noise reduction and signal recovery in low-dose images is expected with minimum artifacts or information loss. The widespread availability of hybrid imaging enabled to incorporate anatomical information in the reconstruction of low-dose PET images [
]. In most previous works, low-dose images were considered as the model’s input whereas full-dose images were considered as the target to perform an end-to-end mapping between low-dose and full-dose images [
]. Such models with a single input channel (only low-dose images) suffer from the lack of sufficient information (for instance anatomical structures) to distinguish noise from genuine biological signals. Therefore, adding anatomical priors into the training procedure would make the model more accurate and robust. For resolution recovery, high-resolution anatomical information obtained from MR imaging was employed along with spatially-variant bluring kernels to avoid information loss during image reconstruction [
]. Some groups devised strategies for deep learning-guided denoising models for synthesizing full-dose sinograms from their corresponding low-dose sinograms [
An elegant study by Xu et al. proposed a U-Net model with concatenation connection and residual learning for full-dose reconstruction from a single 200th low-dose image [
]. Xiang et al. presented a novel deep auto-context CNN model for synthesizing full-dose images from low-dose images complementing T1-weighted MR images. In comparison with state-of-the-art methods, their proposed model was able to generate comparable image quality while being 500 faster [
]. Another study employed a multi-input U-Net to predict 2D transaxial slices of 18F-Florbetaben full-dose PET images from corresponding low-dose images, taking advantage of available T1, T2, and Diffusion-weighted MR sequences [
]. Liu et al. employed three modified U-Net architectures to enhance the noise characteristics of PET images through concurrent MR images without the need for full-dose PET images with a higher signal-to-noise ratio [
] proposed a 3D U-Net model for denoising of PET images acquired with two different radiotracers (68Ga-PRGD2 and 18F-FDG) where the model was trained with MR/CT images and prior high-quality images as input and noisy images as training labels. Using original noisy images instead of high-quality full-dose images makes the training of the model more convenient. Unsupervised networks are always desirable in medical image analysis due to the fact that data collection with accurate labels is challenging and/or time-cosuming. A foremost drawback of the above-mentioned models is that model training was performed in 2D rather than 2.5D or 3D.
The 3D U-Net architecture was able to reduce the noise and PET quantification bias while enhancing image quality of brain and chest 18F-FDG PET images [
]. To compensate for the limited training dataset, they pre-trained the model using simulation studies in the first stage and then fine-tuned the last layers of the network with realistic data. Kaplan et al. [
] trained a residual CNN separately for various body regions, including brain, chest, abdomen, and pelvis to generate full-dose images from 1/10th of the standard injected tracer activity. Training and testing of the model were performed on only two separate whole-body 18F-FDG PET datasets.
GAN networks are widely used for image-to-image transformation tasks, especially image denoising. Conditional GANs (cGAN) and cycle GANs (Cycle-GAN) are two well-established architectures commonly used for style and domain transformation. In cGAN, unlike regular GAN, the generator and discriminator’s output is regularized by an extra-label. For instance, Wang et al. estimated the generator error and used it beside the discriminator loss to train the generator of a 3D cGAN more efficiently for denoising low-dose brain PET images [
Cycle-GAN models do not necessarily require paired images as the model can learn in an unsupervised way to map input images from source to target domains. Because of the iterative feature extraction process and the presence of the inverse path in this architecture, the underlying characteristics of input/output data can be extracted from unrelated images to be used in the image translation process. Zhou et al. proposed a 2D Cycle-GAN for generating full-dose with around 120 million true coincidences (for each bed position) from a low-dose image with only one million true coincidences [
]. They used a generator with residual blocks to learn the difference between low-dose and full-dose images to effectively reduce the noise. The same group presented a similar model incorporating CT images to guide low-dose to full-dose transformation using a relatively small dataset [
]. Their results revealed that the incorporation of CT images can improve the visibility of organ boundaries and decrease bias especially in regions located near bones.
More recent studies implemented the training process using deep learning models in the projection space instead of image space, demonstrating that training a model in the sinogram space could lead to more efficient learning compared to training in the image space. Sanaat et al. trained a U-Net model with a dataset consisting of 120 brain 18F-FDG PET full-dose studies in the sinogram space [
]. The proposed model predicted full-dose from low-dose sinograms and demonstrated the superior performance of deep learning-based denoising in the sinogram space versus denoising in the image space (Fig. 3). Furthermore, another study proposed a prior knowledge-driven deep learning model for PET sinogram denoising [
Lu S, Tan J, Gao Y, Shi Y, Liang Z. Prior knowledge driven machine learning approach for PET sinogram data denoising. Medical Imaging 2020: Physics of Medical Imaging: International Society for Optics and Photonics; 2020. p. 113124A.
] combined Monte Carlo simulations and deep learning algorithms to predict high-quality sinograms from low-quality sinograms produced by two PET scanners equipped with small and large crystals, respectively. In whole-body PET imaging, Sanaat et al. compared the performance of two state-of-the-art deep learning approaches, namely Cycle-GAN and ResNet, to estimate standard whole-body 18F-FDG PET images from a fast acquisition protocol with 1/8th of the standard scan time [
Sanaat A, Shiri I, Arabi H, Mainta I, Nkoulou R, and Zaidi H. Deep learning-assisted ultra-fast/low-dose whole-body PET/CT imaging. Eur J Nucl Med Mol Imaging; 2021 in press.
Though most of the above-described approaches could be applied to SPECT imaging, few studies dedicatedly addressed low-dose and/or fast SPECT imaging studies. Recently, a supervised deep learning network was employed to reduce the noise in myocardial perfusion SPECT images obtained from 1/2th, 1/4th, 1/8th, and 1/16th of the standard-dose protocol across 1052 subjects [
Shiri I, AmirMozafari Sabet K, Arabi H, Pourkeshavarz M, Teimourian B, Ay MR, et al. Standard SPECT myocardial perfusion estimation from half-time acquisitions using deep convolutional residual neural networks. J Nucl Cardiol; 2021 in press.
Song et al. implemented a 3D residual convolutional neural network to estimate standard-dose SPECT-MPI data from four-time reduced dose counterparts using 119 clinical studies [
Song C, Yang Y, Wernick MN, Pretorius PH, King MA. Low-Dose Cardiac-Gated SPECT Studies Using a Residual Convolutional Neural Network. In IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) 2019. p. 653–6.
]. Compared to spatio-temporal non-local mean (NLM) post-reconstruction filtering, the deep learning algorithm achieved significant improvement in spatial resolution of the left ventricular wall. As an alternative to post-reconstruction filtering, Liu et al. employed a coupled-Unet to suppress the noise in conventional SPECT-MPI acquisitions [
]. A noise-to-noise denoising approach was adopted and compared with traditional post-filtering methods owing to the lack of ground truth/reference in clinical studies. The training dataset was generated using a bootstrap procedure where multiple noise realizations were made from clinical list-mode acquisitions and employed for the training of the coupled-Unet model. The clinical study performed on 895 patients revealed improved perfusion defect detection in deep learning-based filtered images reflected by enhancement of the SNR by 23% compared to conventional 3D Gaussian and NLM filtering.
Since Monte Carlo-based iterative image reconstruction algorithms exhibit superior performance over analytic approaches (owing to accurate modeling of photon interactions and collimator effects) in SPECT imaging, Dietze et al. proposed a CNN model to upgrade conventional filtered back-projection generated SPECT images to the quality of Monte Carlo-based reconstructions within a few seconds [
]. The clinical assessment of 128 99mTc-macroaggregated albumin SPECT examinations revealed only a 2% difference between the deep learning- and Monte Carlo-based image reconstruction frameworks, while the deep learning solution exhibited over 200 times faster image reconstruction (5 s vs. 19 min). To reconstruct SPECT images directly from the projection data, Shao et al. trained a deep learning model using a simulation dataset derived from computerized phantoms [
]. The performance of the deep learning-based reconstruction was evaluated on the Zubal human brain phantom and clinical studies wherein higher spatial resolution and quantitative accuracy were observed compared to conventional iterative reconstruction methods. Likewise, Chrysostomou et al. employed projections acquired from digital phantoms to train a deep learning model to enables directly mapping of SPECT images from the projection data [
]. The model exhibited superior performance over filtered backprojection and maximum likelihood expectation maximization reconstruction algorithms.
To reduce the acquisition time of paediatric 99mTc-dimercaptosuccinic acid SPECT acquisitions, Lin et al. trained a deep learning model to estimate full-time (in the image domain) from half-time acquisitions using 112 paediatric renal SPECT scans [
]. Synthetic SPECT images led to 91.7%, 83.3%, and 100% accuracy, sensitivity, and specificity, respectively, in the detection of affected kidneys. Similar efforts were undertaken to decrease 177Lu SPECT acquisition time, where intermediate projections were synthesized using a deep convolutional Unet model to compensate for image degradation due to the reduced angular samples. The deep learning model was trained using 352 clinical SPECT studies, wherein every fourth projection out of a total of 120 projections was fed into the deep learning model to synthesize the other three projections (four-fold reduced acquisition time). Deep learning-based synthesized intermediate projections (from sparse-projection scan) improved remarkably the quality of the final reconstructed images compared to dedicated iterative image reconstruction algorithms for sparsely acquired projections [
Generalizability and robustness of deep learning models are two significant factors that show how much a model is trustable and the results robust and reproducible for normal/abnormal unseen datasets. These two factors are largely linked to the diversity and number of training samples. It is very common to exclude abnormal cases prior to training or evaluation of a model to create a homogeneous training/test sample. Although this results in better results, it will reduce robustness to a realistic dataset with a broad range of abnormalities. It is strongly recommended to use both healthy/normal and unhealthy/abnormal subjects with a realistic distribution of the samples. Moreover, to avoid overfitting and guarantee effective training of the model, application of relevant data augmentation techniques is also recommended.
Using recurrent neural networks to decrease the scanning time and/or injected activity, especially in low-count dynamic PET imaging studies would be an interesting field of research. In addition, applying self-attention concepts to deep learning models would effectively enhance the performance of these models through indirect down-weighting/elimination of irrelevant regions and information in low-dose images while emphasizing the prominent/meaningful properties/information during the training process. Using realistic simulations to produce gold standard data sets beside clinical images would help deep learning models to learn noise distributions from a larger representative sample.
Quantitative imaging
A significant number of emitted photons undergo attenuation and Compton scatter interactions before they reach PET and SPECT detectors. Scatter and attenuation lead to over- and under-estimation of activity concentration, consequently resulting in large quantification errors [
]. To link the detected photons to the radiotracer activity concentration, attenuation and scatter correction (ASC) should be performed in SPECT and PET imaging [
]. In hybrid PET/CT and SPECT/CT images, the attenuation maps reflecting the distribution of linear attenuation coefficients are readily provided by CT images.
The main challenge for ASC arises in SPECT-only, PET-only, as well as PET/MR and SPECT/MR imaging since MR images are not directly correlated to electron density, and as such, do not provide information about attenuation coefficients of biological tissues [
Magnetic resonance-based attenuation correction and scatter correction in neurological positron emission tomography/magnetic resonance imaging—current status with emerging applications.
Magnetic resonance-based attenuation correction and scatter correction in neurological positron emission tomography/magnetic resonance imaging—current status with emerging applications.
]. However, application of this methodology across different radiotracers warrants further investigation.
In addition to emission-based algorithms, MR image-based algorithms, including segmentation and atlas-based algorithms have been developed to estimate attenuation coefficients from concurrent MR images [
Magnetic resonance-based attenuation correction and scatter correction in neurological positron emission tomography/magnetic resonance imaging—current status with emerging applications.
]. In segmentation-based algorithms, different MR sequences, including T1, T2, ultra-short echo (UTE), and zero-time echo (ZTE) have been used to delineate major tissue classes followed by assignment of pre-defined linear attenuation coefficients to each tissue class. In Atlas-based algorithms [
], pairs of co-registered MR and CT images (considered as template or atlas) are aligned to the target MR image to generate a continuous attenuation map. The main disadvantage of atlas-based algorithms is the high dependence on the atlas dataset and sub-optimal performance for subjects presenting with anatomical abnormalities [
] proposed converting non-attenuation corrected (NAC) brain PET images to synthetic CT (sCT) images. A GAN model was trained using 100 patients (in 2D mode) and tested on 28 patients achieving a relative error of less than 1% within 21 brain regions. Dong et al. [
] proposed a novel approach to generate sCT images in 99mTc-tetrofosmin myocardial perfusion SPECT imaging taking advantage of two images produced using different energy windows providing different representations of scattered and primary photon distributions. A multi-channel conditional GAN model was trained using SPECT images reconstructed using different energy windows as input to predict the corresponding sCT image. This model exhibited a normalized mean absolute error (NMAE) of 0.26 ± 0.15%.
] used emission-based generated activity distributions and μ-maps as input to generate high-quality sCT images for 18F-FDG brain PET studies. They reported less than 10% errors for CT values using CED and U-Net models. The same group applied the same approach in whole-body PET imaging using U-Net, achieving a relative error of 2.22 ± 1.77% across 20 subjects [
Generation of PET Attenuation Map for Whole-Body Time-of-Flight (18)F-FDG PET/MRI Using a deep neural network trained with simultaneously reconstructed activity and attenuation maps.
] proposed the estimation of attenuation correction factors from the different TOF sinogram bins using ResNet, reporting an absolute SUV bias of less than 7% in different regions of the brain.
In addition to generating sCTs using PET emission data, direct generation of attenuation and scatter corrected images from NAC images was reported. Shiri et al. [
] trained a 2D U-Net network using brain 18F-FDG PET studies reporting PET quantification bias of less than 5% in different regions of the brain. Arabi et al. [
] applied this approach to different brain molecular imaging probes, including 18F-FDG, 18F-DOPA, 18F-Flortaucipir, and 18F-Flutemetamol and reported SUV bias of less than 9% in different brain regions (Fig. 4). Shiri et al. [
] trained 2D, 3D, and patch-based ResNets on 1000 whole-body 18F-FDG images and tested the proposed models on unseen 150 subjects. They performed ROI-based and voxel-based assessments and reported a relative error of less than 5%. Dong et al. [
] trained a 3D patch-based Cycle-GAN for whole-body 18F-FDG images and reported a mean relative error of less than 5% calculated on malignant lesions. Emission-based ASC approaches using deep learning algorithms are summarized in Table 1.
Fig. 4Comparison of PET images corrected for attenuation using CT‐based, segmentation-based (containing background air and soft-tissue) (SegAC), and deep learning-guided (DLAC) approaches together with the reference CT image for 18F-FDG, 18F-DOPA, 18F-Flortaucipir, and 18F-Flutemetamol radiotracers. Difference SUV error maps are also presented for segmentation- and deep learning-based approaches.
Leynes AP, Ahn SP, Wangerin KA, Kaushik SS, Wiesinger F, Hope TA, et al. Bayesian deep learning Uncertainty estimation and pseudo-CT prior for robust Maximum Likelihood estimation of Activity and Attenuation (UpCT-MLAA) in the presence of metal implants for simultaneous PET/MRI in the pelvis. arXiv preprint arXiv:200103414. 2020.
Generation of PET Attenuation Map for Whole-Body Time-of-Flight (18)F-FDG PET/MRI Using a deep neural network trained with simultaneously reconstructed activity and attenuation maps.
Mostafapour S, Gholamiankhah F, Maroofpour S, Momennezhad M, Asadinezhad M, Zakavi SR, et al. Deep learning-based attenuation correction in the image domain for myocardial perfusion SPECT imaging. arXiv preprint arXiv:210204915. 2021.
The generation of sCT from MR images using deep learning-based regression approaches were reported in a number of studies. Li et al. used a 2D CED model to generate a 3-class probability map from T1-weighted images for 18F-FDG brain images and reported an average bias of less than 1% in different brain regions [
]. Arabi et al. reported on the development of a novel adversarial semantic structure GAN model using T1-weighted MR images to generate synthetic CT images for brain PET studies [
]. They reported a relative error of less than 4% in 64 anatomical brain regions. Leynes et al. used ZTE and Dixon MR sequences in a multi-channel input framework to train a U-Net model [
Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI.
]. The network was trained on 10 subjects using a patch extraction strategy and tested on 16 subjects consisting of the external validation set, reporting a quantification bias less than 5% in different ROIs defined on bones and soft tissues of 18F-FDG and 68Ga-PSMA-11 PET images. Ladefoged et al. evaluated 3D U-Net architectures with UTE MR sequence as input and reported a mean relative error of −0.1% in brain tumours [
]. The main contributions of deep learning-assisted MRI-guided attenuation and scatter correction in emission tomography are summarized in Table 2.
Table 2Summary of studies performed on MRI-guided synthetic CT generation using deep learning approaches. CV: Cross-Validation, ROI: Region of Interest, VOIs: Volume of Interest, HU: Hounsfield Unit.
Pseudo CT Image Synthesis and Bone Segmentation from MR images using adversarial networks with residual blocks for MR-based attenuation correction of brain PET Data.
Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI.
A learning-based automatic segmentation method on left ventricle in SPECT imaging. Medical Imaging Biomedical Applications in Molecular, Structural, and Functional Imaging.
International Society for Optics and Photonics.2019; : 109531M
Most deep learning-based ASC studies focused on brain imaging, which is less challenging compared to whole-body imaging where the anatomical structures are more complex with juxtapositions of various tissues having diverse attenuation properties and irregular shapes. There is obviously a need to evaluate these algorithms in more challenging heterogeneous regions, such as the chest and abdomen [
]. Moreover, the majority of these studies were performed using only one radiotracer (mostly 18F-FDG) which raises questions regarding the generalizability of the models and the need for retraining and reevaluation on other tracers [
]. The size of training and evaluation sets is another limitation of deep learning-based ASC as the performance of these algorithms depends on the training sample. To the best of our knowledge, only two studies, one focusing on brain imaging [
], which used a large number of training sets. Most deep learning-based ASC studies were performed in PET imaging with a limited number of works reported for SPECT imaging [
Regarding attenuation correction in the image domain, Nguyen et al. proposed a 3D Unet-GAN network that takes 3D patches (90 × 90 × 28 voxels) of non-AC images as input to directly predict attenuation corrected MPI-SPECT images [
]. The performance of the proposed network was compared with 2D Unet and 3D Unet models, wherein the 3D Unet-GAN model exhibited superior performance with NMAE = 0.034 and mean square error (MSE) = 294.97. Likewise, Mostafapour et al. examined attenuation correction of MPI-SPECT images in the image domain using ResNet and Unet models (in 2D mode). Chang’s attenuation correction method was also implemented to provide a baseline for performance assessment of the deep learning-based AC approaches. The clinical assessment and quantitative analysis demonstrated excellent agreement between deep learning- and CT-based AC with a mean quantification bias of 0.34 ± 5.03% [
Mostafapour S, Gholamiankhah F, Maroofpour S, Momennezhad M, Asadinezhad M, Zakavi SR, et al. Deep learning-based attenuation correction in the image domain for myocardial perfusion SPECT imaging. arXiv preprint arXiv:210204915. 2021.
Computer-aided tools for the analysis and processing of medical images have been developed to improve the reliability and robustness of the extracted features. Advanced machine-learning techniques are being developed to learn 1) effective similarity features, 2) a common feature representation, or 3) appearance mapping, in order to provide a model that can match large appearance variations [
Accurate organ/tumor delineation from molecular images is mainly used in the context of oncological PET imaging studies for quantitative analysis targeting various aspects, including severity scoring, radiation treatment planning, volumetric quantification, radiomic features extraction, etc. However, this is challenging owing to the poor spatial resolution and high statistical noise of molecular images. In current clinical practice, image segmentation is typically performed manually, which tends to be labor-intensive and prone to intra- and inter-observer variability. A number of recent studies explored the potential of DL-based automated tumor segmentation from PET or hybrid PET/CT examinations [
Mecheter I, Amira A, Abbod M, Zaidi H. Brain MR Imaging Segmentation Using Convolutional Auto Encoder Network for PET Attenuation Correction. In Proceedings of SAI Intelligent Systems Conference: Springer; 2020. p. 430-40.
]. Leung et al. developed a modular deep-learning framework for primary lung tumor segmentation from FDG-PET images with a small-size clinical training dataset, generalized across different scanners, achieving a Dice index of 0.73. They addressed the limitations of the small size of the training dataset as well as the accuracy and variability of manual segmentations used as ground truth by using a realistic simulation dataset [
Attenuation correction using 3D deep convolutional neural network for brain 18F-FDG PET/MR: comparison with Atlas, ZTE and CT based attenuation correction.
]. Wang et al. proposed a deep learning-assisted method for automated segmentation of the left ventricular region using gated myocardial perfusion SPECT [
Roccia et al. used a DL algorithm to predict the arterial input function for quantification of the regional cerebral metabolic rate from dynamic 18F-FDG PET scans [
]. Park et al. developed an automated pipeline for glomerular filtration rate (GFR) quantification of 99mTc-DTPA from SPECT/CT scans using a 3D U-Net model through kidney segmentation [
AI algorithms have been employed to build models exploiting the information extracted from medical images to perform a specific clinical task, e.g. object detection/classification, severity scoring, clinical outcome prediction, treatment planning, and monitoring response to therapy [
]. Xu et al. developed an automated pipeline using two cascaded V-NETs for lesion prediction and segmentation to detect multiple myeloma bone lesions from 68Ga-Pentixafor PET/CT [
]. Togo et al. demonstrated the feasibility of cardiac sarcoidosis detection from 18F-FDG PET scans using Inception-v3 network (83.9% sensitivity and 87% specificity), which outperformed conventional SUVmax- (46.8% sensitivity and 71.0% specificity) and coefficient of variance (CoV)-based (65.5% sensitivity and 75.0% specificity) approaches [
]. Ma et al. modified a DenseNet architecture for the diagnosis of thyroid disease using SPECT images into three categories: Graves’ disease, Hashimoto, and subacute thyroiditis [
18F-FDG PET is extensively used as a diagnostic tool in neurodegenerative disorders, especially Alzheimer Disease (AD) to improve diagnosis and monitor disease progression. The role of AI in AD diagnosis has been recently reviewed by Duffy et al. [
Duffy IR, Boyle AJ, Vasdev N. Improving PET Imaging Acquisition and Analysis With Machine Learning: A Narrative Review With Focus on Alzheimer's Disease and Oncology. Molecular imaging. 2019;18:1536012119869070.
]. Lu et al. developed an AI-based framework for the early diagnosis of AD using multimodal 18F-FDG PET/MR and multiscale deep neural network (82.4% accuracy and 94.23% sensitivity) [
]. Choi and Jin proposed a straightforward deep learning algorithm based on only 18F-FDG PET images for early detection of AD (84.2% accuracy) that outperformed conventional feature-based quantification approaches, e.g. Support-Vector-Machine (76.0% accuracy) and VOI-based (75.4% accuracy) techniques [
]. Machine learning algorithms have shown promising results in the classification of AD using brain PET images. Liu et al. proposed a classification algorithm of FDG PET images composed of 2D CNNs and recurrent neural networks (RNNs) [
]. The CNN model was trained to extract the features in 2D, while the RNN extracted the features in 3D mode (95.3% accuracy for AD vs controls and 83.9% for mild impairment vs controls). In a follow-up work, they proposed a cascaded CNN model to train the multi-level features of multimodal PET/MRI images. First, a patch-based 3D CNN was constructed, and then, a high-level 2D CNN followed by a softmax layer was trained to collect the high-level features. Finally, all features were concatenated followed by a softmax layer for AD classification [
]. The flexibility of AI algorithms enables learning the characteristics from heterogeneous data that have meaningful correlations but not obvious for the human interpreter. Zhou et al. developed a deep learning model for AD diagnosis using genetic input data, e.g. single nucleotide polymorphism in addition to radiological brain images that outperformed classification performance relative to other state‐of‐the‐art methods [
Betancur et al. exploited a deep learning model trained with a large multi-center clinical database for coronary artery disease prediction per-vessel to evaluate the automated prediction of obstructive disease from MPI-SPECT [
]. The effectiveness of the proposed method was compared with the total perfusion deficit (TPD) index. Overall, 1638 patients underwent 99mTc-sestamibi or tetrofosmin MPI-SPECT scans in 9 different sites. The diagnosis based on invasive coronary angiography examinations was considered as reference. The deep learning model led to a higher area under the receiver-operating characteristic curve for prediction of the obstructive disease compared to TPD for both patient-wise (0.80 vs. 0.78) and vessel-wise (0.76 vs. 0.73) analysis. Wang et al. developed a convolutional neural network for left ventricular functional assessment from gated MPI-SPECT images to circumvent the tedious and subjective task of manual segmentation/adjustment and measurement [
]. The evaluation on 56 normal and abnormal patients exhibited a left ventricular volume correlation coefficient of 0.910 ± 0.061 between AI- and physicians-based analysis. The diagnosis of Parkinson’s disease from brain SPECT scans using deep learning approaches has been investigated in [
], wherein 2723 patients from healthy and Parkinson's disease groups were examined. The deep learning approach demonstrated outstanding performance with a sensitivity of 99.04%, specificity of 99.63%, and accuracy of 99.34%, suggesting the remarkable potential in the diagnosis of Parkinson’s disease and its management.
Radiomics and precision medicine
Radiomics refers to a quantitative set of features, e.g. intensity, texture, and geometrical characteristics obtained from radiological images to discriminate quantifiable phenotypes that cannot be extracted through qualitative assessment of images. A radiomics model is commonly built through 4 steps: i) image acquisition/reconstruction; ii) VOI segmentation; iii) quantification/hand-crafted feature extraction; iv) statistical analysis [
]. While data-driven deep learning approaches are different from feature-driven approaches, deep learning has the ability to directly learn discriminative features from data in their natural raw form without the necessity to define VOIs or extract engineered features [
SPECT and PET images represent biological and physiopathological characteristics that can be quantitatively expressed using radiomics. Most studies focused on 18F-FDG PET images for prognosis (staging) or outcome prediction using handcrafted radiomics [
Next-generation radiogenomics sequencing for prediction of EGFR and KRAS mutation status in NSCLC patients using multimodal imaging and machine learning algorithms.
Pretreatment 18F-FDG PET/CT radiomics predict local recurrence in patients treated with stereotactic body radiotherapy for early-stage non-small cell lung cancer: a multicentric study.
]. Application of radiomics in SPECT has also been recently investigated by Ashrafnia et al. for prediction of coronary artery calcification in [99mTc]-sestamibi SPECT myocardial perfusion scans [
]. Rahmim et al. evaluated the extraction of radiomic features from longitudinal Dopamine transporter (DAT) SPECT images for outcome prediction in Parkinson’s disease [
]. DL-based radiomics was compared with feature-driven methods to highlight the advantages of CNNs compared to handcrafted radiomics for response prediction of chemotherapy in oesophageal cancer [
]. Wang et al. reported that CNNs did not outperform traditional radiomics in the classification of mediastinal lymph nodes of non-small lung cancer. Yet, it was preferred, since it was more user-friendly and required less data handling, and was less prone to feature selection bias [
AI has significantly impacted other fields of nuclear medicine through developing methods for radiation dose monitoring, dose reduction strategies, building theranostic decision trees, and dose limit compliance. In the era of precision medicine, personalized dosimetry is increasingly used in nuclear medicine. Targeted Radionuclide Therapy (TRT) has been recently merged with the concept of theranostics, a promising technique in radiation oncology. Despite the growing interest in dosimetry-guided patient-specific TRT, the one-fits-all approach is still used in routine clinical practice. In the context of individualized dose profiling, the construction of patient-specific computational models is the first step toward this goal [
]. Numerous works focused on the development of pipelines for the construction of patient-specific computational models applicable in personalized dosimetry in either therapy or diagnostic procedures [
A method of rapid quantification of patient-specific organ doses for CT using deep-learning-based multi-organ segmentation and GPU-accelerated Monte Carlo dose computing.
Fu W, Sharma S, Abadi E, Iliopoulos A-S, Wang Q, Lo JY, et al. iPhantom: a framework for automated creation of individualized computational phantoms and its application to CT organ dosimetry. arXiv preprint arXiv:200808730. 2020.
]. They used cascaded modules consisting of i) registration of patient CT images to an anchor phantom, ii) segmentation of organs using UNet structure, and iii) registration of segmented organs inside the deformed anchor phantom to generate an individualized computational model that is applicable for personalized dosimetry in both diagnostic and therapeutic procedures. Besides, the automatic segmentation of organs at risk for various application sites of TRT has been extensively studied. Jackson et al. developed a framework for automated monitoring of absorbed dosed in the kidneys of patients undergoing 177Lu-PSMA therapy [
]. They used a 3D CNN architecture for kidney segmentation to provide organ-level dosimetry from post-treatment SPECT imaging to estimate renal radiation doses from TRT. Tang et al. proposed a CNN-based algorithm for liver segmentation for personalized selective internal radiation therapy [
MC simulations using patient-specific anatomical and metabolic features constitute the current gold standard for internal dosimetry calculations. However, the approach suffers from exhaustive computational burden. Recently deep learning approaches have been employed in patient-specific dosimetry for monitoring or treatment plan optimization using molecular images (SPECT and PET). Akhavanallaf et al. developed an AI-based framework based on ResNet architecture for personalized dosimetry in nuclear medicine procedures [
]. They extended the key idea behind the voxel-based MIRD (Medical Internal Radiation Dose ) approach through the prediction of specific S-values according to the density map derived from CT images followed by calculation of the cumulated activity map from the predicted specific kernels (Fig. 5). A physics-informed deep neural network (DNN) was designed to predict the energy deposited in the volume surrounding a unit radioactive source in the center of the kernel. The input channel was fed with a density map whereas the output was MC-based deposited energy maps of the given radiotracer, referred to as specific S-value kernels. Lee et al. proposed a methodology employing deep learning for the direct generation of dose rate maps from 18F-FDG PET/CT images [
Fig. 5Schematic representation of the voxel-scale dosimetry procedure. The top and bottom panels show the deep learning-based specific S-value kernel prediction and MIRD-based voxel dosimetry formalism. Adapted from Ref.
Despite the substantial growth and widespread adoption of patient-specific TRT, the “one-size-fits-all” approach is still commonly used in the clinic. Del Prete et al. reported that in TRT, organs at risk rarely reach the conservative threshold dose while most tumors receive submaximal doses, thus leading to undertreatment of patients [
]. Therefore, retrospective studies involving patients receiving TRT allows the evaluation of the treatment response to the one-dose-fits-all approach and would demonstrate the critical nature of the transition to adaptive dosimetry-guided treatment planning. This technique requires a tool incorporating a module for automatic segmentation of tumors/organs at risk along with a fast and accurate personalized dosimetry module.
Challenges/opportunities and outlook
Over the past decade, there have been significant advances in deep learning-assisted developments which have impacted modern healthcare. The potential of AI-based solutions in various molecular imaging applications has been thoroughly explored in academic and corporate settings during the last decade. This article may, therefore, be viewed as an early album covering some of the many and varied snapshots of this rapidly growing field. At this time, these tools are still available only to experts in the field but there are many reasons to believe that it will be potentially available for routine use in the near future.
The proposed AI-based solutions in PET and SPECT imaging can be divided into two groups: (i) Techniques solely proposed to replace the current algorithms/frameworks due to their superior performance and (ii) approaches that have rendered previously impractical/unfeasible scenarios/frameworks using conventional methods feasible. In the first category, the promise of deep learning approaches consists in providing even slightly better functionality/performance compared to existing methods rather than undertaking an unprecedented functionality previously inconceivable. For example in PET instrumentation, Anger logic is used to determine the location of the interaction within the detector modules. Novel approaches based on deep learning methods tend to solely replace the Anger logic to achieve better localization and energy resolution. In this regard, novel deep learning approaches play the same role and compete with existing methods.
Likewise, in MRI-guided synthetic CT generation, deep learning approaches serve as alternative to atlas- or MRI segmentation-based techniques, whereas in the domain of noise reduction, current analytical models/algorithms are being replaced by deep learning methods. In this regard, the proposed deep learning methods would not revolutionarily alter the current frameworks or produce a paradigm shift, though they hold the promise of providing more accurate outcomes or requiring less human intervention, and easy adaptability to new input data. In this light, this category of AI-based solutions are more likely to be fully employed in clinical practice or on commercial systems since less standardization, protocols and frame redefinition, and staff retraining is required. For instance, deep learning-guided CT image reconstruction developed by GE Medical Systems obtained FDA approval [
Conversely, the extraordinary power of deep learning approaches has rendered many previously impractical/nonfeasible scenarios/frameworks feasible. This includes tasks, such as attenuation and scatter correction in the image domain, estimation of synthetic CT images from the non-attenuation corrected emission images, object completion of truncation date, image translation, and internal dosimetry. These processes are inherently ill-posed and in many cases, there is a lack of a mathematical framework associated with these problems. Such AI-based solutions, though offering unprecedented opportunities in PET and SPECT imaging, face thoughtful challenges with respect to their deployment in clinical practice as they require extensive validation using large clinical databases and a wide range of conditions.
Overall, a clear distinction should be made between the applications of AI-based solutions as processing or decision support tools or the replacement of experts or clinicians in clinical practice. Considering the superior performance of deep learning approaches, some algorithms are sufficiently mature and robust to be deployed in clinical practice as decision support tools. These algorithms are supposed to replace conventional methods owing to their superior performance or robustness. In this regard, any possible failure of the AI-based solution would be treated in a similar way to existing approaches. Conversely, AI-based solutions deemed to fully replace the experts are still considered as fantasy or science-fiction. Such algorithms still require additional development and remarkable evolution to be independently employed in clinical setting. Nevertheless, these algorithms could play a significant role in the short-run as decision support tools to create a synergy between the capabilities of AI and human expertise.
It is gratifying to see in overview the progress that AI has made, from early developments in neural networks to complex deep learning architectures, and more recently towards continuous learning AI in radiology [
]. Challenges remain, particularly in the areas of clinical validation and liability towards wider adoption, ethical and legal aspects and a number of other issues that need to be settled [
This work was supported by the Swiss National Science Foundation under grant SNRF 320030_176052 and the Private Foundation of Geneva University Hospitals under grant RC-06-01.
Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:180403999. 2018.
Isola P, Zhu J-Y, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition2017. p. 1125-34.
Zhu J-Y, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision2017. p. 2223-32.
Gladen R, Chirayath V, Fairchild A, Manry M, Koymen A, Weiss A. Efficient Machine Learning Approach for Optimizing the Timing Resolution of a High Purity Germanium Detector. arXiv preprint arXiv:200400008. 2020.
Lu S, Tan J, Gao Y, Shi Y, Liang Z. Prior knowledge driven machine learning approach for PET sinogram data denoising. Medical Imaging 2020: Physics of Medical Imaging: International Society for Optics and Photonics; 2020. p. 113124A.
Sanaat A, Shiri I, Arabi H, Mainta I, Nkoulou R, and Zaidi H. Deep learning-assisted ultra-fast/low-dose whole-body PET/CT imaging. Eur J Nucl Med Mol Imaging; 2021 in press.
Shiri I, AmirMozafari Sabet K, Arabi H, Pourkeshavarz M, Teimourian B, Ay MR, et al. Standard SPECT myocardial perfusion estimation from half-time acquisitions using deep convolutional residual neural networks. J Nucl Cardiol; 2021 in press.
Song C, Yang Y, Wernick MN, Pretorius PH, King MA. Low-Dose Cardiac-Gated SPECT Studies Using a Residual Convolutional Neural Network. In IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) 2019. p. 653–6.
Magnetic resonance-based attenuation correction and scatter correction in neurological positron emission tomography/magnetic resonance imaging—current status with emerging applications.
Generation of PET Attenuation Map for Whole-Body Time-of-Flight (18)F-FDG PET/MRI Using a deep neural network trained with simultaneously reconstructed activity and attenuation maps.
Zero-Echo-Time and Dixon Deep Pseudo-CT (ZeDD CT): Direct Generation of Pseudo-CT Images for Pelvic PET/MRI attenuation correction using deep convolutional neural networks with multiparametric MRI.
Mostafapour S, Gholamiankhah F, Maroofpour S, Momennezhad M, Asadinezhad M, Zakavi SR, et al. Deep learning-based attenuation correction in the image domain for myocardial perfusion SPECT imaging. arXiv preprint arXiv:210204915. 2021.
Mecheter I, Amira A, Abbod M, Zaidi H. Brain MR Imaging Segmentation Using Convolutional Auto Encoder Network for PET Attenuation Correction. In Proceedings of SAI Intelligent Systems Conference: Springer; 2020. p. 430-40.
Attenuation correction using 3D deep convolutional neural network for brain 18F-FDG PET/MR: comparison with Atlas, ZTE and CT based attenuation correction.
Leynes AP, Ahn SP, Wangerin KA, Kaushik SS, Wiesinger F, Hope TA, et al. Bayesian deep learning Uncertainty estimation and pseudo-CT prior for robust Maximum Likelihood estimation of Activity and Attenuation (UpCT-MLAA) in the presence of metal implants for simultaneous PET/MRI in the pelvis. arXiv preprint arXiv:200103414. 2020.
Pseudo CT Image Synthesis and Bone Segmentation from MR images using adversarial networks with residual blocks for MR-based attenuation correction of brain PET Data.
A learning-based automatic segmentation method on left ventricle in SPECT imaging. Medical Imaging Biomedical Applications in Molecular, Structural, and Functional Imaging.
International Society for Optics and Photonics.2019; : 109531M
Duffy IR, Boyle AJ, Vasdev N. Improving PET Imaging Acquisition and Analysis With Machine Learning: A Narrative Review With Focus on Alzheimer's Disease and Oncology. Molecular imaging. 2019;18:1536012119869070.
Next-generation radiogenomics sequencing for prediction of EGFR and KRAS mutation status in NSCLC patients using multimodal imaging and machine learning algorithms.
Pretreatment 18F-FDG PET/CT radiomics predict local recurrence in patients treated with stereotactic body radiotherapy for early-stage non-small cell lung cancer: a multicentric study.
A method of rapid quantification of patient-specific organ doses for CT using deep-learning-based multi-organ segmentation and GPU-accelerated Monte Carlo dose computing.
Fu W, Sharma S, Abadi E, Iliopoulos A-S, Wang Q, Lo JY, et al. iPhantom: a framework for automated creation of individualized computational phantoms and its application to CT organ dosimetry. arXiv preprint arXiv:200808730. 2020.
00124-1&id=ga1.jpg){kind=link}
00124-1&id=gr1.jpg){kind=link}
00124-1&id=gr2.jpg){kind=link}
00124-1&id=gr3.jpg){kind=link}
00124-1&id=gr4.jpg){kind=link}
00124-1&id=gr5.jpg){kind=link}