Highlights
- •The U-ConformerNet structure is used to exploit the potential of DIP algorithm.
- •The feature distance regularization is used to enhance image quality and detail.
- •The algorithm can be freely switched between supervised and unsupervised modes.
- •The regularization based on image feature is better than traditional TV constraint.
Abstract
Purpose
The reconstruction performance of the deep image prior (DIP) approach is limited by
the conventional convolutional layer structure and it is difficult to enhance its
potential. In order to improve the quality of image reconstruction and suppress artifacts,
we propose a DIP algorithm with better performance, and verify its superiority in
the latest case.
Methods
We construct a new U-ConformerNet structure as the DIP algorithm's network, replacing
the traditional convolutional layer-based U-net structure, and introduce the 'lpips'
deep network based feature distance regularization method. Our algorithm can switch
between supervised and unsupervised modes at will to meet different needs.
Results
The reconstruction was performed on the low dose CT dataset (LoDoPaB). Our algorithm
attained a PSNR of more than 35 dB under unsupervised conditions, and the PSNR under
the supervised condition is greater than 36 dB. Both of which are better than the
performance of the DIP-TV. Furthermore, the accuracy of this method is positively
connected with the quality of the a priori image with the help of deep networks. In
terms of noise eradication and artifact suppression, the DIP algorithm with U-ConformerNet
structure outperforms the standard DIP method based on convolutional structure.
Conclusions
It is known by experimental verification that, in unsupervised mode, the algorithm
improves the output PSNR by at least 2–3 dB when compared to the DIP-TV algorithm
(proposed in 2020). In supervised mode, our algorithm approaches that of the state-of-the-art
end-to-end deep learning algorithms.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Physica Medica: European Journal of Medical PhysicsAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Iterative reconstruction methods in X-ray CT.Phys Med. 2012; 28: 94-108https://doi.org/10.1016/j.ejmp.2012.01.003
Jin X, Li L, Chen Z, Zhang L, Xing Y, Anisotropic total variation for limited-angle CT reconstruction. In IEEE Nuclear Science Symposuim & Medical Imaging Conference. 2010. IEEE. 10.1109/NSSMIC.2010.5874180.
- Deep learning for biomedical image reconstruction: a survey.Artif Intell Rev. 2021; 54: 215-251https://doi.org/10.1007/s10462-020-09861-2
- A review of GPU-based medical image reconstruction.Phys Med. 2017; 42: 76-92https://doi.org/10.1016/j.ejmp.2017.07.024
- Artificial intelligence in image reconstruction: The change is here.Phys Med. 2020; 79: 113-125https://doi.org/10.1016/j.ejmp.2020.11.012
- Machine learning in quantitative PET: A review of attenuation correction and low-count image reconstruction methods.Phys Med. 2020; 76: 294-306https://doi.org/10.1016/j.ejmp.2020.07.028
- AI applications to medical images: From machine learning to deep learning.Phys Med. 2021; 83: 9-24https://doi.org/10.1016/j.ejmp.2021.02.006
- Basic of machine learning and deep learning in imaging for medical physicists.Phys Med. 2021; 83: 194-205https://doi.org/10.1016/j.ejmp.2021.03.026
- Artificial intelligence and machine learning for medical imaging: A technology review.Phys Med. 2021; 83: 242-256https://doi.org/10.1016/j.ejmp.2021.04.016
- Requirements and reliability of AI in the medical context.Phys Med. 2021; 83: 72-78https://doi.org/10.1016/j.ejmp.2021.02.024
- Current challenges of implementing artificial intelligence in medical imaging.Phys Med. 2022; 100: 12-17https://doi.org/10.1016/j.ejmp.2022.06.003
- Artifact removal using a hybrid-domain convolutional neural network for limited-angle computed tomography imaging.Phys Med Biol. 2020; 65155010https://doi.org/10.1088/1361-6560/ab9066
- Structurally-Sensitive Multi-Scale Deep Neural Network for Low-Dose CT Denoising.IEEE Access. 2018; 6: 41839-41855https://doi.org/10.1109/ACCESS.2018.2858196
- Segmentation of renal tumors in CT images by 3D U-Net preserving rotational symmetry in axial slices.Opt Continuum. 2022; 1: 297-305https://doi.org/10.1364/OPTCON.451024
- Image and video restoration and compression artefact removal using a NoGAN approach.in: Proceedings of the 28th ACM International Conference on Multimedia. 2020https://doi.org/10.1145/3394171.3414451
- Task-based characterization of a deep learning image reconstruction and comparison with filtered back-projection and a partial model-based iterative reconstruction in abdominal CT: A phantom study.Phys Med. 2020; 76: 28-37https://doi.org/10.1016/j.ejmp.2020.06.004
- and Öktem O, Solving ill-posed inverse problems using iterative deep neural networks.Inverse Prob. 2017; 33124007https://doi.org/10.1088/1361-6420/aa9581
- Deep Learning Computed Tomography: Learning Projection-Domain Weights From Image Domain in Limited Angle Problems.IEEE Trans Med Imaging. 2018; 37: 1454-1463https://doi.org/10.1109/TMI.2018.2833499
- ADMM-based deep reconstruction for limited-angle CT.Phys Med Biol. 2019; 64115011https://doi.org/10.1088/1361-6560/ab1aba
- Disentangled generative adversarial network for low-dose CT.EURASIP J Adv Signal Process. 2021; 2021: 34https://doi.org/10.1186/s13634-021-00749-z
- LEARN: Learned Experts’ Assessment-Based Reconstruction Network for Sparse-Data CT.IEEE Trans Med Imaging. 2018; 37: 1333-1347https://doi.org/10.1109/TMI.2018.2805692
- Learned Primal-Dual Reconstruction.IEEE Trans Med Imaging. 2018; 37: 1322-1332https://doi.org/10.1109/TMI.2018.2799231
Ronneberger O, Fischer P, and Brox T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. 2015. Springer. 10.1007/978-3-319-24574-4_28.
- Deep image prior.in: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2018: 9446-9454
- Computed tomography reconstruction using deep image prior and learned reconstruction methods.Inverse Prob. 2020; 36094004https://doi.org/10.1088/1361-6420/aba415
Chakrabarty P. and Maji S, The spectral bias of the deep image prior. arXiv preprint arXiv:1912.08905; 2019. doi:10.48550/arXiv.1912.08905.
- Taming transformers for high-resolution image synthesis.in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021: 12873-12883
- Dual-domain sparse-view CT reconstruction with Transformers.Phys Med. 2022; 101: 1-7https://doi.org/10.1016/j.ejmp.2022.07.001
- Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT.IEEE Trans Med Imaging. 2018; 37: 1418-1429https://doi.org/10.1109/TMI.2018.2823768
- Conformer: Local Features Coupling Global Representations for Visual Recognition.in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2021: 367-376
- Image reconstruction by domain-transform manifold learning.Nature. 2018; 555: 487-492https://doi.org/10.1038/nature25988
- Intelligent Parameter Tuning in Optimization-Based Iterative CT Reconstruction via Deep Reinforcement Learning.IEEE Trans Med Imaging. 2018; 37: 1430-1439https://doi.org/10.1109/TMI.2018.2823679
Dranoshchuk AD, and Veselov AI. About perceptual quality estimation for image compression. 2019 Wave Electronics and its Application in Information and Telecommunication Systems (WECONF). IEEE, 2019. Doi: 10.1109/WECONF.2019.8840116.
- The Beer-Lambert Law.J Chem Educ. 1962; 39: 333https://doi.org/10.1021/ed039p333
- Sparse-view x-ray CT reconstruction via total generalized variation regularization.Phys Med Biol. 2014; 59: 2997-3017https://doi.org/10.1088/0031-9155/59/12/2997
- The unreasonable effectiveness of deep features as a perceptual metric.in: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2018: 586-595
- LoDoPaB-CT, a benchmark dataset for low-dose computed tomography reconstruction.Sci Data. 2021; 8: 109https://doi.org/10.1038/s41597-021-00893-z
- Deep Convolutional Neural Network for Inverse Problems in Imaging.IEEE Trans Image Process. 2017; 26: 4509-4522https://doi.org/10.1109/TIP.2017.2713099
Article info
Publication history
Published online: February 08, 2023
Accepted:
January 25,
2023
Received in revised form:
January 4,
2023
Received:
August 11,
2022
Identification
Copyright
© 2023 Associazione Italiana di Fisica Medica e Sanitaria. Published by Elsevier Ltd. All rights reserved.
