A novel edge gradient distance metric for automated evaluation of deformable image registration quality

  • Yihang Xu
    Department of Radiation Oncology, University of Miami Miller School of Medicine, Miami, FL, USA

    Department of Biomedical Engineering, University of Miami College of Engineering, Coral Gables, FL, USA
    Search for articles by this author
  • Jeffery F. Williamson
    Department of Radiation Oncology, Washington University School of Medicine, St. Louis, MO, USA
    Search for articles by this author
  • Nesrin Dogan
    Department of Radiation Oncology, University of Miami Miller School of Medicine, Miami, FL, USA
    Search for articles by this author
  • Taylor Harry
    Department of Radiation Oncology, University of Miami Miller School of Medicine, Miami, FL, USA
    Search for articles by this author
  • John Chetley Ford
    Corresponding author at: Sylvester Comprehensive Cancer Center, University of Miami Miller School of Medicine, Department of Radiation Oncology, 1475 NW 12th Avenue, Suite C123, Miami, FL 33136, USA.
    Department of Radiation Oncology, University of Miami Miller School of Medicine, Miami, FL, USA

    Department of Biomedical Engineering, University of Miami College of Engineering, Coral Gables, FL, USA
    Search for articles by this author
Published:October 08, 2022DOI:


      • A new metric, EGDTA, was developed and tested for automated DIR quality evaluation.
      • The EGDTA map and imposed DVF on phantom and CT images showed good agreement.
      • The EGDTA metric provides the expected dependence on DIR type and ROI location.
      • The EGDTA metric shows potential as an automated means of comparing DIR algorithms.



      To develop a new registration quality metric, based on the distance between image edges, for automated evaluation and comparison of DIR algorithms.


      Canny filter is used to create binary gradient images from input images to be compared. A small subregion of one binary image is translated relative to the other. The translational distance maximizing overlap of edges in the subregion is the local edge gradient distance to agreement (EGDTA); repeating over all subregions provides an EGDTA map. The method was tested on phantom and pelvic CT images, by applying a known deformable vector field (DVF). The method was then applied to evaluate two DIR algorithms (SICLE and Demons) for pelvic CTs from five patients. Three SICLE variants were used: Grayscale-driven (G), Contour-driven (C), and Grayscale + Contour-driven (G + C). For each patient, a planning CT was registered to three repeat CTs using the above DIR algorithms. Mean EGDTA values in concentric ring regions of interest close to and far away from the pelvic organ contours were compared.


      EGDTA maps and imposed DVF deformations on phantom and CT images demonstrated agreement. In comparison of the three variants of SICLE: C had lower EGDTA values close to the pelvic organs, while G showed much better performance in the regions distant from the organs compared to C; and G + C registration exhibited the lowest or comparable EGDTA value among three variants. Demons achieved the lowest EGDTA values.


      The EGDTA metric shows potential as an automated means of evaluating and comparing DIR algorithms.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Physica Medica: European Journal of Medical Physics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Jaffray D.
        • Kupelian P.
        • Djemil T.
        • Macklis R.M.
        Review of image-guided radiation therapy.
        Expert Rev Anticancer Ther. 2007; 7: 89-103
        • Jaffray D.A.
        Image-guided radiotherapy: from current concept to future perspectives.
        Nat Rev Clin Oncol. 2012; 9: 688-699
        • Ghilezan M.
        • Yan D.
        • Martinez A.
        Adaptive radiation therapy for prostate cancer.
        Sem Radiat Oncol. 2010; 20: 130-137
        • Birkner M.
        • Yan D.
        • Alber M.
        • Liang J.
        • Nüsslin F.
        Adapting inverse planning to patient and organ geometrical variation: algorithm and implementation.
        Med Phys. 2003; 30: 2822-2831
        • Chao M.
        • Xie Y.Q.
        • Xing L.
        Auto-propagation of contours for adaptive prostate radiation therapy.
        Phys Med Biol. 2008; 53: 4533-4542
        • Ghilezan M.
        • et al.
        Online image-guided intensity-modulated radiotherapy for prostate cancer: How much improvement can we expect? A theoretical assessment of clinical benefits and potential dose escalation by improving precision and accuracy of radiation delivery.
        Int J Radiat Oncol Biol Phys. 2004; 60: 1602-1610
        • Hardcastle N.
        • et al.
        A multi-institution evaluation of deformable image registration algorithms for automatic organ delineation in adaptive head and neck radiotherapy.
        Radiation Oncology. 2012; 7
        • Schaly B.
        • et al.
        Dosimetric impact of image-guided 3D conformal radiation therapy of prostate cancer.
        Phys Med Biol. 2005; 50: 3083-3101
        • Schulze D.
        • et al.
        Comparison of various online IGRT strategies: The benefits of online treatment plan re-optimization.
        Radiother Oncol. 2009; 90: 367-376
      1. Brock KK, et al., Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132. Med Phys, 2017. 44(7): p. e43–e76.

        • Murphy K.
        • et al.
        Semi-automatic construction of reference standards for evaluation of image registration.
        Med Image Anal. 2011; 15: 71-84
      2. Song JH, et al., Evaluating Image Registration Using NIREP, in Biomedical Image Registration, B. Fischer, B.M. Dawant, and C. Lorenz, Editors. 2010. p. 140–150.

        • Rohlfing T.
        Image similarity and tissue overlaps as surrogates for image registration accuracy: widely used but unreliable.
        IEEE Trans Med Imaging. 2012; 31: 153-163
        • Wang Y.
        • et al.
        Quantitative evaluation and optimization of an algorithm for non-rigidly registering serial CT images to the planning CT during prostate cancer radiation therapy.
        Med Phys. 2010; 37
        • Castillo R.
        • et al.
        A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets.
        Phys Med Biol. 2009; 54: 1849-1870
        • Kumar K.
        • et al.
        A validation framework to assess performance of commercial deformable image registration in lung radiotherapy.
        Physica Med. 2021; 87: 106-114
        • Masi M.
        • et al.
        Comparison of rigid and deformable coregistration between mpMRI and CT images in radiotherapy of prostate bed cancer recurrence.
        Phys Med. 2021; 92: 32-39
        • Paganelli C.
        • et al.
        Patient-specific validation of deformable image registration in radiation therapy: overview and caveats.
        Med Phys. 2018; 45: E908-E922
        • Nobnop W.
        • et al.
        Accuracy of eight deformable image registration (DIR) methods for tomotherapy megavoltage computed tomography (MVCT) images.
        J Med Radiat Sci. 2017; 64: 290-298
        • Dogan N.
        • et al.
        SU-F-J-84: Comparison of quantitative deformable image registration evaluation tools: application to prostate IGART.
        Med Phys. 2016; 43: 3425-3426
        • Yang B.
        • et al.
        A feasible method to evaluate deformable image registration with deep learning–based segmentation.
        Physica Med. 2022; 95: 50-56
        • Galib S.M.
        • et al.
        A fast and scalable method for quality assurance of deformable image registration on lung CT scans using convolutional neural networks.
        Med Phys. 2020; 47: 99-109
        • Muenzing S.E.A.
        • et al.
        Supervised quality assessment of medical image registration: Application to intra-patient CT lung registration.
        Med Image Anal. 2012; 16: 1521-1531
        • Lowe D.G.
        Object recognition from local scale-invariant features.
        Proceedings of the seventh IEEE international conference on computer vision. IEEE, 1999
        • Paganelli C.
        • et al.
        Scale invariant feature transform in adaptive radiation therapy: a tool for deformable image registration assessment and re-planning indication.
        Phys Med Biol. 2012; 58: 287-299
        • Paganelli C.
        • et al.
        Quantification of organ motion based on an adaptive image-based scale invariant feature method.
        Med Phys. 2013; 40111701
        • Landry G.
        • et al.
        Investigating CT to CBCT image registration for head and neck proton therapy as a tool for daily dose recalculation.
        Med Phys. 2015; 42: 1354-1366
        • Lindeberg T.
        Scale selection properties of generalized scale-space interest point detectors.
        J Math Imaging Vision. 2013; 46: 177-210
        • Hauler F.
        • et al.
        Automatic quantification of multi-modal rigid registration accuracy using feature detectors.
        Phys Med Biol. 2016; 61: 5198-5214
      3. Simo-Serra E, et al. Discriminative learning of deep convolutional feature point descriptors. In: IEEE International Conference on Computer Vision. 2015. Santiago, CHILE.

      4. Zagoruyko S, Komodakis N, and Ieee. Learning to Compare Image Patches via Convolutional Neural Networks. in IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2015. Boston, MA.

      5. Yang D, et al., A method to detect landmark pairs accurately between intra-patient volumetric medical images. Med Phys, 2017. 44(11): p. 5859–72.

        • Fu Y.
        • et al.
        Automatic large quantity landmark pairs detection in 4DCT lung images.
        Med Phys. 2019; 46: 4490-4501
        • Christensen G.E.
        • Johnson H.J.
        Consistent image registration.
        IEEE Trans Med Imaging. 2001; 20: 568-582
        • Bender E.T.
        • Tomé W.A.
        The utilization of consistency metrics for error analysis in deformable image registration.
        Phys Med Biol. 2009; 54: 5561
        • Schreibmann E.
        • et al.
        A measure to evaluate deformable registration fields in clinical settings.
        J Appl Clin Med Phys. 2012; 13
        • Chen M.
        • et al.
        A simple fixed-point approach to invert a deformation field.
        Med Phys. 2008; 35: 81-88
        • Vercauteren T.
        • et al.
        Diffeomorphic demons: Efficient non-parametric image registration.
        NeuroImage. 2009; 45: S61-S72
        • Saleh Z.H.
        • et al.
        The distance discordance metric-a novel approach to quantifying spatial uncertainties in intra- and inter-patient deformable image registration.
        Phys Med Biol. 2014; 59: 733-746
        • Kierkels R.G.J.
        • et al.
        An automated, quantitative, and case-specific evaluation of deformable image registration in computed tomography images.
        Phys Med Biol. 2018; 63045026
        • Brejl M.
        • Sonka M.
        Directional 3D edge detection in anisotropic data: detector design and performance assessment.
        Comput Vis Image Underst. 2000; 77: 84-110
        • Monga O.
        • et al.
        Recursive filtering and edge tracking: two primary tools for 3D edge detection.
        Image Vis Comput. 1991; 9: 203-214
        • Canny J.
        A computational approach to edge-detection.
        Ieee Trans Pattern Anal Mach Intelligence. 1986; 8: 679-698
        • Sim D.G.
        • Kwon O.K.
        • Park R.H.
        Object matching algorithms using robust Hausdorff distance measures.
        IEEE Trans Image Process. 1999; 8: 425-429
        • Huttenlocher D.P.
        • Klanderman G.A.
        • Rucklidge W.J.
        Comparing images using the Hausdorff distance.
        IEEE Trans Pattern Anal Mach Intell. 1993; 15: 850-863