Advertisement

The geometric and dosimetric effect of algorithm choice on propagated contours from CT to cone beam CTs

      Highlights

      • Comparison of 5 commercially available software packages for head and neck contour propagation.
      • Geometric and dosimetric comparison against two gold standards (clinician, STAPLE)
      • Geometric differences were seen for manual but not STAPLE gold standard.
      • DVH parameters were consistent for most contours (ICC > 0.75)
      • Large differences in geometric metrics rarely had an impact on DVH parameters.

      Abstract

      Purpose

      Adaptive radiotherapy relies of rapid re-contouring, online more so than offline. Intra-patient contour propagation via non-rigid registration offers a solution but can be of limited accuracy. However, the dosimetric significance of the inaccuracies is unknown. Here we evaluate the dosimetric reliability of contours generated by different commercially-available software packages.

      Method

      Planning CT contours for ten head and neck cancer patients were propagated via five commercial packages to five CBCT scans acquired throughout treatment. The treatment plan was recalculated on each of the CBCTs for each set of propagated contours, and DVH parameters extracted for the spinal cord, brainstem, parotids and larynx. The propagated contours were compared to two gold standard contours: contours manually outlined and a consensus STAPLE contours generated from the propagated contours. Geometrical similarity was evaluated using mean distance to agreement (mDTA), Hausdorff distance, centroid agreement and Dice similarity coefficient. Dosimetric reliability was assessed against clinical constraints and comparing via the intraclass correlation coefficient (ICC).

      Results

      All propagated contours were similar to the STAPLE (mDTA < 1.0 mm) whilst larger differences were seen for the manual contours (mDTA < 3.0 mm). The dosimetric comparison showed that the propagated contours gave excellent dose estimates for most organs. The spinal cord reliability was moderate (ICC > 0.66).

      Conclusions

      Large differences in geometric metrics rarely had a statistically significant impact on DVH parameters for the OARs studied. For that reason, propagated contours on treatment CBCT images are suitable for estimating dose to the OARs.

      Keywords

      Introduction

      Approximately 80% of head and neck cancer patients receive radiotherapy as part of their treatment [
      • Strojan P.
      • Hutcheson K.A.
      • Eisbruch A.
      • Beitler J.J.
      • Langendijk J.A.
      • Lee A.W.M.
      • et al.
      Treatment of late sequelae after radiotherapy for head and neck cancer.
      ]. Patients usually receive highly conformal treatment with sharp dose gradients using techniques such as intensity modulated radiotherapy (IMRT) or volumetric modulated arc therapy (VMAT). IMRT has been shown to reduce the side effects of radiotherapy, specifically xerostomia [
      • Nutting C.M.
      • Morden J.P.
      • Harrington K.J.
      • Urbano T.G.
      • Bhide S.A.
      • Clark C.
      • et al.
      Parotid-sparing intensity modulated versus conventional radiotherapy in head and neck cancer (PARSPORT): a phase 3 multicentre randomised controlled trial.
      ]. These techniques rely on a high degree of patient immobility, requiring immobilising thermoplastic head/neck shells along with regular imaging (e.g. cone beam computed tomography (CBCT)) to correct the patient position for any rigid body translations and rotations – a process known as image guided radiotherapy. However, the patient anatomy changes during treatment, e.g., due to weight loss, and changes of organ’s location or shape [
      • Sonke J.J.
      • Aznar M.
      • Rasch C.
      Adaptive Radiotherapy for Anatomical Changes.
      ]. Changes to the regions affected by cancer, namely the clinical target volumes (CTVs) are also expected, with tumours often shrinking during treatment [
      • Sonke J.J.
      • Aznar M.
      • Rasch C.
      Adaptive Radiotherapy for Anatomical Changes.
      ,
      • Bhide S.A.
      • Davies M.
      • Burke K.
      • McNair H.A.
      • Hansen V.
      • Barbachano Y.
      • et al.
      Weekly Volume and Dosimetric Changes During Chemoradiotherapy With Intensity-Modulated Radiation Therapy for Head and Neck Cancer: A Prospective Observational Study.
      ,
      • Castadot P.
      • Lee J.A.
      • Geets X.
      • Grégoire V.
      Adaptive Radiotherapy of Head and Neck Cancer.
      ,
      • Grégoire V.
      • Jeraj R.
      • Lee J.A.
      • O’Sullivan B.
      Radiotherapy for head and neck tumours in 2012 and beyond: Conformal, tailored, and adaptive?.
      ,
      • Vásquez Osorio E.M.
      • Hoogeman M.S.
      • Al-Mamgani A.
      • Teguh D.N.
      • Levendag P.C.
      • Heijmen B.J.M.
      Local Anatomic Changes in Parotid and Submandibular Glands During Radiotherapy for Oropharynx Cancer and Correlation With Dose, Studied in Detail With Nonrigid Registration.
      ]. These changes potentially result in suboptimal treatment through geometric misses or increased dose to normal tissues [
      • Castadot P.
      • Geets X.
      • Lee J.A.
      • Grégoire V.
      Adaptive functional image-guided IMRT in pharyngo-laryngeal squamous cell carcinoma: Is the gain in dose distribution worth the effort?.
      ,
      • Castadot P.
      • Geets X.
      • Lee J.A.
      • Christian N.
      • Grégoire V.
      Assessment by a deformable registration method of the volumetric and positional changes of target volumes and organs at risk in pharyngo-laryngeal tumors treated with concomitant chemo-radiation.
      ,
      • Hansen E.K.
      • Bucci M.K.
      • Quivey J.M.
      • Weinberg V.
      • Xia P.
      Repeat CT imaging and replanning during the course of IMRT for head-and-neck cancer.
      ,
      • Chen C.
      • Fei Z.
      • Lisha C.
      • Bai P.
      • Lin X.
      • Pan J.
      Will weight loss cause significant dosimetric changes of target volumes and organs at risk in nasopharyngeal carcinoma treated with intensity-modulated radiation therapy?.
      ].
      A method to tackle the impact of these anatomical changes is to use adaptive radiotherapy (ART) [
      • Sonke J.J.
      • Aznar M.
      • Rasch C.
      Adaptive Radiotherapy for Anatomical Changes.
      ]. In ART the original treatment plan is adapted to account for changes in patient anatomy, either offline (i.e. between treatment fractions, can only correct for systematic anatomical or physiological changes) or online (i.e. during the treatment fraction, correcting for both systematic and random changes, assuming the latter remains constant whilst the new plan is created) [
      • Castadot P.
      • Lee J.A.
      • Geets X.
      • Grégoire V.
      Adaptive Radiotherapy of Head and Neck Cancer.
      ,
      • Vickress J.R.
      • Battista J.
      • Barnett R.
      • Yartsev S.
      Online daily assessment of dose change in head and neck radiotherapy without dose-recalculation.
      ]. This requires repeat imaging, usually CT, CBCT or MR guided, and time-consuming repeat contouring [
      • Moazzezi M.
      • Rose B.
      • Kisling K.
      • Moore K.L.
      • Ray X.
      Prospects for daily online adaptive radiotherapy via ethos for prostate cancer patients without nodal involvement using unedited CBCT auto-segmentation.
      ,
      • Klüter S.
      Technical design and concept of a 0.35 T MR-Linac.
      ], with replanning on the treatment CBCT images being ideal due to the patient being in the treatment position. Non-rigid registration, also known as deformable image registration, offers one potential solution for manual contouring, whereby the original planning contours can be propagated to the CBCT (or subsequent CT) scans. Many commercial packages exist for this.
      There has been significant work undertaken evaluating the geometric accuracy of propagated OARs on CBCT with relatively good results, although manual correction is often required [
      • Tsuji S.Y.
      • Hwang A.
      • Weinberg V.
      • Yom S.S.
      • Quivey J.M.
      • Xia P.
      Dosimetric Evaluation of Automatic Segmentation for Adaptive IMRT for Head-and-Neck Cancer.
      ,
      • Paganelli C.
      • Meschini G.
      • Molinelli S.
      • Riboldi M.
      • Baroni G.
      Patient-specific validation of deformable image registration in radiation therapy: Overview and caveats.
      ], with some evidence to suggest manual correction is nearly always required [
      • Teguh D.N.
      • Levendag P.C.
      • Voet P.W.J.
      • Al-Mamgani A.
      • Han X.
      • Wolf T.K.
      • et al.
      Clinical validation of atlas-based auto-segmentation of multiple target volumes and normal tissue (swallowing/mastication) structures in the head and neck.
      ]. However, it is ultimately the dose the patient receives that is important and contouring errors may not correlate with dose [
      • Beasley W.J.
      • McWilliam A.
      • Aitkenhead A.
      • Mackay R.I.
      • Rowbottom C.G.
      The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy.
      ,
      • Hvid C.A.
      • Elstrøm U.V.
      • Jensen K.
      • Alber M.
      • Grau C.
      Accuracy of software-assisted contour propagation from planning CT to cone beam CT in head and neck radiotherapy.
      ]. The local dose environment in the vicinity of the structure may be of more importance, with structures near sharp dose gradients being more sensitive to contouring errors depending on the dose parameter of interest (e.g. maximum or average dose). Therefore, propagated contours potentially could be of sufficient accuracy when assessed based on dosimetric end-points [
      • Eiland R.B.
      • Maare C.
      • SjöStröm D.
      • Samsøe E.
      • Behrens C.F.
      Dosimetric and geometric evaluation of the use of deformable image registration in adaptive intensity-modulated radiotherapy for head-and-neck cancer.
      ].
      The purpose of this work is to assess whether the geometric variability of propagated contours results in meaningful differences in dose to the OAR. For this, five commercial deformation packages have been used to propagate contours. We assessed geometrical and dosimetric differences of the propagated contours against two gold standards. This would inform whether propagated contours can be used for organ dose estimation in adaptive radiotherapy without correction or to inform which contours should be corrected prior to replanning, to allow for a more efficient process.

      Method

      Patient selection and propagation

      Ten head and neck cancer patients were arbitrarily selected, who were treated at a single institution on Elekta linear accelerators with the Agility MLC. The dataset included 5 oropharyngeal, 2 oral cavity, 1 hypopharynx, 1 supraglottic and 1 of unknown primary (target below nasal region) patients. All patients had bilateral irradiation with a single high dose PTV (lateralised or bilateral). All patients had the brainstem, spinal cord and parotids outlined on the planning CT, and 9 also had the larynx outlined. PTVs were created from the CTVs by applying a 4 mm isotropic expansion, whilst planning risk volumes (PRVs) for the brainstem and spinal cord were created by applying a 5 mm isotropic expansion. The planning CT (pCT) scans, clinical structures and a weekly CBCT (for a total of 5 CBCTs) were imported to RayStation v7 (RaySearch Labs, Sweden). The complete brainstem was visible on all CBCT scans. The CBCTs underwent a shading correction [
      • Marchant T.E.
      • Moore C.J.
      • Rowbottom C.G.
      • Mackay R.I.
      • Williams P.C.
      Shading correction algorithm for improvement of cone-beam CT images in radiotherapy.
      ] to improve image quality and dose calculation accuracy. This algorithm applies a voxel-by-voxel based enhancement of CBCT images using data from the pCT. This also has the effect of improving image quality in addition to subsequent dose calculation accuracy by correcting the Hounsfield units to within 1% of the pCT (from 10 to 20% of the value for the unprocessed CBCT).
      Each patient was re-planned with volumetric arc therapy (VMAT) at 6 MV on the pCT. The patients were replanned owning to the availability of sufficient research time on RayStation and unavailability of the clinical treatment planning system for research. All patients were prescribed to the median dose of 65.1 Gy in 30 fractions over 6 weeks to the high-risk PTV and 54 Gy was prescribed to the elective nodal regions. The planning objectives are summarised in Table 1. The complexity of the plans was not considered, as the complexity of the plans does not change when the plans are recalculated on CBCTs.
      Table 1Planning PTV and OAR constraints.
      VolumeConstraintObjective
      High dose PTVD95%
      Dose to 95% of the volume, 2Dose to 2% of the volume, 3Dose to 1 cc of the volume, 4As low as reasonably practicable.
      >61.85 Gy
      D2%2<68.36 Gy
      Low dose PTVD95%>51.30 Gy
      Spinal cordD1cc3Mandatory: <44 Gy
      Spinal cord PRVD1ccMandatory: <46 Gy
      BrainstemD1ccMandatory: <52 Gy
      Brainstem PRVD1ccMandatory: <54 Gy
      ParotidsAverage doseOptimal: <26 Gy
      LarynxAverage doseALARP4
      1 Dose to 95% of the volume, 2Dose to 2% of the volume, 3Dose to 1 cc of the volume, 4As low as reasonably practicable.

      Contour propagation

      The spinal cord, brainstem, parotids and larynx were propagated onto the CBCTs from the pCT using deformable image registration with five commercial packages: (1) RayStation v7 (RaySearch Labs, Sweden), (2) ADMIRE v1.13 (Elekta, UK), (3) Mirada v1.6 (Mirada Medical Systems, UK), (4) ProSoma v4.1 (Medcom, Germany) and (5) Pinnacle3 v16.0 (Philips, Netherlands). Each package used a different type of deformation algorithm; Pinnacle and ProSoma used demons algorithms (details via personal communications), RayStation used a hybrid algorithm (ANACONDA) [
      • Laboratories R.
      RayStation 7 User Manual.
      ], Mirada a free-form algorithm [

      Kessler M, Pouliot J. White paper: Deformable registration: What to ask when assessing the options. Oxford, UK; 2013.

      ] and ADMIRE a hybrid algorithm [
      • Inc E.
      ABAS: Intra-Patient Deformable Image Registration for Adaptive Radiotherapy – A White Paper.
      ]. For all software packages the contours were propagated using a one-to-all methodology, whereby the contours are propagated directly from the pCT to each CBCT independently. All contours were visually inspected. PRVs were created for the serial organs by applying a 5 mm isotropic expansion. The inferior extent of the spinal cord was cropped to the same extent for all patients for consistency.

      Gold standard reference contours

      Two sets of gold standard contours were created. The first were manually drawn by an experienced clinician contouring each of the CBCTs who had used the planning contours as reference to encourage the consistency (note that the original contours were created by different clinicians).
      The second set of gold standard contours was generated from the propagated contours via the Simultaneous Truth and Performance Level Estimation (STAPLE) [
      • Warfield S.K.
      • Zou K.H.
      • Wells W.M.
      Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation.
      ] method. Whilst not equivalent to multi-operator contouring, it was assumed that the consensus of the different packages would be a suitable representation of the true organ for this study. The STAPLE contours were created in the Computational Environmental for Radiotherapy Research (CERR) [
      • Deasy J.O.
      • Blanco A.I.
      • Clark V.H.
      CERR: A computational environment for radiotherapy research.
      ] (https://github.com/cerr/CERR/).

      Geometric metrics

      Using the functionality built into RayStation, accessed via the scripting interface, the Dice similarity coefficient (DSC) [

      Dice LR. Measures of the Amount of Ecologic Association between Species. Ecology 1945;26:297–302. https://doi.org/doi.org/10.2307/1932409.

      ], centroid separation [
      • Beasley W.J.
      • McWilliam A.
      • Aitkenhead A.
      • Mackay R.I.
      • Rowbottom C.G.
      The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy.
      ], Hausdorff Distance (HD) (also known as maximum distance to agreement) [
      • Huttenlocher D.P.
      • Klanderman G.A.
      • Rucklidge W.J.
      Comparing Images Using the Hausdorff Distance.
      ] and mean distance to agreement (mDTA) [
      • Brock K.K.
      • Mutic S.
      • McNutt T.R.
      • Li H.
      • Kessler M.L.
      Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132.
      ] were calculated between each of the propagated contours and each gold standard contour.
      To avoid selection bias on the distance metrics, we calculated symmetric HD and mDTA by combining the metric values from contour A to contour B and from contour B to contour A. We combined both mDTA measures by taking the average of both values. Similarly, we combined both HD by taking the maximum of both values.
      We assessed the differences in geometric metrics for both gold standards using Wilcoxon sign-rank tests. We report the average of each geometric metric calculated across all CBCTs. Additionally, to allow for individual comparisons, we report the distribution of the geometric metrics using box and whisker plots for each organ and CBCT for both the clinician and STAPLE references.

      Intra-observer consistency

      As image quality and inherent intra-observer variation may affect the resulting contours, we assessed intra-observer consistency of the gold standard drawn by the clinician. For this, the same clinician contoured the spinal cord, brainstem, parotids and larynx in the last CBCT for 3 patients. Geometrical metrics, as described above, were calculated between the two sets of clinician-outlined structures and plotted against the mean geometric metrics between the STAPLE contour and the propagated contours for each package, using error bars to represent the standard deviation.

      Dose volume histogram (DVH) parameters from propagated contours

      To assess the doses delivered to each of the propagated contours, the CBCTs were rigidly registered to the pCT using the clinical registration to determine the isocentre position. All CBCTs had a smaller field-of-view than the pCTs, therefore, using the clinical registrations, in RayStation a bulk density override volume was created to account for missing tissue from the CBCTs using the rigid registration and patient outline from the pCT [
      • Laboratories R.
      RayStation 7 User Manual.
      ]. The dose for each plan was then re-calculated in RayStation on the shader-corrected CBCTs.
      The following DVH parameters were extracted from RayStation for the propagated contours and gold standard contours for each CBCTs: (1) the maximum dose to 1 cc (D1cc) for the brainstem and spinal cord and their PRVs and (2) the average dose for the parotids and the larynx. The parotids were grouped into spared parotids and treated parotids due to their different local dose environments, depending on whether the planning constraint of 26 Gy mean dose was exceeded.
      Scatter plots of the gold standard contours’ DVH parameters and each of the packages were created, for each CBCT. The intraclass correlation (ICC) [

      Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016;15:155–63. https://doi.org/10.1016/j.jcm.2016.02.012.

      ] was used to test the agreement between the DVH parameters resulting from each auto-contouring package and the gold standard contours. The ICC is a measure of reliability between measurements (in this case different packages) reflecting the degree of correlation and agreement [

      Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016;15:155–63. https://doi.org/10.1016/j.jcm.2016.02.012.

      ]. A multiple measurements two-way random-effect model was used. Absolute agreement between packages was considered important. The ICC and the 95% confidence intervals are reported. The ICC was calculated in Matlab using code by Salerian [

      Salarian A. Intraclass correlation coefficient 2020. https://uk.mathworks.com/matlabcentral/fileexchange/22099-intraclass-correlation-coefficient-icc (accessed October 20, 2020).

      ]. According to the interpretation proposed by Koo and Li [

      Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016;15:155–63. https://doi.org/10.1016/j.jcm.2016.02.012.

      ], ICC values<0.50, between 0.50 and 0.75, between 0.75 and 0.90, and >0.90 are indicative of poor, moderate, good, and excellent reliability, respectively.
      Defining clinically significant differences is more challenging. For the purpose of this study, we distinguish serial organs and parallel organs [

      Chang DS, Lasley FD, Das IJ, Mendonca MS, Dynlacht JR. Normal Tissue Radiation Responses. Basic Radiother. Phys. Biol., New York: Springer; 2014, p. 265–75. https://doi.org/10.1007/978-3-319-06841-1_26.

      ]. As serial organs, the D1cc for both the spinal cord and brainstem PRVs were compared against the clinical constraint; requiring<44 Gy for the spinal cord and 54 Gy for the brainstem. For parallel organs, which may have exceeded the clinical constraint on the original plan, we assessed trends over treatment, on the assumption that an increased dose will result in further impaired organ function. The volumes of the treated and spared parotids for the pCT and each CBCT were extracted.

      Results

      Qualitative assessment of propagation

      All contours were successfully propagated from the pCT to the CBCTs via all the specified packages. One patient showed considerable weight loss. Qualitatively, the STAPLE contours were a close match to the propagated contours, as would be expected. The clinician contours were a poorer match to the propagated contours. This is likely due to the very poor quality of the CBCTs, with in some instances (such as for the brainstem), the clinician having to rely on bony anatomy to judge the positions of the organs. The spinal cord also proved to be very challenging to contour, again due to the poor image quality and inconsistencies in pCT contouring (spinal cord vs. spinal canal). Particular regions of disagreement were the superior extents of the parotids, the diameter of the spinal cord volume and the superior aspects of the brainstem. These are regions that particularly depend upon the soft tissue contrast, that was lacking from the CBCTs. Fig. 1 shows an example of the contours for one of the patients for the fifth CBCT.
      Figure thumbnail gr1
      Fig. 1Example contours from the fifth CBCT for one of the patients, showing the spinal cord, brainstem, parotids and larynx. The blue contours are the contours from each of the packages with the STAPLE contour in red. The yellow contours were contoured by the clinician. Colour in online version. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

      Geometric metrics

      Table 2 shows the mean DSC, HD, mDTA and centroid separations for each of the organs at risk over all CBCTs compared against the clinician and STAPLE contours. The data shows a statistically significant difference (p < 0.01 for all) between the STAPLE and clinician structures, with generally poorer results for comparisons against the clinician contours.
      Table 2The DSC, HD, mDTA and centroid separation for each organ across all CBCTs for both the clinician drawn structures and the STAPLE contours. All measures show average (standard deviation) across all patients and CBCTs.
      OrganClinician drawn

      gold standard
      STAPLE

      gold standard
      DSC
      Brainstem0.68 (0.09)0.93 (0.04)
      Spinal cord0.62 (0.14)0.87 (0.04)
      Larynx0.75 (0.10)0.93 (0.04)
      Left parotid0.72 (0.08)0.93 (0.06)
      Right parotid0.76 (0.06)0.92 (0.03)
      mDTA (mm)
      Brainstem2.9 (0.1)0.8 (0.5)
      Spinal cord1.5 (0.5)0.5 (0.2)
      Larynx2.2 (1.1)0.5 (0.3)
      Left parotid2.2 (0.5)0.5 (0.2)
      Right parotid2.0 (0.5)0.5 (0.2)
      HD (mm)
      Brainstem10.8 (3.5)4.4 (2.7)
      Spinal cord7.1 (2.8)4.3 (2.7)
      Larynx10.2 (4.5)3.5 (1.5)
      Left parotid12.9 (4.8)3.5 (1.1)
      Right parotid12.2 (3.9)3.4 (1.1)
      Centroid separation (all in mm)
      Brainstem5.7 (2.9)1.6 (1.3)
      Spinal cord9.7 (5.8)2.8 (2.1)
      Larynx3.2 (2.7)0.9 (1.0)
      Left parotid3.6 (1.6)0.9 (0.6)
      Right parotid3.1 (1.4)0.9 (0.6)
      Fig. 2 shows the mDTA for the spinal cord and brainstem, with the parotids and larynx shown in the Supplementary materials (Fig. S1). The HD, DSC and centroid separation can be found in the Supplementary materials (Figs. S2–S4). The results generally show that the STAPLE contours agree well with the propagated contours (as expected), whilst the clinician drawn contours showing poorer agreement with generally limited overlap of the interquartile ranges of the clinician and the STAPLE comparison. This is in accordance with Table 2.
      Figure thumbnail gr2
      Fig. 2The mDTA for each CBCT and package for the brainstem and spinal cord for both the STAPLE and Clinician contours as the gold standard for ADMIRE (A), Mirada (M), Pinnacle (PN), ProSoma (PS) and RayStation (RS). Lower values are better. Note that the data is grouped by CBCT from left to right. No clear time-trends were found.

      Intra-observer consistency

      The DSC, mDTA, centroid separation and HD are displayed in Supplementary information (Fig. S5) for each of the 3 patients who were re-contoured by the clinician for each of the OARs. With the exception of the larynx, the data does not lie on the identify line implying that the intra-observer variation is larger than the difference between the propagated contours. This is likely due to poor image quality on the CBCTs.

      DVH parameters from propagated contours

      Fig. 3 compared the gold standard and doses estimated from contour propagated for brainstem PRV. The spinal cord (and PRV), brainstem, larynx, spared parotids and treated parotids are shown in the Supplementary materials (Figs. S6–S11). The ICC and confidence interval is shown in Table 3. The agreement between the gold standard contours and the propagated contours is generally good, except for the spinal cord PRV where ICC is between 0.67 and 0.80. The spinal cord itself shows very good agreement. This is potentially due to the dose gradient reducing over the spinal cord compared to its PRV, as illustrated schematically in Fig. 4 for a typical patient. The brainstem and its PRV ICCs are excellent (see Fig. 3), with very narrow confidence intervals. The treated parotids are also good.
      Figure thumbnail gr3
      Fig. 3Scatter plots for the D1cc to the brainstem PRV obtained for the gold standard contours (clinical or STAPLE) vs propagated contours for each of the packages.
      Table 3The ICC and confidence intervals for the DVH parameters for each organ for the doctor contours and the STAPLE contours. Asterisks indicate categories according to Koo and Li: none for excellent reliability (ICC>=0.9), * for good reliability (0.75 < ICC < 0.9) and ** for moderate reliability (0.5 < ICC < 0.75).
      CBCT 1CBCT 2CBCT 3CBCT 4CBCT 5
      Clinician’s contours
      Spinal cord0.98 (0.94–0.99)0.97 (0.94–0.99)0.97 (0.92–0.99)0.97 (0.94–0.99)0.91 (0.80–0.97)
      Spinal cord PRV0.80 (0.61–0.93)*0.71 (0.48–0.90)**0.73 (0.50–0.91)**0.78 (0.56–0.93)*0.66 (0.41–0.88)**
      Brainstem0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.99–1.00)
      Brainstem PRV0.99 (0.99–1.00)1.00 (0.99–1.00)1.00 (0.99–1.00)0.99 (0.99–1.00)1.00 (0.99–1.00)
      Spared parotids0.91 (0.78–0.98)0.90 (0.74–0.98)0.84 (0.64–0.97)*0.83 (0.62–0.96)*0.78 (0.54–0.95)*
      Treated parotids0.97 (0.93–0.99)0.98 (0.96–0.99)0.98 (0.96–0.99)0.98 (0.95–0.99)0.96 (0.92–0.99)
      Larynx0.93 (0.83–0.98)0.95 (0.88–0.98)0.95 (0.88–0.98)0.95 (0.88–0.98)0.92 (0.83–0.98)
      STAPLE contours
      Spinal cord0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.99–1.00)
      Spinal cord PRV0.83 (0.65–0.95)*0.75 (0.52–0.92)*0.82 (0.64–0.94)*0.84 (0.68–0.95)*0.88 (0.73–0.96)*
      Brainstem1.00 (0.99–1.00)1.00 (0.99–1.00)1.00 (0.99–1.00)1.00 (0.99–1.00)1.00 (0.99–1.00)
      Brainstem PRV1.00 (1.00–1.00)1.00 (1.00–1.00)1.00 (1.00–1.00)1.00 (1.00–1.00)1.00 (1.00–1.00)
      Spared parotids0.98 (0.94–1.00)0.98 (0.95–1.00)0.98 (0.95–1.00)0.95 (0.87–0.99)0.94 (0.83–0.99)
      Treated parotids0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.98–1.00)0.99 (0.97–1.00)
      Larynx0.99 (0.97–1.00)0.99 (0.97–1.00)0.99 (0.96–1.00)0.99 (0.96–1.00)0.98 (0.95–0.99)
      Figure thumbnail gr4
      Fig. 4Schematic example of a dose gradient through the spinal cord (yellow region) showing the dose (left axes) and its numerical derivative/gradient (right axes), demonstrating the sharp gradient over the PRV before levelling off in the spinal cord itself. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
      The ICC of the spared parotids degrades over the course of the 5 CBCTs. For the clinician’s contours the ICC starts at 0.91 (indicating excellent agreement) progressively dropping until 0.78 (indicating moderate reliability). A similar trend is observed for the STAPLE contours, although in this case the minimum ICC is 0.93. No time trend is demonstrated in the geometrical measures, but the change in ICC could be indicated by either weight loss or changing volume of the parotids (see Supplementary information, Fig. S12).
      The spinal cord PRV has the worst ICC for both the clinician and the STAPLE. The clinician contours for CBCT 5 have the lowest ICC at 0.67 (or lower confidence interval 0.42). There is a general trend for the clincian’s contours to show a worsening of reliability over time for the spinal cord PRV, but this is not reflected in the STAPLE contours. Fig. S8 demonstrates this issue, where compared to the other OARS the spread of the data around the identity line does support the poorer ICC.
      Comparing the STAPLE contours against the clinician’s contours, the STAPLE contours consistently outperform the clinician contours, but the differences are rarely significant. This suggests that any of the packages can reliably be used to estimate the relevant DVH parameters of the organs at risk.
      In terms of the clinical impact, the spinal cord or brainstem never exceeded a clinical tolerance regardless of method. For the spinal cord PRV the clinical tolerance was exceeded for different packages, the STAPLE or the clinician-drawn 14 times over 3 different patients and for the brainstem PRV the clinical tolerance was exceeded 5 times, all on the same patient for different packages, STAPLE and clinician. Note, however, that the contours that failed were planned close to tolerance. No particular method (or the clinician) failed a tolerance more than any other. In terms of increase in dose compared to clinical tolerance, the spinal cord PRV was between 0.07 and 2.50 Gy over tolerance and the brainstem PRV was 0.29–1.99 Gy over tolerance.
      Any trends over treatment were assessed for the parotids (split between treated and spared parotids) and for the larynx. The mean parotid dose for each CBCT showed a gradual decrease in spared parotid mean dose for most packages whilst the treated parotids were inconsistent. No trends were statistically significant. The volumes of the parotids showed a decrease over treatment (see Supplementary materials, Fig. S12 and Table S1). The mean dose to the larynx showed a very weak non-statistically significant difference over treatment.

      Discussion

      To the best of the authors knowledge, this work is the first study directly reviewing the clinical dose implications of using different non-rigid registration algorithms for head and neck cancer OARs compared to the geometric accuracy, and the first reviewing and comparing five widely available commercial packages [
      • Beasley W.J.
      • McWilliam A.
      • Aitkenhead A.
      • Mackay R.I.
      • Rowbottom C.G.
      The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy.
      ,
      • Hvid C.A.
      • Elstrøm U.V.
      • Jensen K.
      • Grau C.
      Cone-beam computed tomography (CBCT) for adaptive image guided head and neck radiation therapy.
      ,
      • Ayyalusamy A.
      • Vellaiyan S.
      • Shanmugam S.
      • Ilamurugu A.
      • Gandhi A.
      • Shanmugam T.
      • et al.
      Feasibility of offline head & neck adaptive radiotherapy using deformed planning CT electron density mapping on weekly cone beam computed tomography.
      ] with most previous studies focussing on only assessing the geometric accuracy of one such packages [
      • Woerner A.J.
      • Choi M.
      • Harkenrider M.M.
      • Roeske J.C.
      • Surucu M.
      Evaluation of Deformable Image Registration-Based Contour Propagation From Planning CT to Cone-Beam CT.
      ,
      • Thor M.
      • Petersen J.B.B.
      • Bentzen L.
      • Høyer M.
      • Paul M.L.
      Deformable image registration for contour propagation from CT to cone-beam CT scans in radiotherapy of prostate cancer.
      ,
      • Cole A.J.
      • Veiga C.
      • Johnson U.
      • D’Souza D.
      • Lalli N.K.
      • McClelland J.R.
      Toward adaptive radiotherapy for lung patients: Feasibility study on deforming planning CT to CBCT to assess the impact of anatomical changes on dosimetry.
      ,
      • Lou J.
      • Huang P.u.
      • Ma C.
      • Zheng Y.
      • Chen J.
      • Liang Y.
      • et al.
      Parotid gland radiation dose-xerostomia relationships based on actual delivered dose for nasopharyngeal carcinoma.
      ,
      • Thörnqvist S.
      • Petersen J.B.B.B.
      • Høyer M.
      • Bentzen L.N.
      • Paul M.L.
      Propagation of target and organ at risk contours in radiotherapy of prostate cancer using deformable image registration.
      ,
      • Riegel A.C.
      • Antone J.G.
      • Zhang H.
      • Jain P.
      • Raince J.
      • Rea A.
      • et al.
      Deformable image registration and interobserver variation in contour propagation for radiation therapy planning.
      ]. There has been some limited work in the related field of automatically, deep learning, generated contours for prostate cancer [
      • Kawula M.
      • Purice D.
      • Li M.
      • Vivar G.
      • Ahmadi S.-A.
      • Parodi K.
      • et al.
      Dosimetric impact of deep learning-based CT auto-segmentation on radiation therapy treatment planning for prostate cancer.
      ,
      • Duan J.
      • Bernard M.
      • Downes L.
      • Willows B.
      • Feng X.
      • Mourad W.F.
      • et al.
      Evaluating the clinical acceptability of deep learning contours of prostate and organs-at-risk in an automated prostate treatment planning process.
      ,
      • Mandal S.
      • Kale S.N.
      • Kinhikar R.A.
      A mathematical and dosimetric approach to validate auto-contouring by Varian Smart segmentation for prostate cancer patients.
      ]. These have shown similar conclusions to here; with the generated contours producing acceptable dose distributions although these studies have not assessed the reliability of propagated contours. We demonstrated that large differences in geometric metrics rarely have a clinical or statistically significant impact on DVH parameters.

      Geometrical metrics

      Previous authors have generally attempted to use multiple oncologists to gain good quality gold standard contours [
      • Beasley W.J.
      • McWilliam A.
      • Aitkenhead A.
      • Mackay R.I.
      • Rowbottom C.G.
      The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy.
      ]. In this work, delineations on the CBCTs from a single experienced clinician were used as one of the gold standards. We assessed the consistency of the observer by recontouring several CBCTs (Supplementary information, Fig. S5), showing that due to poor image quality the manual contouring uncertainty is high. The use of the STAPLE contours, created from the deformed contours, attempted to compensate for these inconsistencies, with the differences to the STAPLE being representative of the variation inherent in the different propagated contours. It should be noted that the clinician creating the CBCT contours did not originally contour the pCTs. The clinician had access to the pCT contours for reference, and noted a lack of consistency within patients, especially for the larynx and spinal cord.
      As expected, when comparing the propagated contours to the gold standard, all metrics based on the clinician-drawn contours show much greater errors than metrics based on STAPLE, regardless of algorithm (Table 2). This behaviour is consistent with the large observer variation, which is present in the gold standard, but not in the propagated contours. No particular algorithm significantly differed when evaluated against the clinician. Considering each metric:
      • Considering the average mDTAs for the clinician-based comparisons, they are within the magnitude of the slice thickness of the pCT (3 mm) with relatively small standard deviations, consistent with the recommendations of TG-132 [
        • Brock K.K.
        • Mutic S.
        • McNutt T.R.
        • Li H.
        • Kessler M.L.
        Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132.
        ]. The STAPLE is consistently good with regards to mDTA.
      • TG-132 suggest DSC for a good registration should be within the contouring uncertainty of the structure at ∼ 0.8–0.9 [
        • Brock K.K.
        • Mutic S.
        • McNutt T.R.
        • Li H.
        • Kessler M.L.
        Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132.
        ], although other authors have used lower limits (e.g. Hvid et al [
        • Hvid C.A.
        • Elstrøm U.V.
        • Jensen K.
        • Alber M.
        • Grau C.
        Accuracy of software-assisted contour propagation from planning CT to cone beam CT in head and neck radiotherapy.
        ] at 0.7). Again, results for STAPLE-based comparisons are better than for the clinician-based comparison: while all contours achieve this limit for the STAPLE-based comparisons, the clinician contours did not achieve the TG-132 limits.
      • The HD, being the maximum difference to agreement, is sensitive to local differences in structures, potentially on a single location. The HD is always large for the clinician-based comparisons, with smaller values for the STAPLE, although the HDs reported for the STAPLE here are still large compared to Hvid et al [
        • Hvid C.A.
        • Elstrøm U.V.
        • Jensen K.
        • Alber M.
        • Grau C.
        Accuracy of software-assisted contour propagation from planning CT to cone beam CT in head and neck radiotherapy.
        ] using revised propagated contours, with 3.4–4.4 mm averages for the STAPLE.
      • The centroid separation is generally small for the propagated contours for the STAPLE, although it is larger for the brainstem and spinal cord – a similar although much magnified separation is seen for the clinician for these organs. The variation on the spinal cord is likely due to the structure being a long tubular structure so small differences in diameter can affect the centroid position, despite being trimmed to the same length.
      For the spinal cord, parotids and larynx the geometrical metrics are broadly similar regardless of the used package, considering the overlapping inter-quartile ranges, for the clinician and STAPLE contours. For the brainstem, it is clear that ADMIRE is creating contours that are different (considering the interquartile range) to the other packages, using STAPLE as gold standard. This effect is shown in the DSC, mDTA, HD and centroid separations. This is most evident considering the HD; the brainstem HD with ADMIRE is consistently larger than the HD of the other packages: approximately 9 mm for ADMIRE versus 2–4 mm for the other packages. This behaviour is not seen in the clinician brainstem comparisons, where no algorithm outperforms another.
      The spinal cord within the spinal canal also was difficult to identify due to noise on the CBCT, particularly in the inferior aspects of the organ. A possible alternative for future work could mean propagating the spinal canal, as its bony boundaries is likely to make a more robust contouring target when image quality is impaired. The brainstem suffered from poor soft tissue contrast, with the superior extents difficult to identify on the CBCTs – these being the areas of typical disagreement from the qualitative review of the contours.

      Dosimetric comparisons

      Considering the generally poor geometric metrics, dosimetric differences were quite small, as shown in Fig. 3 and Table 3, although the clinician was slightly poorer, there was not a significant difference between using the STAPLE and the clinician considering the ICC. This therefore implies that regardless of package (or gold standard used), the propagated contours allow a reliable predictor of dose for nearly all organs on CBCTs.
      An exception to this is the spinal cord PRV, which was deemed of moderate reliability (Table 3), likely due to contouring uncertainty. This was both for the clinician-based comparisons as well as for STAPLE-based. The spinal cord itself was generally more reliable. This is likely due to the sharp dose gradient present, as illustrated in Fig. 4. In this case, the greatest dose gradient is present at the edge of the PRV, but has plateaued before reaching the cord itself. This re-enforces the concept that the reliability of the spinal cord structure and its PRV could have been improved by contouring the spinal canal. However, new margins for the PRV would need to be investigated to avoid any negative impact on PTV coverage.
      The clinical significance of these dosimetric uncertainties is likely to be minimal; the cord planning tolerance is conservative at 46 Gy in 30 fractions to the PRV. As a serial organ, the primary concern at planning is the maximum dose (D1cc here), with attempts to drive dose to the cord down as low as possible being unnecessary and likely to impede PTV coverage. In this study, we observed a small dose excess for the cases where tolerance failed (7–250 cGy, against a planning tolerance of 44 Gy) however, it may not have triggered a replan – with the decision based on professional judgement at the time.
      Large geometric differences were observed for the brainstem. However, the dose estimates are reliable regardless of package used and the dose rarely deviates from identity line (Fig. 3). This can be explained by two main factors: 1) at the base of the brainstem the bony anatomy is a good surrogate for the organ, and 2) the maximum dose falls in this reliable region as no nasopharynx or superior tumours were included in the dataset. Therefore, there would be no benefit in correcting contours for these patients especially considering the difficulty identifying the brainstem away from bony anatomy on the CBCTs. Occasionally brainstem planning tolerance on PRV was exceeded, but only by small amounts (29–199 cGy). The brainstem tolerance was planned in a less conservative manner than for the spinal cord (54 Gy D1cc) but again the decision to replan would be a clinician decision. Note that this does not hold for superior treatments, like nasopharyngeal, where the optical structures would also need to be considered.
      The parotid dose measure showed a slight decrease in reliability over time – possibly due to expected volume changes over treatment [
      • Sreejeev A.T.
      • Joseph D.
      • Krishnan A.S.
      • Ahuja R.
      • Sikdar D.
      • Raut S.
      • et al.
      Serial assessment of parotid volume changes during radical chemoradiation of locally advanced head and neck cancer: Its implications in practice of adaptive radiotherapy.
      ], although this did not correlate with a large dose change. Little work has been done on dose variation to the parotids, with most work focussing on volume, but dose increases have been reported [
      • Sreejeev A.T.
      • Joseph D.
      • Krishnan A.S.
      • Ahuja R.
      • Sikdar D.
      • Raut S.
      • et al.
      Serial assessment of parotid volume changes during radical chemoradiation of locally advanced head and neck cancer: Its implications in practice of adaptive radiotherapy.
      ] but at larger magnitudes than shown here. Work is ongoing though in characterising the dosimetric impact on the parotids function [
      • Kabarriti R.
      • Brodin P.
      • Ahmed S.
      • Tome W.A.
      • Guha C.
      • Kalnicki S.
      • et al.
      Mid-treatment assessment of dose to parotid gland stem cell region and change in parotid gland volume predicts for long-term patient-reported xerostomia.
      ,
      • Wilkie J.R.
      • Mierzwa M.L.
      • Casper K.A.
      • Mayo C.S.
      • Schipper M.J.
      • Eisbruch A.
      • et al.
      Predicting late radiation-induced xerostomia with parotid gland PET biomarkers and dose metrics.
      ].
      A limitation of this work is that it is retrospective. As the patients were treated with a different plan than the ones considered here, any dose-time relationship needs to be taken with caution. The dose changes shown here would almost certainly not cause a replan in an offline adaption protocol, but in a daily adaption protocol, would result in changes to plan. The data suggests that propagated contours are reliable to use for day-to-day replan without correction. However, a study where plans are re-optimised using the propagated contours is still required to form a final conclusion.
      Whilst there is some uncertainty in the quality of the gold standard contours, due to image quality, this work has demonstrated that propagated contours from a variety of packages are suitable for estimating dose to the OARs, and identified which organs may benefit from review. In particular, the spinal cord has shown most uncertainty in predicted dose with different contouring packages and therefore at times when the spinal cord is close to a clinical tolerance closer scrutiny of this organ may be appropriate. The choice of contouring volume (for example, spinal cord or canal) may influence the result, and therefore, especially where image quality may be sub-optimal (CBCT), choosing a bony target such as the spinal canal may be more consistent. The data for the brainstem also supports this theory.
      The clinical significance depends on the adaption protocol followed [
      • Green O.L.
      • Henke L.E.
      • Hugo G.D.
      Practical Clinical Workflows for Online and Offline Adaptive Radiation Therapy.
      ]. Offline adaption, whether ad-hoc, based on changes during treatment, or as part of a planned protocol, can present significant workflow challenges. The first issue is generally to flag when a replan is needed, followed by creating the replan itself. There are usually time pressures as the patient’s treatment should not be paused whilst creating the new plan, necessitating the quick (and often unplanned) replan. Traditionally this would result from onboard imaging indicating the potential for a replan, followed by a CT scan which is then contoured and reviewed by the Oncologist before deciding to replan, at which point the replan is created. This is a very time-consuming process, which has to be fit around already busy workloads, especially as the OARs as well as the CTVs need to be redrawn. The work presented here demonstrates that propagated contours, even without correction, just generated onto the shading corrected-CBCTs, can be used reliably to estimate the dose to the organs at risk with minimal manual correction needed, so may result in efficiencies in the process, potentially even avoiding the need for the repeat planning CT until after reviewing the dosimetry on the CBCT.
      There are benefits to the clinical workflow achievable with this work. It has been demonstrated that the geometric accuracy of the propagated contours is small compared to the dose delivered to these contours. Therefore, if a patient presents for potential adaption, it should be possible to propagate the contours to the repeat CBCT before recontouring the target itself – thus reducing the contouring need and creating efficiencies in the workflow. Alternatively, propagated contours could be used to guide the replans for patients, but this needs further investigation.
      Online adaption protocols are becoming more prevalent, with the advent of the MR-LINAC [
      • Kerkmeijer L.G.W.
      • Fuller C.D.
      • Verkooijen H.M.
      • Verheij M.
      • Choudhury A.
      • Harrington K.J.
      • et al.
      The MRI-Linear Accelerator Consortium: Evidence-Based Clinical Introduction of an Innovation in Radiation Oncology Connecting Researchers, Methodology, Data Collection, Quality Assurance, and Technical Development.
      ] and also the Varian (Palo Alto, CA) Ethos system [
      • Archambault Y.
      • Boylan C.
      • Bullock D.
      • Morgas T.
      • Peltola J.
      • Ruokokoski E.
      • et al.
      Making on-Line Adaptive Radiotherapy Possible Using Artificial Intelligence and Machine Learning for Efficient Daily Re-Planning.
      ]. This work is off less relevance to the MR-LINAC, where the superior image quality of MR would be expected to benefit both manual and auto-contouring, but the Varian Ethos system uses CBCT. In this work we used CBCT images corrected with a shader correction [
      • Marchant T.E.
      • Moore C.J.
      • Rowbottom C.G.
      • Mackay R.I.
      • Williams P.C.
      Shading correction algorithm for improvement of cone-beam CT images in radiotherapy.
      ] on Elekta CBCTs whilst the Ethos system uses the advanced iterative iCBCT algorithm [
      • Moazzezi M.
      • Rose B.
      • Kisling K.
      • Moore K.L.
      • Ray X.
      Prospects for daily online adaptive radiotherapy via ethos for prostate cancer patients without nodal involvement using unedited CBCT auto-segmentation.
      ,
      • Maslowski A.
      • Wang A.
      • Sun M.
      • Wareing T.
      • Davis I.
      • Star-Lack J.
      Acuros CTS: A fast, linear Boltzmann transport equation solver for computed tomography scatter – Part I: Core algorithms and validation.
      ,
      • Wang A.
      • Maslowski A.
      • Messmer P.
      • Lehmann M.
      • Strzelecki A.
      • Yu E.
      • et al.
      Acuros CTS: A fast, linear Boltzmann transport equation solver for computed tomography scatter - Part II: System modeling, scatter correction, and optimization.
      ] to both improve image quality and accuracy of the HU/dose calculation. How the image quality compares between the shading corrected CBCTs and iCBCT on the Ethos requires further investigation, but it is likely that the work undertaken here is relevant to the Ethos workflow where auto-contours are used for the organs at risk. This would allow the clinician to target their corrections to the organs most likely to impact on the dosimetry to the organs, and this work would imply this is primarily for the serial organs when close to planning tolerance.

      Conclusion

      We showed in this study that for most organs, despite for some poor geometric agreement, the DVH parameters of propagated contours gave a reliable estimate of the organ dose, independent of which of the five commercially available packages were used. Therefore, despite the uncertainty in geometric accuracy, it can be concluded that unedited propagated contours are useful for plan evaluation in an adaptive radiotherapy pathway.

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Acknowledgements

      This work was supported by Cancer Research UK via funding to the Cancer Research Manchester Centre (C147/A25254) and the NIHR Manchester Biomedical Research Centre.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Strojan P.
        • Hutcheson K.A.
        • Eisbruch A.
        • Beitler J.J.
        • Langendijk J.A.
        • Lee A.W.M.
        • et al.
        Treatment of late sequelae after radiotherapy for head and neck cancer.
        Cancer Treat Rev. 2017; 59: 79-92
        • Nutting C.M.
        • Morden J.P.
        • Harrington K.J.
        • Urbano T.G.
        • Bhide S.A.
        • Clark C.
        • et al.
        Parotid-sparing intensity modulated versus conventional radiotherapy in head and neck cancer (PARSPORT): a phase 3 multicentre randomised controlled trial.
        Lancet Oncol. 2011; 12: 127-136
        • Sonke J.J.
        • Aznar M.
        • Rasch C.
        Adaptive Radiotherapy for Anatomical Changes.
        Semin Radiat Oncol. 2019; 29: 245-257https://doi.org/10.1016/j.semradonc.2019.02.007
        • Bhide S.A.
        • Davies M.
        • Burke K.
        • McNair H.A.
        • Hansen V.
        • Barbachano Y.
        • et al.
        Weekly Volume and Dosimetric Changes During Chemoradiotherapy With Intensity-Modulated Radiation Therapy for Head and Neck Cancer: A Prospective Observational Study.
        Int J Radiat Oncol Biol Phys. 2010; 76: 1360-1368
        • Castadot P.
        • Lee J.A.
        • Geets X.
        • Grégoire V.
        Adaptive Radiotherapy of Head and Neck Cancer.
        Semin Radiat Oncol. 2010; 20: 84-93https://doi.org/10.1016/j.semradonc.2009.11.002
        • Grégoire V.
        • Jeraj R.
        • Lee J.A.
        • O’Sullivan B.
        Radiotherapy for head and neck tumours in 2012 and beyond: Conformal, tailored, and adaptive?.
        Lancet Oncol. 2012; 13: e292-e300https://doi.org/10.1016/S1470-2045(12)70237-1
        • Vásquez Osorio E.M.
        • Hoogeman M.S.
        • Al-Mamgani A.
        • Teguh D.N.
        • Levendag P.C.
        • Heijmen B.J.M.
        Local Anatomic Changes in Parotid and Submandibular Glands During Radiotherapy for Oropharynx Cancer and Correlation With Dose, Studied in Detail With Nonrigid Registration.
        Int J Radiat Oncol Biol Phys. 2008; 70: 875-882https://doi.org/10.1016/j.ijrobp.2007.10.063
        • Castadot P.
        • Geets X.
        • Lee J.A.
        • Grégoire V.
        Adaptive functional image-guided IMRT in pharyngo-laryngeal squamous cell carcinoma: Is the gain in dose distribution worth the effort?.
        Radiother Oncol. 2011; 101: 343-350https://doi.org/10.1016/j.radonc.2011.06.011
        • Castadot P.
        • Geets X.
        • Lee J.A.
        • Christian N.
        • Grégoire V.
        Assessment by a deformable registration method of the volumetric and positional changes of target volumes and organs at risk in pharyngo-laryngeal tumors treated with concomitant chemo-radiation.
        Radiother Oncol. 2010; 95: 209-217https://doi.org/10.1016/j.radonc.2010.03.007
        • Hansen E.K.
        • Bucci M.K.
        • Quivey J.M.
        • Weinberg V.
        • Xia P.
        Repeat CT imaging and replanning during the course of IMRT for head-and-neck cancer.
        Int J Radiat Oncol Biol Phys. 2006; 64: 355-362https://doi.org/10.1016/j.ijrobp.2005.07.957
        • Chen C.
        • Fei Z.
        • Lisha C.
        • Bai P.
        • Lin X.
        • Pan J.
        Will weight loss cause significant dosimetric changes of target volumes and organs at risk in nasopharyngeal carcinoma treated with intensity-modulated radiation therapy?.
        Med Dosim. 2014; 39: 34-37https://doi.org/10.1016/j.meddos.2013.09.002
        • Vickress J.R.
        • Battista J.
        • Barnett R.
        • Yartsev S.
        Online daily assessment of dose change in head and neck radiotherapy without dose-recalculation.
        J Appl Clin Med Phys. 2018; 19: 659-665https://doi.org/10.1002/acm2.12432
        • Moazzezi M.
        • Rose B.
        • Kisling K.
        • Moore K.L.
        • Ray X.
        Prospects for daily online adaptive radiotherapy via ethos for prostate cancer patients without nodal involvement using unedited CBCT auto-segmentation.
        J Appl Clin Med Phys. 2021; 22: 82-93https://doi.org/10.1002/acm2.13399
        • Klüter S.
        Technical design and concept of a 0.35 T MR-Linac.
        Clin TranslRadiat Oncol. 2019; 18: 98-101https://doi.org/10.1016/j.ctro.2019.04.007
        • Tsuji S.Y.
        • Hwang A.
        • Weinberg V.
        • Yom S.S.
        • Quivey J.M.
        • Xia P.
        Dosimetric Evaluation of Automatic Segmentation for Adaptive IMRT for Head-and-Neck Cancer.
        Int J Radiat Oncol Biol Phys. 2010; 77: 707-714https://doi.org/10.1016/j.ijrobp.2009.06.012
        • Paganelli C.
        • Meschini G.
        • Molinelli S.
        • Riboldi M.
        • Baroni G.
        Patient-specific validation of deformable image registration in radiation therapy: Overview and caveats.
        Med Phys. 2018; 45 (e908–22)https://doi.org/10.1002/mp.13162
        • Teguh D.N.
        • Levendag P.C.
        • Voet P.W.J.
        • Al-Mamgani A.
        • Han X.
        • Wolf T.K.
        • et al.
        Clinical validation of atlas-based auto-segmentation of multiple target volumes and normal tissue (swallowing/mastication) structures in the head and neck.
        Int J Radiat Oncol Biol Phys. 2011; 81: 950-957
        • Beasley W.J.
        • McWilliam A.
        • Aitkenhead A.
        • Mackay R.I.
        • Rowbottom C.G.
        The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy.
        J Appl Clin Med Phys. 2016; 17: 41-49https://doi.org/10.1120/jacmp.v17i2.5889
        • Hvid C.A.
        • Elstrøm U.V.
        • Jensen K.
        • Alber M.
        • Grau C.
        Accuracy of software-assisted contour propagation from planning CT to cone beam CT in head and neck radiotherapy.
        Acta Oncol (Madr). 2016; 55: 1324-1330https://doi.org/10.1080/0284186X.2016.1185149
        • Eiland R.B.
        • Maare C.
        • SjöStröm D.
        • Samsøe E.
        • Behrens C.F.
        Dosimetric and geometric evaluation of the use of deformable image registration in adaptive intensity-modulated radiotherapy for head-and-neck cancer.
        J Radiat Res. 2014; 55: 1002-1008https://doi.org/10.1093/jrr/rru044
        • Marchant T.E.
        • Moore C.J.
        • Rowbottom C.G.
        • Mackay R.I.
        • Williams P.C.
        Shading correction algorithm for improvement of cone-beam CT images in radiotherapy.
        Phys Med Biol. 2008; 53: 5719-5733https://doi.org/10.1088/0031-9155/53/20/010
        • Laboratories R.
        RayStation 7 User Manual.
        Stockholm, Sweden2017
      1. Kessler M, Pouliot J. White paper: Deformable registration: What to ask when assessing the options. Oxford, UK; 2013.

        • Inc E.
        ABAS: Intra-Patient Deformable Image Registration for Adaptive Radiotherapy – A White Paper.
        Crawley, UK2018
        • Warfield S.K.
        • Zou K.H.
        • Wells W.M.
        Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation.
        IEEE Trans Med Imaging. 2004; 23: 903-921https://doi.org/10.1109/TMI.2004.828354
        • Deasy J.O.
        • Blanco A.I.
        • Clark V.H.
        CERR: A computational environment for radiotherapy research.
        Med Phys. 2003; 30: 979-985https://doi.org/10.1118/1.1568978
      2. Dice LR. Measures of the Amount of Ecologic Association between Species. Ecology 1945;26:297–302. https://doi.org/doi.org/10.2307/1932409.

        • Huttenlocher D.P.
        • Klanderman G.A.
        • Rucklidge W.J.
        Comparing Images Using the Hausdorff Distance.
        IEEE Trans Pattern Anal Mach Intell. 1993; 15: 850-863https://doi.org/10.1109/34.232073
        • Brock K.K.
        • Mutic S.
        • McNutt T.R.
        • Li H.
        • Kessler M.L.
        Use of image registration and fusion algorithms and techniques in radiotherapy: Report of the AAPM Radiation Therapy Committee Task Group No. 132.
        Med Phys. 2017; 44: e43-e76
      3. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016;15:155–63. https://doi.org/10.1016/j.jcm.2016.02.012.

      4. Salarian A. Intraclass correlation coefficient 2020. https://uk.mathworks.com/matlabcentral/fileexchange/22099-intraclass-correlation-coefficient-icc (accessed October 20, 2020).

      5. Chang DS, Lasley FD, Das IJ, Mendonca MS, Dynlacht JR. Normal Tissue Radiation Responses. Basic Radiother. Phys. Biol., New York: Springer; 2014, p. 265–75. https://doi.org/10.1007/978-3-319-06841-1_26.

        • Hvid C.A.
        • Elstrøm U.V.
        • Jensen K.
        • Grau C.
        Cone-beam computed tomography (CBCT) for adaptive image guided head and neck radiation therapy.
        Acta Oncol (Madr). 2018; 57: 552-556https://doi.org/10.1080/0284186X.2017.1398414
        • Ayyalusamy A.
        • Vellaiyan S.
        • Shanmugam S.
        • Ilamurugu A.
        • Gandhi A.
        • Shanmugam T.
        • et al.
        Feasibility of offline head & neck adaptive radiotherapy using deformed planning CT electron density mapping on weekly cone beam computed tomography.
        Br J Radiol. 2017; 90: 20160420https://doi.org/10.1259/bjr.20160420
        • Woerner A.J.
        • Choi M.
        • Harkenrider M.M.
        • Roeske J.C.
        • Surucu M.
        Evaluation of Deformable Image Registration-Based Contour Propagation From Planning CT to Cone-Beam CT.
        Technol Cancer Res Treat. 2017; 16: 801-810https://doi.org/10.1177/1533034617697242
        • Thor M.
        • Petersen J.B.B.
        • Bentzen L.
        • Høyer M.
        • Paul M.L.
        Deformable image registration for contour propagation from CT to cone-beam CT scans in radiotherapy of prostate cancer.
        Acta Oncol (Madr). 2011; 50: 918-925https://doi.org/10.3109/0284186X.2011.577806
        • Cole A.J.
        • Veiga C.
        • Johnson U.
        • D’Souza D.
        • Lalli N.K.
        • McClelland J.R.
        Toward adaptive radiotherapy for lung patients: Feasibility study on deforming planning CT to CBCT to assess the impact of anatomical changes on dosimetry.
        Phys Med Biol. 2018; 63155014https://doi.org/10.1088/1361-6560/aada96
        • Lou J.
        • Huang P.u.
        • Ma C.
        • Zheng Y.
        • Chen J.
        • Liang Y.
        • et al.
        Parotid gland radiation dose-xerostomia relationships based on actual delivered dose for nasopharyngeal carcinoma.
        J Appl Clin Med Phys. 2018; 19: 251-260
        • Thörnqvist S.
        • Petersen J.B.B.B.
        • Høyer M.
        • Bentzen L.N.
        • Paul M.L.
        Propagation of target and organ at risk contours in radiotherapy of prostate cancer using deformable image registration.
        Acta Oncol (Madr). 2010; 49: 1023-1032https://doi.org/10.3109/0284186X.2010.503662
        • Riegel A.C.
        • Antone J.G.
        • Zhang H.
        • Jain P.
        • Raince J.
        • Rea A.
        • et al.
        Deformable image registration and interobserver variation in contour propagation for radiation therapy planning.
        J Appl Clin Med Phys. 2016; 17: 347-357
        • Kawula M.
        • Purice D.
        • Li M.
        • Vivar G.
        • Ahmadi S.-A.
        • Parodi K.
        • et al.
        Dosimetric impact of deep learning-based CT auto-segmentation on radiation therapy treatment planning for prostate cancer.
        Radiat Oncol. 2022; 17https://doi.org/10.1186/s13014-022-01985-9
        • Duan J.
        • Bernard M.
        • Downes L.
        • Willows B.
        • Feng X.
        • Mourad W.F.
        • et al.
        Evaluating the clinical acceptability of deep learning contours of prostate and organs-at-risk in an automated prostate treatment planning process.
        Med Phys. 2022; 49: 2570-2581
        • Mandal S.
        • Kale S.N.
        • Kinhikar R.A.
        A mathematical and dosimetric approach to validate auto-contouring by Varian Smart segmentation for prostate cancer patients.
        Precis Radiat Oncol. 2022; 6: 46-58
        • Sreejeev A.T.
        • Joseph D.
        • Krishnan A.S.
        • Ahuja R.
        • Sikdar D.
        • Raut S.
        • et al.
        Serial assessment of parotid volume changes during radical chemoradiation of locally advanced head and neck cancer: Its implications in practice of adaptive radiotherapy.
        Ann Oncol. 2020; 31: S1351https://doi.org/10.1016/j.annonc.2020.10.272
        • Kabarriti R.
        • Brodin P.
        • Ahmed S.
        • Tome W.A.
        • Guha C.
        • Kalnicki S.
        • et al.
        Mid-treatment assessment of dose to parotid gland stem cell region and change in parotid gland volume predicts for long-term patient-reported xerostomia.
        Int J Radiat Oncol. 2020; 106: 1203
        • Wilkie J.R.
        • Mierzwa M.L.
        • Casper K.A.
        • Mayo C.S.
        • Schipper M.J.
        • Eisbruch A.
        • et al.
        Predicting late radiation-induced xerostomia with parotid gland PET biomarkers and dose metrics.
        Radiother Oncol. 2020; 148: 30-37
        • Green O.L.
        • Henke L.E.
        • Hugo G.D.
        Practical Clinical Workflows for Online and Offline Adaptive Radiation Therapy.
        Semin Radiat Oncol. 2019; 29: 219-227https://doi.org/10.1016/j.semradonc.2019.02.004
        • Kerkmeijer L.G.W.
        • Fuller C.D.
        • Verkooijen H.M.
        • Verheij M.
        • Choudhury A.
        • Harrington K.J.
        • et al.
        The MRI-Linear Accelerator Consortium: Evidence-Based Clinical Introduction of an Innovation in Radiation Oncology Connecting Researchers, Methodology, Data Collection, Quality Assurance, and Technical Development.
        Front Oncol. 2016; 6https://doi.org/10.3389/fonc.2016.00215
        • Archambault Y.
        • Boylan C.
        • Bullock D.
        • Morgas T.
        • Peltola J.
        • Ruokokoski E.
        • et al.
        Making on-Line Adaptive Radiotherapy Possible Using Artificial Intelligence and Machine Learning for Efficient Daily Re-Planning.
        Med Phys Int J. 2020; 8: 77-86
        • Maslowski A.
        • Wang A.
        • Sun M.
        • Wareing T.
        • Davis I.
        • Star-Lack J.
        Acuros CTS: A fast, linear Boltzmann transport equation solver for computed tomography scatter – Part I: Core algorithms and validation.
        Med Phys. 2018; 45: 1899-1913https://doi.org/10.1002/mp.12850
        • Wang A.
        • Maslowski A.
        • Messmer P.
        • Lehmann M.
        • Strzelecki A.
        • Yu E.
        • et al.
        Acuros CTS: A fast, linear Boltzmann transport equation solver for computed tomography scatter - Part II: System modeling, scatter correction, and optimization.
        Med Phys. 2018; 45: 1914-1925https://doi.org/10.1002/mp.12849