Prediction and classification of VMAT dosimetric accuracy using plan complexity and log-files analysis

Published:October 14, 2022DOI:


      • Complexity index (MCS) and log-files analysis were used for PSQA accuracy assessment.
      • Prediction accuracy for gamma-pass rate (γ%) was 2.1%.
      • Precision, recall and F-score performances for γ% was greater than 90%.
      • An MCS-based traffic light protocol was implemented to “a-priori” flag delivery accuracy.
      • The optimal MCS threshold values for failed and pass plans were <0.130 and >0.270.



      We presented different machine learning models based on log files analysis and complexity indexes to predict and classify the dosimetric accuracy of VMAT plans.


      A total of 1302 VMAT arcs from 651 treatment plans were analyzed using the modulation complexity score (MCS) and the dynamic log-files generated by the linac. Predicted and measured fluences were compared using γ-analysis in terms of mean γ-values (γmean) and γ-pass rate (γ%). A kernel regression model was developed aiming to predict individual γ% and γmean values. Multinomial logistic regression (LR), Naïve-Bayes (NB) and support vector machine (SVM) models were developed based on MCS values to classify QA results as “pass” (γ%greater than90 % and γmean < 0.5), “control” (80 % < γ% < 90 % and 0.50 < γmean < 0.75) and “fail” (γ% < 80 % and γmean > 0.75). Training, validation and testing groups were used to evaluate the model reliability. A complexity-based traffic light protocol was implemented to flag pass (green light), control (orange light) and failed plans (red light).


      Prediction accuracy of residuals for γ% was 2.1 % and 2.2 % in the training and testing cohorts, respectively. For 2 %(local)/2mm, both γ% and γmean classification performances reported weighted precision, recall and F1-values greater than 90 % for all machine learning models. The optimal MCS threshold value for the identification of failed plans was 0.130, with a sensibility and specificity of 0.994 and 0.952, respectively. The optimal MCS threshold for reliable plans was 0.270, with a sensitivity and specificity of 0.925 and 0.922, respectively.


      Machine learning can accurately predict the dosimetric accuracy of VMAT treatments, representing an efficient tool to assist patient-specific QA.

      Graphical abstract


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Physica Medica: European Journal of Medical Physics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Das I.J.
        • Ding G.X.
        • Ahnesjö A.
        Small fields: nonequilibrium radiation dosimetry.
        Med Phys. 2008; 35: 206-215
        • Oliver M.
        • Gagne I.
        • Bush K.
        • et al.
        Clinical significance of multileaf collimator positional errors for volumetric modulated arc therapy.
        Radiother Oncol. 2010; 97: 554-560
        • Craft D.
        • Süss P.
        • Bortfeld T.
        The tradeoff between treatment plan quality and required number of monitor units in intensity modulated radiotherapy.
        Int J Radiat Oncol Biol Phys. 2007; 67: 1596-1605
        • Moran J.M.
        • Dempsey M.
        • Eisbruch A.
        • et al.
        Safety considerations for IMRT: executive summary.
        Pract Radiat Oncol. 2011; 1: 190-195
        • Miften M.
        • Olch A.
        • Mihailidis D.
        • et al.
        Tolerance limits and methodologies for IMRT measurement-based verification QA: recommendations of AAPM Task Group No. 218.
        Med Phys. 2018; 45: e53-e83
        • Antoine M.
        • Ralite F.
        • Soustiel C.
        • et al.
        Use of metrics to quantify IMRT and VMAT treatment plan complexity: a systematic review and perspectives.
        Phys Med. 2019; 64: 98-108
        • McNiven A.L.
        • Sharpe M.B.
        • Purdie T.G.
        A new metric for assessing IMRT modulation complexity and plan deliverability.
        Med Phys. 2010; 37: 505-515
        • Masi L.
        • Doro R.
        • Favuzza V.
        • et al.
        Impact of plan parameters on the dosimetric accuracy of volumetric modulated arc therapy.
        Med Phys. 2013; 40071718
        • Park J.M.
        • Park S.Y.
        • Kim H.
        • et al.
        Modulation indices for volumetric modulated arc therapy.
        Phys Med Biol. 2014; 59: 7315-7340
        • Hernandez V.
        • Saez J.
        • Pasler M.
        • et al.
        Comparison of complexity metrics for multi-institutional evaluations of treatment plans in radiotherapy.
        Phys Imag Radiat Oncol. 2018; 5: 37-43
        • Agnew C.E.
        • Irvine D.M.
        • McGarry C.K.
        Correlation of phantom-based and log file patient-specific QA with complexity scores for VMAT.
        J Appl Clin Med Phys. 2014; 15: 4994
        • Glenn M.C.
        • Hernandez V.
        • Saez J.
        • et al.
        Treatment plan complexity does not predict IROC Houston anthropomorphic head and neck phantom performance.
        Phys Med Biol. 2018; 63205015
        • Avanzo M.
        • Porzio M.
        • Lorenzon L.
        • et al.
        Artificial intelligence applications in medical imaging: a review of the medical physics research in Italy.
        Phys Med. 2021; 83: 221-241
        • Zanca F.
        • Hernandez-Giron I.
        • Avanzo M.
        • et al.
        Expanding the medical physicist curricular and professional programme to include Artificial Intelligence.
        Phys Med. 2021; 83: 174-183
        • Wall P.D.H.
        • Fontenot J.D.
        Quality assurance-based optimization (QAO): Towards improving patient-specific quality assurance in volumetric modulated arc therapy plans using machine learning.
        Phys Med. 2021; 87: 136-143
        • Granville D.A.
        • Sutherland J.G.
        • Belec J.G.
        • et al.
        Predicting VMAT patient-specific QA results using a support vector classifier trained on treatment plan characteristics and linac QC metrics.
        Phys Med Biol. 2019; 64095017
        • Wall P.D.
        • Fontenot J.D.
        Application and comparison of machine learning models for predicting quality assurance outcomes in radiation therapy treatment planning.
        Inf Med Unlocked. 2020; 18100292
        • Noblet C.
        • Duthy M.
        • Coste F.
        • Saliou M.
        • Samain B.
        • Drouet F.
        • et al.
        Implementation of volumetric-modulated arc therapy for locally advanced breast cancer patients: dosimetric comparison with deliverability consideration of planning techniques and predictions of patient-specific QA results via supervised machine learning.
        Phys Med. 2022; 96: 18-31
        • Valdes G.
        • Scheuermann R.
        • Hung C.Y.
        • et al.
        A mathematical framework for virtual IMRT QA using machine learning.
        Med Phys. 2016; 43: 4323
        • Li J.
        • Wang L.
        • Zhang X.
        • et al.
        Machine learning for patient-specific quality assurance of VMAT: prediction and classification accuracy.
        Int J Radiat Oncol Biol Phys. 2019; 105: 893-902
        • Ono T.
        • Hirashima H.
        • Iramina H.
        • et al.
        Prediction of dosimetric accuracy for VMAT plans using plan complexity parameters via machine learning.
        Med Phys. 2019; 46: 3823-3832
        • Hirashima H.
        • Ono T.
        • Nakamura M.
        • et al.
        Improvement of prediction and classification performance for gamma passing rate by using plan complexity and dosiomics features.
        Radiat Oncol. 2020; 8140: 250-257
        • Tomori S.
        • Kadoya N.
        • Kajikawa T.
        • et al.
        Systematic method for a deep learning-based prediction model for gamma evaluation in patient-specific quality assurance of volumetric modulated arc therapy.
        Med Phys. 2021; 48: 1003-1018
        • Maes D.
        • Bowen S.R.
        • Regmi R.
        • Bloch C.
        • Wong T.
        • Rosenfeld A.
        • et al.
        A machine learning-based framework for delivery error prediction in proton pencil beam scanning using irradiation log-files.
        Phys Med. 2020; 78: 179-186
        • Pasler M.
        • Kaas J.
        • Perik T.
        • Geuze J.
        • Dreindl R.
        • Künzler T.
        • et al.
        Linking log files with dosimetric accuracy–A multi-institutional study on quality assurance of volumetric modulated arc therapy.
        Radiother Oncol. 2015; 117: 407-411
        • Viola P.
        • Romano C.
        • Craus M.
        • Macchia G.
        • Buwenge M.
        • Indovina L.
        • et al.
        Prediction of VMAT delivery accuracy using plan modulation complexity score and log-files analysis.
        Biomed Phys Eng Express. 2022; 8
        • Szeverinski P.
        • Kowatsch M.
        • Künzler T.
        • Meinschad M.
        • Clemens P.
        • DeVries A.F.
        Error sensitivity of a log file analysis tool compared with a helical diode array dosimeter for VMAT delivery quality assurance.
        J Appl Clin Med Phys. 2020; 21: 163-171
        • Tibshirani R.
        Regression shrinkage and selection via the lasso.
        J Roy Stat Soc B. 1996; : 267-288
        • David V.
        • Sanchez A.
        Advanced support vector machines and kernel methods.
        Neurocomputing. 2003; 55: 5-20
        • Kononenko I.
        Inductive and Bayesian learning in medical diagnosis.
        Appl Artif Intell. 1993; 7: 317-337
        • Fluss R.
        • Faraggi D.
        • Reiser B.
        Estimation of the Youden index and its associated cut-off point.
        Biometrical Journal. 2005; 47: 458-472
        • Khan R.
        • Darafsheh A.
        • Goharian M.
        • et al.
        Evolution of clinical radiotherapy physics practice under COVID-19 constraints.
        Radioth Oncol. 2020; 148: 274-278
        • Hussein M.
        • Rowshanfarzad P.
        • Ebert M.A.
        • et al.
        A comparison of the gamma index analysis in various commercial IMRT/VMAT QA systems.
        Radiother Oncol. 2013; 109: 370-376
        • Heilemann G.
        • Poppe B.
        • Laub W.
        On the sensitivity of common gamma-index evaluation methods to MLC misalignments in Rapidarc quality assurance.
        Med Phys. 2013; 40031702
        • Nelms B.E.
        • Chan M.F.
        • Jarry G.
        • et al.
        Evaluating IMRT and VMAT dose accuracy: practical examples of failure to detect systematic errors when applying a commonly used metric and action levels.
        Med Phys. 2013; 40111722
        • Saiful Huq M.
        • Fraass B.A.
        • Dunscombe P.B.
        • et al.
        The report of Task Group 100 of the AAPM: application of risk analysis methods to radiation therapy quality management.
        Med Phys. 2016; 43: 4209-4262
        • Ono T.
        • Nakamura M.
        • Ono Y.
        • et al.
        Development of a plan complexity mitigation algorithm based on gamma passing rate predictions for volumetric-modulated arc therapy.
        Med Phys. 2022; 49: 1793-1802
        • Kerns J.R.
        • Stingo F.
        • Followill D.S.
        • Howell R.M.
        • Melancon A.
        • Kry S.F.
        Treatment planning system calculation errors are present in most imaging and radiation oncology core-houston phantom failures.
        Int J Radiat Oncol Biol Phys. 2017; 98: 1197-1203