Influence of optimisation parameters on directly deliverable Pareto fronts explored for prostate cancer

Purpose: In inverse radiotherapy treatment planning, the Pareto front is the set of optimal solutions to the multi-criteria problem of adequately irradiating the planning target volume (PTV) while reducing dose to organs at risk (OAR). The Pareto front depends on the chosen optimisation parameters whose influence (clinically relevant versus not clinically relevant) is investigated in this paper. Methods: Thirty-one prostate cancer patients treated at our clinic were randomly selected. We developed an in-house Python script that controlled the commercial treatment planning system RayStation to calculate directly deliverable Pareto fronts. We calculated reference Pareto fronts for a given set of objective functions, varying the PTV coverage and the mean dose of the primary OAR (rectum) and fixing the mean doses of the secondary OARs (bladder and femoral heads). We calculated the fronts for different sets of objective functions and different mean doses to secondary OARs. We compared all fronts using a specific metric (clinical distance measure). Results: The in-house script was validated for directly deliverable Pareto front calculations in two and three dimensions. The Pareto fronts depended on the choice of objective functions and fixed mean doses to secondary OARs, whereas the parameters most influencing the front and leading to clinically relevant differences were the dose gradient around the PTV, the weight of the PTV objective function, and the bladder mean dose. Conclusions: Our study suggests that for multi-criteria optimisation of prostate treatments using external therapy, dose gradient around the PTV and bladder mean dose are the most influencial parameters.


Introduction
In radiotherapy, the dose distribution is usually optimised using inverse treatment planning algorithms in order to deliver a treatment plan that is as close as possible to the desired dose distribution [1][2][3][4].The latter is specified by using objective functions and weights that represent the importance of achieving the corresponding objectives.Because there are multiple objectives (e.g.uniform dose to the planned target volume (PTV), dose as low as possible to the various organs at risk (OAR)), radiotherapy treatment planning is a multi-objective problem, also known as multi-criteria optimisation (MCO).The related continuum of optimal solutions is called the Pareto front, where a solution is called Pareto optimal if there is no other solution that improves at least one criterion without worsening another [5][6][7].In order to reduce complexity and visualise a radiotherapy Pareto front on a twodimensional graph, one can sample the front by varying only two objectives and keeping all other parameters constant [8].Keeping all other parameters constant ensures to maintain control over parameters that are not involved in the trade-off under consideration.
Previous studies have investigated how to efficiently approximate and navigate on the Pareto front for a given set of objective functions [9][10][11][12][13][14], built fully or partially automated treatment planning systems [15][16][17][18][19][20], and predicted the Pareto front solely on the anatomical features of the patient [21].When using multi-criteria Pareto optimisation either for online navigation on the Pareto front or for automated planning, it is important to know which optimisation parameters induce clinically relevant changes in the Pareto front and which ones do not.However, there is little literature investigating the influence of optimisation parameters related to the dose gradient around the PTV, the use of different objective functions and, in particular, different sets of objective functions on the PTV.To investigate this, we wrote and validated an inhouse script that controlled the TPS RayStation and calculated directly deliverable Pareto fronts.We decided to write a script instead of using the MCO module of RayStation because it was shown that in certain cases the navigated Pareto front may not be representative of the real deliverable Pareto front [12,22].Some of the reasons why they may not be representative may also change parameters that are not involved in the trade-off being studied.In addition, the script may be useful to other RayStation users because it densely samples the Pareto front, calculates each treatment plan from the ground up, and all plans are deliverable.
We applied the script to thirty-one prostate cancer patients and investigated the influence of the chosen objective functions and weights, as well as the mean doses to secondary OARs (bladder and femoral heads) on the variability of the Pareto front.The results of this investigation gave insights into the working principle of the used TPS optimiser, showing which objective functions have a major or minor impact on the Pareto front.Furthermore, varying doses for secondary OARs allowed us to better understand the interplay between trade-offs.

Materials and methods
We generated directly deliverable Pareto fronts for several sets of optimisation parameters using an in-house made Python script that controlled the TPS RayStation (RaySearch Laboratories, Sweden) [23,24].In order to evaluate whether the differences between the obtained fronts were clinically relevant, we used the clinical distance measure [25].

Script and Pareto fronts generation
The in-house script was written in Python 2.7 and controlled the TPS

Table 1
Set of objective functions and built-in constraints used for calculating a Pareto front of a prostate cancer patient with a prescription of 78 Gy in 39 fractions using HT.The technical structure''ring'' was a uniform expansion of the PTV of 2 cm.The gradient in the dose fall-off function was defined by the distance (1.5 cm) at which the dose should have been half of the prescribed dose (39 Gy).The Pareto generation script systematically changed the goal value of the max EUD objective function of the rectum.The script did not change any other parameter.In RayStation, different objective functions and corresponding weights could be assigned for different regions of interest according to their category of PTV or OAR.Once the user had defined a set of objective functions and weights, the RayStation optimiser searched for a treatment plan that was as close as possible to this set by minimising a quadratic cost function.Hereby, the optimiser tried to fulfil the objective functions as best as possible.
To control dose homogeneity in the PTV, the minimum dose (min dose), maximum dose (max dose), and uniform dose objective functions were chosen.We used the dose fall-off objective function in a 2 cm uniform expansion outside of the PTV to steepen the dose gradient in the vicinity of the PTV.We used the maximum equivalent uniform dose (max EUD) objective function (based on the one-parameter model equivalent uniform dose) in order to further reduce dose specifically in certain OARs [24,26].The EUD was defined as follows: where v i was the partial volume absorbing the dose D i , N the number of voxels and A a parameter.Hereby, we chose the parameter A = 1 such that the EUD equalled the mean dose.All objective functions used in this work were convex which was a criterium for having a convex optimisation problem meaning a problem with one single optimal solution [27,28].In addition to the objective functions, we could also define built-in constraints that would control the dose to secondary OARs.In contrast to objective functions, built-in hard constraints had to be fulfilled anyhow.
The script was executable in RayStation for any tumour site and treatment technique.In this work, we applied it to prostate cancer using the helical tomotherapy (HT) treatment technique.HT is a convex optimisation problem that was solved in RayStation using direct aperture optimisation, i.e. all treatment plans in this study were deliverable [24,29].The script was based on a scalarisation algorithm and sampled the Pareto front for a given set of objective functions.Scalarisation algorithms solve the problem of quadratic cost function minimisation by varying the weighting factors [9].In RayStation, the treatment plan optimisations diverged for some of the patients when using this standard scalarisation algorithm.We therefore developed an adapted version of the scalarisation algorithm where we varied the goal mean dose of the OAR objective function instead of the weighting factors.Like this, the treatment plan optimisations converged for all patients of the cohort.
The script generated Pareto optimal plans in two dimensions for a given set of objective functions (Table 1).It started with a treatment plan with high PTV coverage and gradually decreased the goal mean dose of a selected OAR which compromised PTV coverage.At the same time, the mean doses to the other OARs were kept constant by applying built-in constraints that were fulfilled with an accuracy of ±0.01 Gy.The script reset the optimisation before calculating each new Pareto optimal plan.By applying the script that calculated the Pareto front in two dimensions several times using different fixed mean doses on a second OAR, we were able to obtain a Pareto surface in three dimensions with the PTV under-coverage and the mean doses to the first and second OARs as axes parameters.In this study, the term''fixed mean dose'' refers to the principle of keeping the mean dose constant.This can be achieved using different methods, one of which is the use of built-in constraints of RayStation.
In RayStation, we used a uniform dose grid with a resolution of 0.3 cm per voxel.The optimiser performed at least two times forty iterations and calculated the dose with the "Collapsed Cone v5.2" algorithm at the end of each fortieth iteration.The iterations were stopped at convergence using a RayStation optimisation tolerance of 10 -5 .We ran the script on a laptop with an NVIDIA Quadro RTX 5000 GPU processor and 32 GB RAM memory.These settings led to a script execution time of about one and a half hour per case, where about 30 treatment plans were generated.

Validation of the script 2.2.1. Equivalence of adapted scalarisation and classical scalarisation algorithm
While classical scalarisation algorithms solve the quadratic cost function minimisation problem by varying the weighting factors, our adapted version varied the goal mean dose of the rectum objective function.A priori, we did not know whether the two approaches would lead to the same result because we did not know the exact mathematic formulation of the cost function in RayStation.We tested the equivalence of the two approaches by running them on one prostate cancer patient, in which the optimisations converged using the classical scalarisation algorithm, and evaluated if any potential differences between the two approaches were clinically relevant.

Consistency test
In order to obtain a Pareto surface in three dimensions, we applied the script that calculated the Pareto front in two dimensions (PTV coverage versus rectum mean dose) several times using different fixed mean doses on the bladder.For testing the consistency, we then exchanged the role of the rectum and the bladder and calculated the Pareto surface again.Finally, we evaluated whether the clinical distances of the two surfaces were clinically relevant.This consistency test was performed on three prostate cancer patients (one for each Group A, B and C).

Built-in constraint test
The script for generating Pareto fronts in two dimensions (PTV coverage versus rectum mean dose) fixed the mean doses to the bladder and femoral heads using built-in constraints from RayStation.A priori, we did not know how these built-in constraints were implemented in the Table 2 Colour code for the maximal clinical distances (max cd) of the Pareto fronts for different sets of objective functions with respect to the reference Pareto front for all patients.Grey-scale: max cd not clinically relevant, yellow: max cd > 0.28 which may be clinically relevant (lower limit of confidence interval), orange-red-purplescale: max cd > 0.32 which is clinically relevant.The sets of objective functions that led to these Pareto fronts are listed in Appendix C. (For interpretation of the references to colour in this table, the reader is referred to the web version of this article.) RayStation optimiser and whether it would have been possible to get better treatment plans (lower rectum mean dose for the same PTV coverage and mean doses to the bladder and femoral heads) if optimising manually by iteratively changing the goal mean doses and weights of the rectum, bladder, and femoral heads objective functions.We tested this on one prostate cancer patient where we optimised treatment plans by manually adapting the goal mean doses and weights of the objective function for the bladder (strategy 1) or the femoral heads (strategy 2) or both of them (strategy 3).All manual treatment plans had the same mean doses to the bladder and the femoral heads as the treatment plans calculated with our script.The uncertainty on those mean doses was ± 0.01 Gy for plans calculated with the script and ± 0.1 Gy for manual treatment plans.Each treatment plan was calculated independently of other plans, meaning that we reset the optimisation before calculating a new plan.In the case that there were some manual plans that were better than the ones obtained with our script, we had to evaluate if this advantage was clinically relevant.

Clinical distance measure
The clinical distance is a special case of the efficiency score used in data envelopment analysis, which is a standard metric in economy [30] and has also been applied to radiotherapy in recent years [31][32][33][34][35].A detailed description can be found elsewhere [25].In summary, the clinical distance between a treatment plan α with a set of evaluation parameters a = (a 1 , a 2 , …, a n ) and a Pareto front B consisting of the Pareto optimal plans β 1 , β 2 , …, β M with the sets of evaluation parameters b i,1 , b i,2 , …, b i,M for i = 1 to n was defined as follows: The clinical scaling factor k = (k 1 , k 2 , …, k n ) determined the clinical importance of the evaluation parameters [25,[36][37][38].The study proposed to take 0.5 for the PTV under-coverage and 0.05 Gy − 1 for the rectum and bladder mean doses [25].Note that equation (2) describes the distance between a point (one single treatment plan) and a line (Pareto front).The study showed that non-Pareto optimal plans located at a clinical distance value>0.32 (0.28 -0.35) from the Pareto front were clinically considered to be of lower quality.In this work, the term''clinically relevant'' therefore refers to a clinical distance>0.32 (0.28 -0.35).For this study, we wanted to investigate whether changing a certain optimisation parameter p could lead to clinically relevant different treatment plans.Therefore, we calculated the clinical distances of all treatment plans α 1 , α 2 , … α N obtained with this changed optimisation parameter and the reference Pareto front B obtained with the unchanged optimisation parameter.If the maximum of all those clinical distances exceeded 0.32 (0.28 -0.35), the optimisation parameter p might affect the Pareto front in a clinically relevant way.In addition, for comparing the positions of the reference Pareto fronts of several patients, we calculated the clinical distances of these fronts to the origin of the coordinate system.

Influence of optimisation parameters
We randomly selected thirty-one prostate cancer patients treated at our clinic and applied a prescription of 78 Gy in 39 fractions using HT.
For each patient, we first calculated a two-dimensional reference Pareto front for a given set of objective functions (Table 1), varying the tradeoff PTV under-coverage (100% -V 95% ) and mean dose to the primary OAR (rectum), where the latter was related to the risk of faecal incontinence [39][40][41][42][43][44][45][46].In our clinic, a treatment plan is acceptable if the PTV under-coverage does not exceed 5%.For the Pareto front calculations, we fixed the mean doses to secondary OARs (bladder and femoral heads) at given values (Table A1 in Appendix A).Those values have been chosen in the following way: Among others, we wanted to compare the Pareto fronts of several patients.Since the fixed mean doses to the bladder and femoral heads influence the position of the Pareto front, such a comparison is only meaningful if these fixed mean doses are the same in all patients.However, for each patient anatomy, there is only a certain range of possible fixed mean doses that allow the calculation of clinically acceptable Pareto optimal plans.In other words, if the mean dose to the bladder is chosen too high, the obtained plans are not Pareto optimal meaning that it is possible to find other plans with the same PTV under-coverages and rectum mean doses using a lower fixed mean dose to the bladder; if the fixed mean dose to the bladder is chosen too low, D. Wüthrich et al. the obtained Pareto optimal treatment plans will have a PTV undercoverage smaller than 5%, which is inacceptable in our clinic.The same applies to the fixed mean doses to the femoral heads.We determined the range of possible fixed mean doses that allow the calculation of clinically acceptable Pareto optimal plans using a trial and error procedure.Once, this range was known for each patient, we divided the patient cohort into three groups (A, B and C) with the same fixed mean doses for all patients of one group (bladder 21 Gy (A), 28 Gy (B) and 38 Gy (C); femoral heads 7 Gy).Hereby, we wanted the Pareto fronts to cover almost the whole range of clinically acceptable PTV under-

Table 3
Colour code for the maximal clinical distances (max cd) of the Pareto fronts for different fixed bladder mean doses with respect to the reference Pareto front for all patients.Grey-scale: max cd not clinically relevant (max cd = 0 for reference Pareto front), yellow: max cd > 0.28 which may be clinically relevant (lower limit of confidence interval), orange-red-purple-scale: max cd > 0.32 which is clinically relevant.Empty white fields mean that the corresponding bladder mean dose did not enable calculating Pareto optimal plans for this patient.(For interpretation of the references to colour in this coverage (1.5% -5%) in order to have enough points on the front for doing meaningful data analysis.Some patients could not be assigned to those three groups because the range of possible fixed mean doses to the bladder or the femoral heads was too low or too high.Group D therefore consists of those patients that could not be assigned to groups A, B and C. 85% of the so-obtained Pareto optimal treatment plans were acceptable by the standards of our clinic (acceptance criteria in Table B1 in Appendix B).
We calculated the fronts for different sets of objective functions (Appendix C) and compared those fronts to the reference Pareto fronts using the clinical distance.For the different sets of objective functions we varied the weight on the PTV and rectum objective functions.We also varied the gradient of the dose fall-off function around the PTV and chose several combinations of objective functions on the PTV.In a next step, we calculated the Pareto fronts using the same set of objective functions as for the reference Pareto fronts but with different mean doses Fig. 6.Reference Pareto fronts for Group A (left).Clinical distances of these fronts to the origin of the coordinate system calculated using equation ( 2) plotted against the PTV-rectum-overlap volume relative to the rectum volume for the same patients (right).Each colour and symbol represents one patient.(For interpretation of the references to colour in this figure, the reader is referred to the web version of this article.)

Table A1
Patient groups, patient numbers, fixed mean doses to the bladder and femoral heads for the reference Pareto fronts as well as the PTV-bladder-overlap volume relative to the bladder volume and the PTV-rectum-overlap volume relative to the rectum volume.

Table B1
Criteria for clinical acceptability of prostate cancer treatment plans at our clinic.to one of the secondary OARs (bladder or femoral heads).Again, we compared the obtained fronts to the reference Pareto fronts using the clinical distance measure.Finally, we tested if the PTV-rectum overlap was correlated to the clinical distances of the reference Pareto fronts to the origin using the Pearson correlation.Correlations with a two-tailed p-value less than 0.05 were considered statistically significant.

Validation of the script used for generating Pareto fronts
Our adapted version of the scalarisation algorithm lead to the same result as the classical scalarisation algorithm (Fig. 1).
Fig. 2 shows that the script for generating Pareto surfaces in three dimensions was consistent under the exchange of the role of the rectum and the bladder (three-dimensional rotatable image in supplementary material).The two surfaces had a mean clinical distance of 0.068, 0.048, 0.061 and a maximum clinical distance of 0.384, 0.181, 0.196 for the chosen patients of groups A, B, and C, respectively.
Concerning the built-in constraints test, most of the manual plans were worse (higher rectum mean dose for the same PTV coverage) than the plans obtained using the script (Fig. 3).Some manual plans were slightly better than the plans obtained with the script.However, this difference was not clinically relevant.

Variation of objective functions
The Pareto front changed its shape and position when changing certain objective functions (Fig. 4 and Table 2).The parameter most influencing the position of the Pareto front was the dose fall-off objective function (clinically relevant change for the vast majority of patients for the 1 cm dose fall-off function).The second most influencing parameter on the Pareto front was the weight of the PTV objective function.Changing the weight of the rectum objective function or using several objective functions on the PTV induced no clinically relevant changes in the Pareto front.When interpreting Fig. 4 and Table 2, it is important to note that the clinical distance measure in equation ( 2) does not simply consider the mean rectal dose difference for a given PTV under-coverage of two Pareto fronts (vertical direction in Fig. 4), but rather the shortest distance between a treatment plan and the reference Pareto front (diagonal direction in Fig. 4).This is especially important when the Pareto fronts are steep.

Table C3
Set of objective functions called ''5 objective functions on PTV''.

Variation of the mean doses to secondary OARs
The Pareto front also changed its shape and position when changing the fixed mean dose to the bladder (Fig. 5 and Table 3).A change of 1 Gy sometimes lead to a clinically relevant change.Changing the fixed mean dose to the femoral heads by up to ±2 Gy induced no clinically relevant changes (data not shown).

Positions of reference Pareto fronts and PTV-rectum overlap
The clinical distances of the reference Pareto fronts to the origin of the coordinate system were positively correlated to the relative PTVrectum overlap volumes with the Pearson correlation coefficient r and two-tailed p-value p of r = 0.78, p = 0.02 for Group A, r = 0.81, p = 0.01 for Group B and r = 0.39, p = 0.45 for Group C (Fig. 6 and Appendix D).The correlation was statistically significant for groups A and B.

Discussion
Our in-house script was validated and gave a good approximation of the directly deliverable Pareto fronts.The resulting Pareto fronts depended on the choice of objective functions and fixed mean doses to secondary OARs (bladder and femoral heads).The parameters most influencing the front were the dose gradient around the PTV, the weight of the PTV objective function, the fixed bladder mean dose, as well as the relative PTV-rectum-overlap volume.Concerning variations in objective functions (Fig. 4 and Table 2), the parameter most influencing the position of the Pareto front was the dose fall-off objective function, because a steep dose gradient around the PTV (dose fall-off: 1 cm) forced the TPS optimisation algorithm to lower the dose close to the PTV and therefore led to a worse PTV coverage.This was clinically relevant for eighteen and might be clinically relevant for four out of 31 patients.The second most influencing parameter on the Pareto front was the weight of the PTV objective function.The higher the weight, the better the Pareto frontmeaning a better PTV coverage for the same rectum mean dose.When choosing a weighting factor that is too low (weight PTV: 50), the shift in the Pareto front was clinically relevant for two and might be clinically relevant for four out of 31 patients.The Pareto front was independent of the choice of the weighting factor of the OAR objective function, because a lowering of the goal mean dose of the OAR objective function led to the same result as increasing its weighting factor (Fig. 1).The fact that the weight of the PTV objective function changed the Pareto front while the weight of the OAR objective function did not was likely because the weighting factors of PTV and OAR objective functions were differently scaled in the RayStation optimiser.For most patients, there was no advantage when using many different objective functions on the PTV.Using the uniform dose objective function only gave the best results which was probably linked to the architecture of the RayStation optimisation algorithm.Concerning variations in the fixed bladder mean dose (Fig. 5 and Table 3), a change of 1 Gy sometimes lead to a clinically relevant change of the front whereas the front went up (higher rectum mean dose for the same PTV coverage) if the bladder mean dose was decreased.Some patients had a wider and some a narrower range of possible bladder mean doses.The femoral heads mean doses, on the other hand, did not influence the Pareto front in a clinically relevant way.As the fixed mean doses to the bladder highly influenced the positions of the Pareto fronts, one could argue that it would have been better to group the patients according to an anatomical trait related to the bladder instead of doing a long trial and error procedure to find the best values for the fixed bladder mean doses.Indeed, the PTV-bladder overlap volume relative to the bladder volume was (3.9 ± 0.9)% for Group A, (6.1 ± 2.8)% for Group B and (10.9 ± 2.2)% for Group C, which means that the bigger the chosen fixed bladder mean dose the bigger this overlap volume was on average.However, simply grouping the patients according to the relative PTV-bladder-overlap was not possible.To give an example, patient number 16 of Group B had a smaller overlap than patient number 7 of Group A (Table A1 in Appendix A) but neither patient number 16 could have been assigned to Group A nor patient number 7 to Group B (Table 3).Finally, the correlation of the positions of the reference Pareto fronts with the relative PTV-rectum overlap volumes could be explained by the fact that the bigger the overlap, the more difficult it was to spare the rectum and the farther the Pareto front was from the origin (Fig. 6, D1 and D2 in Appendix D).
In summary, the chosen optimisation parameters influenced the Pareto front.Some parameters influenced the front more than others, for example, the weight of the PTV objective function influenced the front more than that of the rectum objective function, which was probably related to the optimisation algorithm of the TPS.The results of this study can therefore help RayStation users to better understand the architecture of the optimisation algorithm, which will enable them to design better treatment plans.More generally, the dependence of the Pareto front on the chosen optimisation parameters means that we should be cautious about considering the Pareto front as the 'best' plans, as there could be better plans if we adjust the optimisation parameters.However, in most cases the Pareto fronts were stable, which means that changing the objective functions and weights changed the Pareto fronts, but those changes were not clinically relevant.The addition of a third dimension, on the other hand, namely the third trade-off bladder mean dose, changed the front in a clinically relevant way.This was not the case for the trade-off femoral heads mean dose.This means that the interplay between trade-offs can play an important role for Pareto front calculations.
There already exist other solutions to calculate Pareto optimal plans.Nevertheless, we decided to write our own script for the following reasons.Some solutions did calculate only one single Pareto optimal plan on the Pareto front [15][16][17][18][19][20]47], which was not what we were looking for.Instead we were looking for solutions that calculate a whole representation of the Pareto front.Among others, the so called PGEN algorithm is such a solution [9]: It approximates the Pareto front for one set of objective functions.That algorithm is very similar to the commercially available MCO module of RayStation [10][11][12][13][14].We did not use the MCO module of RayStation because the navigated Pareto front may not be representative of the real deliverable Pareto front and because we wanted to fix the mean doses to secondary OARs at given values.That latter was important in order to reduce the multidimensional Pareto hypersurface to two-dimensions without losing control over the important parameters that were not involved in the trade-off investigated.The commercially available MCO module of Eclipse is very similar to the one of RayStation [48].Another algorithm predicts the whole Pareto front solely on the anatomical features of the patient [21].Also this approach is not designed for fixing the mean doses to secondary OARs at certain values.Finally, another example of a solution that calculates a whole representation of the Pareto front is a script that controls the TPS Pinnacle and calculates a Pareto front for several sets of objective functions [49].Similar to our work, it is an inhouse Python script that controls a commercial TPS.Unlike our work, it calculates the front for several sets of objective functions rather than just one.This allows a better approximation of the''real'' Pareto front, but at the cost of generating many unnecessary plans.In our work, by using built-in constraints from RayStation, we could avoid generating many unnecessary plans and were able to converge directly to the Pareto front.
In the literature, other studies investigated the influence of optimisation parameters on Pareto fronts for prostate cancer [9,12].We confirmed the findings that enforcing the tumour dose worsened the rectum mean dose and that femoral heads mean dose had a small impact on the Pareto front.In contrast to other studies, we also evaluated the trade-off bladder mean dose and dose gradient around the PTV.While there are some studies that investigate the interplay between tumour dose and mean doses to different OARs, to the authors' knowledge, there is no other study about the stability of the Pareto front with respect to the choice of the set of objective functions.For Erasmus iCycle [15] as well as for SALO [18], the stability of the generated Pareto optimal treatment plan with respect to the chosen wish-list (called priority-list in SALO) was studied.However, the results of these studies cannot be directly compared to our study because of the following considerations.While both, iCycle and SALO, applied a constraint optimisation method with constraint relaxation, the commercial TPS that we used for this work was based on a weighted-sum method.Even though the same output treatment plan can be obtained with the two methods, the translation of the input wish-list and information about the magnitude of constraint relaxation into an input set of objective functions and weights is patient dependent and cannot be made a priori [27].Finally, we confirmed the findings of a previous study on three patients suggesting that the positions of the reference Pareto fronts were correlated to the relative PTV-rectum overlap volumes [36].
Our in-house script is limited to two or three dimensions, due to script execution time.A further limitation of the script is that it cannot be directly used clinically, since navigation on the front seeing the dose distribution of a selected plan in real-time is not possible.Nevertheless, the script enabled us to extract information about Pareto front variability that can be transferred to existing clinical MCO online navigation tools.The results on the variability of the Pareto front have been obtained in a specific context and should be taken cautiously when expanding them beyond the context of this study, i.e. to another treatment site than prostate, another treatment technique than HT or another TPS than RayStation.For example, the result that the weight of the PTV objective function is more important than that of the rectum is probably only related to the optimisation algorithm used by our TPS.On the other hand, the results that the dose gradient around the PTV and the mean bladder dose are the most influential parameters likely also apply to other treatment techniques and TPSs.
Apart from the clinical distance measure used in this work [25], there are numerous metrics that quantify the distance of a given Pareto front to a reference Pareto front [49][50][51][52][53][54][55][56][57][58][59].The two metrics most commonly used in the literature are the efficiency score in data envelopment analysis [30][31][32][33][34][35] and the hypervolume indicator [60][61][62][63][64][65], also referred to as S-metric or Lebesgue measure by some authors.These two metrics have in common to be Pareto compliant meaning that for multiple non-overlapping fronts, the metrics assign higher values to the fronts that are further away from the reference Pareto front and zero value when the front is identical to the reference Pareto front.The hypervolume indicator accounts for the clinical importance of the evaluation parameters by normalising them to a clinically meaningful range but it is limited by the fact that its value depends on the selection of an arbitrary reference point.The efficiency score, on the other hand, has more freedom in accounting for the clinical importance of evaluation parameters through the use of weights.These weights correspond to the clinical scaling factors used in the clinical distance measure used in this work.When accounting for the clinical importance of the evaluation parameters, be it in any conceivable metric, one automatically introduces a certain degree of subjectivity, as each person assigns a different importance to each parameter.In a previous study, we therefore tried to eliminate this subjectivity by doing blinded comparisons of treatment plans with twelve different clinicians and across five different prostate cancer patients [25].

Conclusions
We developed a script that calculates a good approximation to the directly deliverable Pareto front for a given set of objective functions and fixed mean doses for secondary OARs (bladder and femoral heads).The selected objective functions and weights influence the front whereas the most influencial objective function is the dose gradient around the PTV.However, apart from this specific objective function, the variability of the front is not clinically relevant in most cases when chosing different objective functions and weights.On the other hand, the introduction of a third dimension, namely the mean dose of the bladder, changes the front in a clinically relevant way.This is not the case for the mean dose of the femoral heads.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

1 Fig. 1 .Fig. 2 .Fig. 3 .
Fig. 1.Varying the goal mean dose of the rectum objective function (adapted scalarisationblue circles) lead to the same Pareto front as varying the weighting factor (scalarisationred squares) shown on a patient representative of the series.The red squares had a mean clinical distance of 0.027 and a maximum clinical distance of 0.071 from the blue squares.(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4 .
Fig. 4. Example of Pareto fronts for patient number 7 of group A with different sets of objective functions plotted on the same graph as the reference Pareto front.The sets of objective functions that led to these Pareto fronts are listed in Appendix C.

Fig. 5 .
Fig. 5. Pareto fronts for different fixed bladder mean doses (17 Gy purple to 36 Gy red) for patient number 10 of group B (left).Maximal clinical distances of these Pareto fronts to the reference Pareto front for the same patient (right).Positive values mean that the corresponding front was above the reference front meaning higher rectum mean dose and lower PTV coverage.Clinically relevant ranges on the plot are marked with a background colour (absolute values > 0.28 yellow lower limit of confidence interval, > 0.32 red).In both plots, the same colours are assigned to each value of the bladder mean dose.The reference Pareto front (28 Gy bladder mean dose) is shown using black squares.(For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. D1 .
Fig. D1.Reference Pareto fronts for Group B (left).Clinical distances of the Pareto fronts to the origin of the coordinate system for the same patients (right).The parameter on the x-axis is the PTV-rectum-overlap volume relative to the rectum volume.Each colour and symbol represents one patient.(For interpretation of the references to colour in this figure, the reader is referred to the web version of this article.)

Fig. D3 .
Fig. D3.Reference Pareto fronts for Group D (left).Each colour and symbol represents one patient.
table, the reader is referred to the web version of this article.) Colour code for the maximal clinical distance being bigger or equal to the following values.D.Wüthrich et al.

Table C1 Sets
of objective functions called "Weight PTV: 50, 100, 400 and 800" by adapting the weight of the objective function of the PTV, "Dose fall-off: 1 cm and 2 cm'' by adapting the distance in the dose fall-off objective function and "Weight rectum: 1, 25, 250 and 2500" by adapting the weight of the objective function of the rectum.

Table C2
Set of objective functions called ''3 objective functions on PTV''.