Advertisement
Research Article| Volume 107, 102546, March 2023

Download started.

Ok

CT-derived radiomic analysis for predicting the survival rate of patients with non-small cell lung cancer receiving radiotherapy

  • Author Footnotes
    1 These authors contributed equally to this work.
    Nannan Zhang
    Footnotes
    1 These authors contributed equally to this work.
    Affiliations
    Modern Educational Technology and Experiment Center, Harbin Normal University, Harbin, China
    Search for articles by this author
  • Author Footnotes
    1 These authors contributed equally to this work.
    Xinxin Zhang
    Footnotes
    1 These authors contributed equally to this work.
    Affiliations
    College of Life Science and Technology, Harbin Normal University, Harbin, China
    Search for articles by this author
  • Junheng Li
    Affiliations
    Basic Medicine College, Harbin Medical University, Harbin, China
    Search for articles by this author
  • Jie Ren
    Affiliations
    Basic Medicine College, Harbin Medical University, Harbin, China
    Search for articles by this author
  • Luyang Li
    Affiliations
    Basic Medicine College, Harbin Medical University, Harbin, China
    Search for articles by this author
  • Wenlei Dong
    Correspondence
    Corresponding authors at: Basic Medicine College, Harbin Medical University, Harbin 150086, China (Yixin Liu). Department of Radiotherapy Technology Center, Harbin Medical University Cancer Hospital, Harbin 150086, China (Wenlei Dong).
    Affiliations
    Department of Radiotherapy Technology Center, Harbin Medical University Cancer Hospital, Harbin, China
    Search for articles by this author
  • Yixin Liu
    Correspondence
    Corresponding authors at: Basic Medicine College, Harbin Medical University, Harbin 150086, China (Yixin Liu). Department of Radiotherapy Technology Center, Harbin Medical University Cancer Hospital, Harbin 150086, China (Wenlei Dong).
    Affiliations
    Basic Medicine College, Harbin Medical University, Harbin, China
    Search for articles by this author
  • Author Footnotes
    1 These authors contributed equally to this work.
Open AccessPublished:February 14, 2023DOI:https://doi.org/10.1016/j.ejmp.2023.102546

      Highlights

      • The radiomic signature can noninvasively predict radiological response for NSCLC.
      • Our approach significantly improves the predictive performance, particularly early stage NSCLC patients.
      • Radiogenomics analysis link our signature is favorable for clinical application.

      Abstract

      Background

      Radiomics provides an opportunity to minimize adverse effects and optimize the efficacy of treatments noninvasively. This study aims to develop a computed tomography (CT) derived radiomic signature to predict radiological response for the patients with non-small cell lung cancer (NSCLC) receiving radiotherapy.

      Methods

      Total 815 NSCLC patients receiving radiotherapy were sourced from public datasets. Using CT images of 281 NSCLC patients, we adopted genetic algorithm to establish a predictive radiomic signature for radiotherapy that had optimal C-index value by Cox model. Survival analysis and receiver operating characteristic curve were performed to estimate the predictive performance of the radiomic signature. Furthermore, radiogenomics analysis was performed in a dataset with matched images and transcriptome data.

      Results

      Radiomic signature consisting of three features was established and then validated in the validation dataset (log-rank P = 0.0047) including 140 patient, and showed a significant predictive power in two independent datasets totaling 395 NSCLC patients with binary 2-year survival endpoint. Furthermore, the novel proposed radiomic nomogram significantly improved the prognostic performance (concordance index) of clinicopathological factors. Radiogenomics analysis linked our signature with important tumor biological processes (e.g. Mismatch repair, Cell adhesion molecules and DNA replication) associated with clinical outcomes.

      Conclusions

      The radiomic signature, reflecting tumor biological processes, could noninvasively predict therapeutic efficacy of NSCLC patients receiving radiotherapy and demonstrate unique advantage for clinical application.

      Keywords

      1. Introduction

      Lung cancer is the most common type of cancer worldwide and a leading cause of death in both men and women [
      • Siegel R.L.
      • Miller K.D.
      • Fuchs H.E.
      • Jemal A.
      Cancer Statistics, 2021.
      ], with non-small cell lung cancer (NSCLC) accounts for 80 % to 85 % of cases [
      • Bade B.C.
      • Dela Cruz C.S.
      Lung Cancer 2020: Epidemiology, Etiology, and Prevention.
      ]. More than approximately 16 % of patients present with early stage (T1–T2, N0) disease at diagnosis [
      • Postmus P.E.
      • Kerr K.M.
      • Oudkerk M.
      • Senan S.
      • Waller D.A.
      • Vansteenkiste J.
      • et al.
      Early and locally advanced non-small-cell lung cancer (NSCLC): ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up.
      ]. Stereotactic body radiotherapy (SBRT) [
      • Timmerman R.
      • Paulus R.
      • Galvin J.
      • Michalski J.
      • Straube W.
      • Bradley J.
      • et al.
      Stereotactic body radiation therapy for inoperable early stage lung cancer.
      ,
      • Dissaux G.
      • Visvikis D.
      • Da-Ano R.
      • Pradier O.
      • Chajon E.
      • Barillot I.
      • et al.
      Pretreatment (18)F-FDG PET/CT Radiomics Predict Local Recurrence in Patients Treated with Stereotactic Body Radiotherapy for Early-Stage Non-Small Cell Lung Cancer: A Multicentric Study.
      ] uses stereotactic targeting to facilitate the accurate delivery of a short course of high-dose radiation to the target [
      • Shinde A.
      • Li R.
      • Kim J.
      • Salgia R.
      • Hurria A.
      • Amini A.
      Stereotactic body radiation therapy (SBRT) for early-stage lung cancer in the elderly.
      ]. SBRT has demonstrated high local control rates (85 %–90 %) comparable to those obtained with surgery in multiple prospective trials [
      • Chang J.Y.
      • Senan S.
      • Paul M.A.
      • Mehran R.J.
      • Louie A.V.
      • Balter P.
      • et al.
      Stereotactic ablative radiotherapy versus lobectomy for operable stage I non-small-cell lung cancer: a pooled analysis of two randomised trials.
      ] and is now a guideline-recommended treatment for patients with early stage NSCLC who are medically unfit or unwilling to undergo surgery [
      • Schneider B.J.
      • Daly M.E.
      • Kennedy E.B.
      • Antonoff M.B.
      • Broderick S.
      • Feldman J.
      • et al.
      Stereotactic Body Radiotherapy for Early-Stage Non-Small-Cell Lung Cancer: American Society of Clinical Oncology Endorsement of the American Society for Radiation Oncology Evidence-Based Guideline.
      ,
      • Luo Y.
      • Jolly S.
      • Palma D.
      • Lawrence T.S.
      • Tseng H.H.
      • Valdes G.
      • et al.
      A situational awareness Bayesian network approach for accurate and credible personalized adaptive radiotherapy outcomes prediction in lung cancer patients.
      ]. However, therapeutic results of radiotherapy are nonetheless highly variable among these patients, and it is imperative to develop a clinically feasible signature to stratify patients who might benefit from radiotherapy, avoiding the side effects of unnecessary treatment.
      Medical imaging can be used to noninvasively and cost-effectively visualize the characteristics of entire tumour, providing dynamic information that can be used to monitor the occurrence and development of tumors [
      • Aerts H.J.
      The Potential of Radiomic-Based Phenotyping in Precision Medicine: A Review.
      ,
      • Seijo L.M.
      • Peled N.
      • Ajona D.
      • Boeri M.
      • Field J.K.
      • Sozzi G.
      • et al.
      Biomarkers in Lung Cancer Screening: Achievements, Promises, and Challenges.
      ]. Currently, computed tomography (CT), which is the most commonly used imaging modality in oncology, especially for lung cancer, allows the non-invasive detection of tissue density and describes tumor spatial heterogeneity [
      • Choi E.R.
      • Lee H.Y.
      • Jeong J.Y.
      • Choi Y.L.
      • Kim J.
      • Bae J.
      • et al.
      Quantitative image variables reflect the intratumoral pathologic heterogeneity of lung adenocarcinoma.
      ,
      • Tamponi M.
      • Crivelli P.
      • Montella R.
      • Sanna F.
      • Gabriele D.
      • Poggiu A.
      • et al.
      Exploring the variability of radiomic features of lung cancer lesions on unenhanced and contrast-enhanced chest CT imaging.
      ]. Radiomics converts medical images into high-throughput quantitative features; this is a new field that could be the vanguard of precision medicine [
      • Aerts H.J.
      • Velazquez E.R.
      • Leijenaar R.T.
      • Parmar C.
      • Grossmann P.
      • Carvalho S.
      • et al.
      Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
      ,
      • Lambin P.
      • Leijenaar R.T.H.
      • Deist T.M.
      • Peerlings J.
      • de Jong E.E.C.
      • van Timmeren J.
      • et al.
      Radiomics: the bridge between medical imaging and personalized medicine.
      ,
      • Liu Y.
      • Qi H.
      • Wang C.
      • Deng J.
      • Tan Y.
      • Lin L.
      • et al.
      Predicting Chemo-Radiotherapy Sensitivity With Concordant Survival Benefit in Non-Small Cell Lung Cancer via Computed Tomography Derived Radiomic Features.
      ], which offers the possibility to minimize adverse effects and optimize the efficacy of treatments [
      • Trebeschi S.
      • Drago S.G.
      • Birkbak N.J.
      • Kurilova I.
      • Calin A.M.
      • Delli Pizzi A.
      • et al.
      Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers.
      ]. In order to provide support in patient management and achieve maximum clinical benefit, the development of CT-derived radiomic signature for predicting the patients response to radiotherapy needs to be assessed to predict the therapeutic benefit of radiotherapy [
      • Walls G.M.
      • Osman S.O.S.
      • Brown K.H.
      • Butterworth K.T.
      • Hanna G.G.
      • Hounsell A.R.
      • et al.
      Radiomics for Predicting Lung Cancer Outcomes Following Radiotherapy: A Systematic Review.
      ].
      In this study, using CT images of patients, we developed and validated a non-invasive radiomic signature for the NSCLC patients receiving radiotherapy, which might help to accurately predict therapeutic efficacy of radiotherapy with improved 2-year survival. Finally, based on the dataset with matched CT images and gene expression profiles, we performed radiogenomics analysis to characterize the underlying biological function reflected by the radiomic signature.

      2. Materials and Methods

      2.1 Data sources

      In this study, three radiotherapy datasets of NSCLC-Radiomics (NR), HarvardRT (HAR) and Radboud (RAD) were sourced from public databases (Table 1). The inclusion criteria of the samples for radiotherapy planning were as follows: 1) available treatment-naive CT scans or CT-derived radiomic feature profiles; 2) confirmed NSCLC; 3) patients receiving radiotherapy; 4) available survival information. These details and applications of the datasets are displayed in Fig. 1. NR dataset [
      • Aerts H.J.
      • Velazquez E.R.
      • Leijenaar R.T.
      • Parmar C.
      • Grossmann P.
      • Carvalho S.
      • et al.
      Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
      ] with DICOM CT scans was downloaded from The Cancer Imaging Archive (TCIA, https://www.cancerimagingarchive.net/, 2020), including 422 patients. After applying the inclusion criteria, 421 patients with NSCLC receiving radiotherapy were preselected and divided into discovery and validation datasets based on patient identification number (pid), that is, the 281 patients whose pid wasn’t divisible by 3 were used as a RT discovery dataset to develop a radiomic signature for radiotherapy, and the remaining 140 patients were assigned to the RT validation dataset. Data regarding HAR and RAD datasets with CT-derived radiomic feature profiles were downloaded from Hosny’s study [
      • Hosny A.
      • Parmar C.
      • Coroller T.P.
      • Grossmann P.
      • Zeleznik R.
      • Kumar A.
      • et al.
      Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study.
      ], including 293 and 104 NSCLC patients receiving radiotherapy, which was used to validate 2-year survival rate of patients receiving radiotherapy.
      Table 1Baseline clinical characteristics of patients in the analysed datasets.
      RT Discovery (n = 281)RT validation (n = 140)HarvardRT (n = 291)Radboud (n = 104)NRG (n = 89)
      2-year survival (sample)281140291104
      Age (years)
      ≤ 65170 (60.5 %)80 (57.1 %)
      > 6597 (34.5 %)52 (37.1 %)
      Gender
      Female86 (30.6 %)45 (32.1 %)29 (32.6 %)
      Male195 (69.4 %)95 (67.9 %)60 (67.4 %)
      T stage
      T154 (19.2 %)39 (27.9 %)23 (25.8 %)
      T2115 (40.9 %)41 (29.3 %)41 (46.1 %)
      T334 (12.1 %)18 (12.9 %)19 (21.3 %)
      T475 (26.7 %)42 (30.0 %)3 (3.4 %)
      N stage
      N0111 (39.5 %)59 (42.1 %)59 (66.3 %)
      N114 (5.0 %)8 (5.7 %)18 (20.2 %)
      N297 (34.5 %)44 (31.4 %)7 (7.9 %)
      N356 (19.9 %)29 (20.7 %)
      Histologic subtype
      ADC34 (12.1 %)15 (10.7 %)42 (47.2 %)
      SCC102 (36.3 %)50 (35.7 %)33 (37.1 %)
      LC76 (27.0 %)38 (27.1 %)
      NOS40 (14.2 %)22 (15.7 %)12 (13.5 %)
      Note: NRG, NSCLC-Radiomics-Genomics; ADC, Adenocarcinoma; SCC, Squamous cell carcinoma; LCC, Large-cell lung carcinoma; NOS, Not otherwise specified subtype.
      Figure thumbnail gr1
      Fig. 1Flowchart of developing and validating of a computer tomography (CT)-derived radiomic signature for the patients with NSCLC receiving radiotherapy.
      To clarify the underlying biological processes associated with radiomic signature, NSCLC-Radiomics-Genomics (NRG) dataset with DICOM CT scans and matched gene expression profile was used, including 89 patients treated with curative resection alone without adjuvant therapy. Detailed for NRG dataset, the DICOM CT scans were downloaded from TCIA and the gene expression profile (GSE58661) was downloaded from Gene Expression Omnibus (GEO; http://www.ncbi.nlm.nih.gov/geo/). For GSE58661 dataset generated by Affymetrix platforms, the robust multiarray average algorithm [
      • Irizarry R.A.
      • Hobbs B.
      • Collin F.
      • Beazer-Barclay Y.D.
      • Antonellis K.J.
      • Scherf U.
      • et al.
      Exploration, normalization, and summaries of high density oligonucleotide array probe level data.
      ] (RMA) was used for pre-processing the raw data. Probe IDs were matched with Gene IDs using the corresponding platform files. For each sample, the expression measurements of all probe IDs corresponding to the same Gene ID were averaged to obtain a single measurement. Probes that did not match any Gene ID or that matched multiple Gene IDs were deleted, resulting in a total of 21,766 unique genes.

      2.2 Image segmentation and radiomic feature extraction

      Regions of interest (ROIs) of CT scans in the NR and NRG datasets were publicly available. ROIs of CT scans in the NRG dataset were delineated and modified by two experienced radiologists. 3D slicer software was used for three-dimensional manual CT image segmentation (Version 4.8.1; https://download.slicer.org/). In general, the three-dimensional radiomic features that enabled the quantification of the tumor characteristics were divided into ten groups according to the: I) Tumor intensity, II) Shape, III) Texture, IV) wavelet filters, V) Laplacian of Gaussian filters, VI) Logarithm filters, VII) Square filters, VIII) Exponential filters, IX) Gradient filters and X) Squareroot filters features. The radiomic feature extraction was performed for each CT scan with ROIs using free and open-source PyRadiomics (v2.2.0) libraries. An extraction intensity bin width was set at 25 HU and the slice thicknesses of all scans were interpolated to a voxel size of 1 × 1 × 1 mm3. The quantitative values of 1781 radiomic features were calculated according to feature definitions in the PyRadiomics documentation [
      • van Griethuysen J.J.M.
      • Fedorov A.
      • Parmar C.
      • Hosny A.
      • Aucoin N.
      • Narayan V.
      • et al.
      Computational Radiomics System to Decode the Radiographic Phenotype.
      ] (https://pyradiomics.readthedocs.io/en/latest/index.html).
      For the HAR and RAD datasets, the CT-derived radiomic feature profiles included 1004 radiomic features that were calculated using the feature definitions in Hosny’s study. Overall, 793 radiomic features (Table S1) were overlapped among the four datasets (NR, HAR, RAD and NRG), which were in compliance with identical feature definitions present in the PyRadiomics documentation and Hosny’s study, as described by the Imaging Biomarker Standardization Initiative [
      • Zwanenburg A.
      • Vallieres M.
      • Abdalah M.A.
      • Aerts H.
      • Andrearczyk V.
      • Apte A.
      • et al.
      The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.
      ].

      2.3 Construction of radiomic signature for radiotherapy

      In the RT discovery dataset, radiomic features whose quantitative values were significantly associated with 2-year survival were identified as survival associated features. To avoid over fitting, we further adopted the minimum-redundancy maximum-relevance (mRMR) algorithm implemented in the “mRMRe” R package [
      • De Jay N.
      • Papillon-Cavanagh S.
      • Olsen C.
      • El-Hachem N.
      • Bontempi G.
      • Haibe-Kains B.
      mRMRe: an R package for parallelized mRMR ensemble feature selection.
      ] on survival associated features with respect to 2-year survival to select the highest ranked 10 feature as RT-associated features.
      Next, based on the RT-associated features, we adopted the genetic algorithm (GA) using‘GA’ R package to establish an optimal predictive model, and defined it as a predictive radiomic signature for radiotherapy. The GA was implemented with a population size of 500 and a maximum number of iterations of 100; it was terminated if the optimization objective of the best subset was not improved in 10 generations, and obtained an optimum model with maximal. The score of each sample was obtained by the additive sum of features in the signature. And C-index was estimated by univariate Cox model using the sample score in each iteration. After the best combination of features was selected, C-index of discovery and validation datasets were obtained using sample scores and 2-year survival by univariate Cox model. The risk scores of samples were calculated as follows:
      Riskscore=i=1nwiFeatureValueiin


      where i represents the ith feature in the signature; wi represents the weight of the ith feature derived from Cox model; FeatureValuei represents the quantitative value of the ith feature; and n represents the number of features contained in the signature.
      The risk score of a patient was calculated by the additive sum of features in the signature, and the risk cut-off was determined by a time-dependent receiver operating characteristic (ROC) curve.

      2.4 Statistics analyses

      ComBat function harmonization method was performed to correct the radiomic features for the batch effect introduced by different scanners [
      • Orlhac F.
      • Frouin F.
      • Nioche C.
      • Ayache N.
      • Buvat I.
      Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics.
      ] using the “neuroCombat” package.
      Overall survival (OS) was defined as the time from the date of initial respective treatment for the radiotherapy to the date of death or last visit (censored). The survival data were then right-censored and the 2-year survival rate of patients was used as the end point of interest. This setup allowed for a binary 2-year survival endpoint of 1 for deceased patients and 0 for alive patients. Survival curves were estimated using Kaplan–Meier method and statistically compared using the log-rank test [
      • Bland J.M.
      • Altman D.G.
      The logrank test.
      ]. Univariate Cox regression model was used to analyze the associations between the different influencing factors and the survival rate of the patients. Multivariate Cox regression model was used to test the independent association of the radiomic signature with the survival rate after adjusting for the clinical parameters recorded in the data. Hazard ratio (HR) and 95 % confidence intervals (CIs) were generated using the Cox proportional hazards models and the C-index [
      • Harrell Jr., F.E.
      • Lee K.L.
      • Mark D.B.
      Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
      ] was also used to estimate the predictive performance of clinical factors. ROC curve was performed to evaluate the radiomic signature’s performance in predicting 2-year survival rate using the “pROC” package [
      • Heagerty P.J.
      • Lumley T.
      • Pepe M.S.
      Time-dependent ROC curves for censored survival data and a diagnostic marker.
      ]. A radiomic nomogram was constructed using multivariable linear regression analysis using the “rms” package, and its performance was evaluated based on the C-index, calibration curve, and decision curve analysis. The net reclassification improvement (NRI) [
      • Pencina M.J.
      • D'Agostino Sr., R.B.
      • D'Agostino Jr., R.B.
      • Vasan R.S.
      Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.
      ] and integrated discrimination improvement (IDI) [

      Cook NR, Paynter NP. Comments on 'Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers' by M. J. Pencina, R. B. D'Agostino, Sr. and E. W. Steyerberg. Stat Med 2012, 31(1):93-95; author reply 96-97. https://10.1002/sim.4209.

      ] values were then determined to quantify the radiomic signature’s incremental prognostic improvement using the “nricens” package.
      The time-dependent ROC curve at 2 years was used to determine the optimal cut-off of the signature using the “survivalROC” package. The nearest neighbour estimation method was used to estimate the ROC curve. The risk score of the signature corresponded to the shortest distance between the ROC curve and the point representing the 100 % true-positive rate and 0 % false-positive rate was used as the cut-off value.
      Spearman’s rank correlation was applied to evaluate the correlation between the radiomic signature and clinical parameters. The clusterProfiler R package was used to conduct the functional enrichment analysis of the genes that are correlated with the radiomic features based on the current Kyoto Encyclopaedia of Genes and Genomes (KEGG) databases, wherein a hypergeometric test was employed.
      Statistical analyses were performed using R, version 4.0.3; P values were adjusted using Benjamini–Hochberg procedure for multiple testing to control the false discovery rate (FDR). Statistical significance was defined as two-sided P < 0.05 or FDR < 0.05 for multiple testing.

      3. Results

      3.1 Identification of a predictive radiomic signature for radiotherapy

      In the RT discovery dataset comprising 281 NSCLC patients receiving radiotherapy, we focused on analyzing the 793 overlapped features among four datasets (See Materials and Methods). After harmonization using ComBat, 244 radiomic features which were potentially significantly associated with 2-year survival were extracted (Univariate Cox regression, FDR < 0.05). Using mRMR algorithm, we further extracted 10 non-redundant, highly informative ranked set of complementary features with respect to 2-year survival as RT-associated radiomic features. The RT-associated features were selected as inputs for GA model to generate a radiomic signature consisting of three weighted features (denoted as 3-RFS). The weighted sum of these three radiomic features gave a risk score for each sample (Table S2). Probability density diagrams of homogeneity before and after ComBat harmonization for the features distribution of 3-RFS are displayed in Figure S1.
      Time-dependent ROC curve analysis determined an optimal cut-off (-0.0586, Figure S2) for placing samples, wherein if the risk score is more than cut-off, the sample would be classified into high-risk group, otherwise low-risk group; the predicted high- and low-risk patients had significantly 2-year survival rate differences in the RT discovery dataset (high-risk vs low-risk = 145: 136, log-rank P = 1.76E-06, HR = 2.06, 95 % CIs: 1.52–2.79, C-index = 0.59, Fig. 2A). The time-dependent ROC curve of the 3-RFS in predicting the 2-year survival rate is shown in Fig. 2B, and the AUC was 0.67. In the multivariable Cox regression model, 3-RFS remained significantly associated with 2-year survival (P = 2.30E-05, HR = 2.04, 95 % CI: 1.47–2.84, Fig. 2C) after adjusting TNM stage, age, gender and histologic subtype.
      Figure thumbnail gr2
      Fig. 2Survival analysis for the patients with NSCLC receiving radiotherapy in the RT discovery dataset based on 3-RFS. (A) Kaplan–Meier curves of 2-year survival for the 281 patients. (B) The receiver operating characteristic (ROC) curve of 3-RFS in predicting 2-year survival rate. (C) Multivariate Cox analyses of 3-RFS after adjusting the clinical factors.

      3.2 Validation of the radiomic signature

      The predictive performance of 3-RFS were effectively validated in RT validation dataset, consisting of 140 NSCLC patients receiving radiotherapy. According to the trained cutoff (-0.0586) of 3-RFS, the patients were divided into high- and low-risk groups with significantly different 2-year survival rate (high-risk vs low-risk = 66: 74, log-rank P = 0.0047, HR = 1.95, 95 % CIs: 1.22–3.11, C-index = 0.58, Fig. 3A). The time-dependent ROC curve confirmed that 3-RFS had a good performance for predicting 2-year survival rates (Fig. 3B), and the AUC was 0.67. Multivariate Cox analysis revealed that 2-year survival was independently predicted by 3-RFS (P = 0.0063, HR = 2.15, 95 % CI: 1.24–3.72, Fig. 3C) after adjusting TNM stage, age, gender and histologic subtype. Subsequently, the predictive performance of 3-RFS was effectively validated in two independent public datasets consisting of 291 and 104 NSCLC patients receiving radiotherapy with binary 2-year survival endpoint. The results showed that 3-RFS had a significant predictive power in predicting 2-year survival rate in the HAR (AUC = 0.62, Fig. 3D) and RAD datasets (AUC = 0.62, Fig. 3E), respectively.
      Figure thumbnail gr3
      Fig. 3Validation of 3-RFS. (A) Kaplan–Meier curves of 2-year survival for the 140 patients in the RT validation dataset. (B) The ROC curve of 3-RFS in predicting 2-year survival rate in the RT validation dataset. (C) Multivariate Cox analyses of 3-RFS after adjusting the clinical factors in the RT validation dataset. (D) The ROC curve of 3-RFS in predicting 2-year survival rate in the HarvardRT dataset. (E) The ROC curve of 3-RFS in predicting 2-year survival rate in the Radboud dataset.
      Furthermore, based on NR dataset, the subgroup analyses showed that radiomic signature could predict the 2-year survival of the patients without lymph node metastasis (N0 stage: high-risk vs low-risk = 86: 84, log-rank P = 0.0035, HR = 1.84, 95 % CIs: 1.21–2.77, C-index = 0.58, AUC = 0.67, Fig. 4A and S3A), and of the patients with lymph node metastasis (integrated N1, N2 and N3 stages: high-risk vs low-risk = 124: 124, log-rank P = 3.86E-06, HR = 2.11, 95 % CIs: 1.53–2.91, C-index = 0.59, AUC = 0.66, Fig. 4B and S3B). Similar results were observed for the T stage analysis; that is, the high-risk patients exhibited significantly shorter 2-year survival than the low-risk patients in the early T stage (integrated T1 and T2 stages: high-risk vs low-risk = 100: 149, log-rank P = 9.37E-08, HR = 2.39, 95 % CIs: 1.72–3.33, C-index = 0.61, AUC = 0.69; Fig. 4C and S3C) and the advanced T stage (integrated T3 and T4 stages: high-risk vs low-risk = 110: 59, log-rank P = 0.0136, HR = 1.70, 95 % CIs: 1.11–3.62, C-index = 0.55, AUC = 0.64, Fig. 4D and S3D).
      Figure thumbnail gr4
      Fig. 4Kaplan–Meier curves of subgroup analyses in the NSCLC-Radiomics dataset. (A) Kaplan-Meier curves of 2-year survival for patients in the lymph node non-metastasis (N0, n = 170). (B) Kaplan-Meier curves of 2-year survival for patients in the lymph node metastasis (integrated N1, N2 and N3, n = 248). (C) Kaplan-Meier curves of 2-year survival for patients with early T stage disease (integrated T1 and T2, n = 249). (D) Kaplan-Meier curves of 2-year survival for patients with advanced T stage disease (integrated T3 and T4, n = 169).

      3.3 Incremental value of the radiomic signature

      To further investigate whether 3-RFS could provide incremental value for therapeutic evaluation of NSCLC patients receiving radiotherapy, we generated a radiomic nomogram (Fig. 5A) that incorporated clinical factors (TNM stage, age, gender and histologic subtype) and 3-RFS. The radiomic nomogram showed a significantly higher C-index relative to that of the clinical nomogram (Fig. S4A) and 3-RFS alone based on the NRI and IDI index (P < 0.05, Fig. S4B, C) in the RT discovery dataset (Cindex = 0.61, Table 2) and RT validation dataset (C-index = 0.64, Table 2). The calibration curves corresponding to the radiomic nomogram at 2-year survival rates showed good agreement between the estimations and the clinical outcomes in the RT discovery (Fig. 5B) and validation datasets (Fig. 5C). Furthermore, the decision curve analysis showed that the radiomic nomogram exhibited superior performance compared with the clinical nomogram across the majority of the range of reasonable threshold probabilities in the RT discovery (Fig. 5D) and validation datasets (Fig. 5E).
      Figure thumbnail gr5
      Fig. 5Radiomic nomogram and its performance for patients with NSCLC receiving radiotherapy. (A) Survival radiomic nomogram that incorporated with 3-RFS and the clinical factors trained in the RT discovery cohort (n = 281). The points of 3-RFS and the clinical factors were obtained based on the top ‘points’ bar (scale: 0 – 100). The total point was calculated by summing the two points, and a line was drawn downward to the survival axes to determine the likelihood of 2-year survival rate. (B, C) Calibration curves for the radiomic nomogram in the RT discovery and validation datasets; the diagonal gray line represents an ideal evaluation. (D, E) Decision curves for the radiomic nomogram in the RT discovery and validation datasets.
      Table 2Predictive performances of different models.
      C-index (95 % CIs)
      RT Discovery datasetRT Validation dataset
      Radomic nomogram0.61 (0.56–0.65)0.64 (0.57–0.72)
      3-RFS0.59 (0.55–0.63)0.58 (0.53–0.64)
      Clinical nomogram0.55 (0.50–0. 60)0.58 (0.51–0.66)

      3.4 Biological function of the radiomic signature

      The biological basis of the radiomic signature was evaluated in the independent NRG dataset (n = 89) with matched DICOM CT scans and gene expression profile. We first identified 1621 correlated genes, whose expression values were significantly associated with the risk scores of 3-RFS in the NRG dataset (Spearman's rank correlation, P < 0.01) and performed functional enrichment analyses for these correlated genes based on the KEGG databases. The functional enrichment analysis results showed that correlated genes were significantly enriched in 9 KEGG functional pathways (hypergeometric test, FDR < 0.05; Fig. 6A), including several functions related to DNA damage repair such as Mismatch repair [
      • Jin Y.
      • Xiao W.
      • Wang X.
      • Cui Y.
      • Li B.
      • Liu X.
      Response to toripalimab combined with radiotherapy in advanced non-small cell lung cancer-not otherwise specified: A case report.
      ] and DNA replication [
      • Lim Y.C.
      • Ensbey K.S.
      • Offenhauser C.
      • D'Souza R.C.J.
      • Cullen J.K.
      • Stringer B.W.
      • et al.
      Simultaneous targeting of DNA replication and homologous recombination in glioblastoma with a polyether ionophore.
      ]. We further identified the genes significantly correlated with each feature of 3-RFS and performed functional enrichment analysis for these genes. It was observed that 2 of the 3 features were significantly enriched in 23 KEGG functional pathways (hypergeometric test, FDR < 0.05; Fig. 6B and Table S3) including Cell adhesion molecules [
      • McGranahan N.
      • Swanton C.
      Clonal Heterogeneity and Tumor Evolution: Past, Present, and the Future.
      ], Mismatch repair and DNA replication.
      Figure thumbnail gr6
      Fig. 6Molecular characteristics associated with 3-RFS in non-small cell lung cancer. (A) Gene-enrichment analysis of correlated genes with risk score of 3-RFS based on the Kyoto Encyclopedia of Genes and Genomes (KEGG) database in the NRG dataset. (B) Gene-enrichment analysis of correlated genes with each feature of 3-RFS based on the KEGG database in the NRG dataset. (C) Molecular scores associated with the risk scores calculated by 3-RFS.The correlation was estimated by Spearman rank correlation.
      Furthermore, We investigated the association of 3-RFS with the hypoxia score, proliferation score, stemness score and immune score calculated based on mRNA expression profiles [
      • Eustace A.
      • Mani N.
      • Span P.N.
      • Irlam J.J.
      • Taylor J.
      • Betts G.N.
      • et al.
      A 26-gene hypoxia signature predicts benefit from hypoxia-modifying therapy in laryngeal cancer but not bladder cancer.
      ,
      • Miranda A.
      • Hamilton P.T.
      • Zhang A.W.
      • Pattnaik S.
      • Becht E.
      • Mezheyeuski A.
      • et al.
      Cancer stemness, intratumoral heterogeneity, and immune response across cancers.
      ,
      • Subramanian A.
      • Tamayo P.
      • Mootha V.K.
      • Mukherjee S.
      • Ebert B.L.
      • Gillette M.A.
      • et al.
      Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles.
      ,
      • Whitfield M.L.
      • George L.K.
      • Grant G.D.
      • Perou C.M.
      Common markers of proliferation.
      ] (Supplementary Methods) in the NRG dataset using spearman's rank correlation analysis (Fig. 6C). We observed that the high-risk patients predicted by 3-RFS exhibited significantly positively correlated with hypoxia score (Rho = 0.2753, P = 0.0092), proliferation score (Rho = 0.3604, P = 0.0006), stemness score (Rho = 0.3236, P = 0.0021) and significantly negatively correlated with immune score (Rho = -0.2052, P = 0.0537, Fig. 6C), suggesting high-risk patients by 3-RFS with a high grade of tumor malignancy and a low infiltration levels.

      4. Discussion

      NSCLC is a clinically heterogeneous disease with large variations in clinical outcomes even among patients with the same TNM stage [
      • Ninomiya K.
      • Arimura H.
      Homological radiomics analysis for prognostic prediction in lung cancer patients.
      ,
      • Ubaldi L.
      • Valenti V.
      • Borgese R.F.
      • Collura G.
      • Fantacci M.E.
      • Ferrera G.
      • et al.
      Strategies to develop radiomics and machine learning models for lung cancer stage and histology prediction using small data samples.
      ]. For patients with early stage NSCLC who are medically unfit or unwilling to undergo surgery, Radiotherapy is now a guideline-recommended treatment and has demonstrated high local control rates, but the treatment results still are nonetheless highly variable among these patients. It is therefore necessary to develop a novel signature to stratify patients who might benefit from radiotherapy. Radiomics is an emerging technique that converts traditional medical images into high-dimensional features, and has been widely applied in early diagnosis, prognosis and therapeutic efficacy evaluation, guiding clinicians to develop individualized treatment plans for patients.
      In this study, we established a CT-derived radiomic signature (3-RFS) based on the advantages of radiomics, which is predictive of the therapeutic efficacy in the NSCLC patients receiving radiotherapy. 3-RFS successfully stratified NSCLC patients receiving radiotherapy into the high- and low-risk groups; the 2-year survival rate of high-risk patients was 0.24, which apparently was lower than that of the low-risk patients (2-year survival rate: 0.49) in the RT discovery dataset. Subsequently, the predictive performance of 3-RFS were effectively validated in the RT validation dataset with 140 NSCLC patients (2-year survival rate: high-risk vs low-risk = 0.36 vs 0.60). Next, the predictive performance of 3-RFS also showed a significant predictive power in predicting 2-year survival rate in two independent HAR and RAD datasets totaling 395 NSCLC patients receiving radiotherapy with binary 2-year survival endpoint. According to the trained cutoff (-0.0586) of 3-RFS, the patients were divided into high-risk and low-risk groups with a significantly and marginally significantly sample distribution for 2-year survival rate in the HAR (high-risk vs low-risk = 68: 223, Fisher’s exact test, P = 0.0081) and RAD (high-risk vs low-risk = 22: 82, Fisher’s exact test, P = 0.0841) datasets, respectively.
      Subgroup analyses also supported the predictive value of 3-RFS for early-stage patients without any lymphatic metastasis (integrated T1 and T2 stages: log-rank P = 9.37E-08, C-index = 0.61, AUC = 0.69; N0 stage: log-rank P = 0.0035, C-index = 0.58, AUC = 0.67). The similar result was also observed in the predicting 2-year survival rate of the patients with lymph node metastasis disease (integrated N1, N2 and N3 stages: log-rank P = 3.86E-06, C-index = 0.59, AUC = 0.66) and advanced T stage disease (integrated T3 and T4 stages: log-rank P = 0.0136, C-index = 0.55, AUC = 0.64). Furthermore, the novel proposed radiomic nomogram could significantly improve the prognostic performance of the clinical TNM staging system (Table 2), indicating that 3-RFS could provide additional prognostic information for patients with the same clinicopathological factors. These results indicated that 3-RFS will be worthwhile to develop as a non-invasive decision-making tools for clinical application.
      The underlying biological progression of the radiomic signature for radiotherapy is favorable for clinical application. We first revealed that several known cancer-related functional processes, such as Mismatch repair, Cell adhesion molecules and DNA replication, may be reflected by the radiomic features in 3-RFS. For example, we observed that “log_sigma_3_0_mm_3D_glszm_SizeZoneNonUniformity” showed a strong negative correlation with genes enriched in “cell adhesion” and positive correlation with genes enriched in “mismatch repair” (Table S3). The feature measures the variability of size zone volumes throughout the image, represents tumour heterogeneity. Higher value represents greater tumour heterogeneity, which might reflect the low adhesion ability and high repair ability of a tumour with high invasiveness and radiotherapy resistance. Next, we found that high-risk patients identified by 3-RFS were characterized by higher hypoxia and stemness scores, and lower immune score, providing evidence that the predicted low-risk patients with higher infiltration levels and low grade of tumor malignancy might benefit from radiotherapy in the molecular mechanism.
      This study still had some limitations. First, our study proved that the radiomic signature could identify the locally advanced stage NSCLC low-risk patients receiving concurrent chemo-radiotherapy, which should be further estimated in the independent datasets. Second, the radiomic feature filters method (wavelet, Laplacian of Gaussian, Logarithm, Square, Exponential, Gradient and Squareroot filters), while maintaining a direct relationship with the original radiomic features (e.g. Tumor intensity, Texture), are not standardised by the guidance of the IBSI, because a change in the number of gray levels can affect some radiomic features. Third, although Combat harmonization method has been used to realign radiomic features computed from different CT imaging protocols in the harmonization process, whether the quantitative radiomic signatures preprocessed by normalization are suitable for clinical individualized application remains to be explored.

      5. Conclusion

      The radiomic signature developed in this study could be applied to identify patients with NSCLC, who might benefit from radiotherapy, thereby allowing clinicians to monitor the progress of patients. Therefore, this non-invasive radiomic signature should be tested in a prospective clinical trial.

      Author Contributions

      Experiment conceiving: Y.X.L and N.N.Z; Manuscript writing: Y.X.L and N.N.Z; Results interpretation: Y.X.L and N.N.Z; Experimental design and execution: Y.X.L, X.X.Z, JHL, JR and LYL; Image collection, reading, and interpretation: W.L.D and LYL; Data collection and analysis: N.N.Z and X.X.Z; Manuscript editing: all authors.

      Funding

      This work was supported by the National Undergraduate Innovation and Entrepreneurship Training Program (Grant number: 202210226055) and Basic Computer Education Research Foundation of Association of Fundamental Computing Education in Chinese Universities (Grant No: 2021-AFCEC-192).

      Ethics approval

      The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. No ethical approval or written informed consent was required for this retrospective study. The manuscript has not been published elsewhere.

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Siegel R.L.
        • Miller K.D.
        • Fuchs H.E.
        • Jemal A.
        Cancer Statistics, 2021.
        CA Cancer J Clin. 2021; 71: 7-33
        • Bade B.C.
        • Dela Cruz C.S.
        Lung Cancer 2020: Epidemiology, Etiology, and Prevention.
        Clin Chest Med. 2020; 41: 1-24
        • Postmus P.E.
        • Kerr K.M.
        • Oudkerk M.
        • Senan S.
        • Waller D.A.
        • Vansteenkiste J.
        • et al.
        Early and locally advanced non-small-cell lung cancer (NSCLC): ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up.
        Ann Oncol. 2017; 28: iv1-iv21
        • Timmerman R.
        • Paulus R.
        • Galvin J.
        • Michalski J.
        • Straube W.
        • Bradley J.
        • et al.
        Stereotactic body radiation therapy for inoperable early stage lung cancer.
        JAMA. 2010; 303: 1070-1076
        • Dissaux G.
        • Visvikis D.
        • Da-Ano R.
        • Pradier O.
        • Chajon E.
        • Barillot I.
        • et al.
        Pretreatment (18)F-FDG PET/CT Radiomics Predict Local Recurrence in Patients Treated with Stereotactic Body Radiotherapy for Early-Stage Non-Small Cell Lung Cancer: A Multicentric Study.
        J Nucl Med. 2020; 61: 814-820
        • Shinde A.
        • Li R.
        • Kim J.
        • Salgia R.
        • Hurria A.
        • Amini A.
        Stereotactic body radiation therapy (SBRT) for early-stage lung cancer in the elderly.
        Semin Oncol. 2018; 45: 210-219
        • Chang J.Y.
        • Senan S.
        • Paul M.A.
        • Mehran R.J.
        • Louie A.V.
        • Balter P.
        • et al.
        Stereotactic ablative radiotherapy versus lobectomy for operable stage I non-small-cell lung cancer: a pooled analysis of two randomised trials.
        Lancet Oncol. 2015; 16: 630-637
        • Schneider B.J.
        • Daly M.E.
        • Kennedy E.B.
        • Antonoff M.B.
        • Broderick S.
        • Feldman J.
        • et al.
        Stereotactic Body Radiotherapy for Early-Stage Non-Small-Cell Lung Cancer: American Society of Clinical Oncology Endorsement of the American Society for Radiation Oncology Evidence-Based Guideline.
        J Clin Oncol. 2018; 36: 710-719
        • Luo Y.
        • Jolly S.
        • Palma D.
        • Lawrence T.S.
        • Tseng H.H.
        • Valdes G.
        • et al.
        A situational awareness Bayesian network approach for accurate and credible personalized adaptive radiotherapy outcomes prediction in lung cancer patients.
        Phys Med. 2021; 87: 11-23
        • Aerts H.J.
        The Potential of Radiomic-Based Phenotyping in Precision Medicine: A Review.
        JAMA Oncol. 2016; 2: 1636-1642
        • Seijo L.M.
        • Peled N.
        • Ajona D.
        • Boeri M.
        • Field J.K.
        • Sozzi G.
        • et al.
        Biomarkers in Lung Cancer Screening: Achievements, Promises, and Challenges.
        J Thorac Oncol. 2019; 14: 343-357
        • Choi E.R.
        • Lee H.Y.
        • Jeong J.Y.
        • Choi Y.L.
        • Kim J.
        • Bae J.
        • et al.
        Quantitative image variables reflect the intratumoral pathologic heterogeneity of lung adenocarcinoma.
        Oncotarget. 2016; 7: 67302-67313
        • Tamponi M.
        • Crivelli P.
        • Montella R.
        • Sanna F.
        • Gabriele D.
        • Poggiu A.
        • et al.
        Exploring the variability of radiomic features of lung cancer lesions on unenhanced and contrast-enhanced chest CT imaging.
        Phys Med. 2021; 82: 321-331
        • Aerts H.J.
        • Velazquez E.R.
        • Leijenaar R.T.
        • Parmar C.
        • Grossmann P.
        • Carvalho S.
        • et al.
        Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach.
        Nat Commun. 2014; 5: 4006
        • Lambin P.
        • Leijenaar R.T.H.
        • Deist T.M.
        • Peerlings J.
        • de Jong E.E.C.
        • van Timmeren J.
        • et al.
        Radiomics: the bridge between medical imaging and personalized medicine.
        Nat Rev Clin Oncol. 2017; 14: 749-762
        • Liu Y.
        • Qi H.
        • Wang C.
        • Deng J.
        • Tan Y.
        • Lin L.
        • et al.
        Predicting Chemo-Radiotherapy Sensitivity With Concordant Survival Benefit in Non-Small Cell Lung Cancer via Computed Tomography Derived Radiomic Features.
        Front Oncol. 2022; 12832343
        • Trebeschi S.
        • Drago S.G.
        • Birkbak N.J.
        • Kurilova I.
        • Calin A.M.
        • Delli Pizzi A.
        • et al.
        Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers.
        Ann Oncol. 2019; 30: 998-1004
        • Walls G.M.
        • Osman S.O.S.
        • Brown K.H.
        • Butterworth K.T.
        • Hanna G.G.
        • Hounsell A.R.
        • et al.
        Radiomics for Predicting Lung Cancer Outcomes Following Radiotherapy: A Systematic Review.
        Clin Oncol (R Coll Radiol). 2022; 34: e107-e122
        • Hosny A.
        • Parmar C.
        • Coroller T.P.
        • Grossmann P.
        • Zeleznik R.
        • Kumar A.
        • et al.
        Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study.
        PLoS Med. 2018; 15: e1002711
        • Irizarry R.A.
        • Hobbs B.
        • Collin F.
        • Beazer-Barclay Y.D.
        • Antonellis K.J.
        • Scherf U.
        • et al.
        Exploration, normalization, and summaries of high density oligonucleotide array probe level data.
        Biostatistics. 2003; 4: 249-264
        • van Griethuysen J.J.M.
        • Fedorov A.
        • Parmar C.
        • Hosny A.
        • Aucoin N.
        • Narayan V.
        • et al.
        Computational Radiomics System to Decode the Radiographic Phenotype.
        Cancer Res. 2017; 77: e104-e107
        • Zwanenburg A.
        • Vallieres M.
        • Abdalah M.A.
        • Aerts H.
        • Andrearczyk V.
        • Apte A.
        • et al.
        The Image Biomarker Standardization Initiative: Standardized Quantitative Radiomics for High-Throughput Image-based Phenotyping.
        Radiology. 2020; 295: 328-338
        • De Jay N.
        • Papillon-Cavanagh S.
        • Olsen C.
        • El-Hachem N.
        • Bontempi G.
        • Haibe-Kains B.
        mRMRe: an R package for parallelized mRMR ensemble feature selection.
        Bioinformatics. 2013; 29: 2365-2368
        • Orlhac F.
        • Frouin F.
        • Nioche C.
        • Ayache N.
        • Buvat I.
        Validation of A Method to Compensate Multicenter Effects Affecting CT Radiomics.
        Radiology. 2019; 291: 53-59
        • Bland J.M.
        • Altman D.G.
        The logrank test.
        BMJ. 2004; 328(7447):1073
        • Harrell Jr., F.E.
        • Lee K.L.
        • Mark D.B.
        Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
        Stat Med. 1996; 15: 361-387
        • Heagerty P.J.
        • Lumley T.
        • Pepe M.S.
        Time-dependent ROC curves for censored survival data and a diagnostic marker.
        Biometrics. 2000; 56: 337-344
        • Pencina M.J.
        • D'Agostino Sr., R.B.
        • D'Agostino Jr., R.B.
        • Vasan R.S.
        Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.
        Stat Med. 2008; 27 (discussion 207–112.): 157-172
      1. Cook NR, Paynter NP. Comments on 'Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers' by M. J. Pencina, R. B. D'Agostino, Sr. and E. W. Steyerberg. Stat Med 2012, 31(1):93-95; author reply 96-97. https://10.1002/sim.4209.

        • Jin Y.
        • Xiao W.
        • Wang X.
        • Cui Y.
        • Li B.
        • Liu X.
        Response to toripalimab combined with radiotherapy in advanced non-small cell lung cancer-not otherwise specified: A case report.
        Medicine (Baltimore). 2021; 100: e27581
        • Lim Y.C.
        • Ensbey K.S.
        • Offenhauser C.
        • D'Souza R.C.J.
        • Cullen J.K.
        • Stringer B.W.
        • et al.
        Simultaneous targeting of DNA replication and homologous recombination in glioblastoma with a polyether ionophore.
        Neuro Oncol. 2020; 22: 216-228
        • McGranahan N.
        • Swanton C.
        Clonal Heterogeneity and Tumor Evolution: Past, Present, and the Future.
        Cell. 2017; 168: 613-628
        • Eustace A.
        • Mani N.
        • Span P.N.
        • Irlam J.J.
        • Taylor J.
        • Betts G.N.
        • et al.
        A 26-gene hypoxia signature predicts benefit from hypoxia-modifying therapy in laryngeal cancer but not bladder cancer.
        Clin Cancer Res. 2013; 19: 4879-4888
        • Miranda A.
        • Hamilton P.T.
        • Zhang A.W.
        • Pattnaik S.
        • Becht E.
        • Mezheyeuski A.
        • et al.
        Cancer stemness, intratumoral heterogeneity, and immune response across cancers.
        Proc Natl Acad Sci U S A. 2019; 116: 9020-9029
        • Subramanian A.
        • Tamayo P.
        • Mootha V.K.
        • Mukherjee S.
        • Ebert B.L.
        • Gillette M.A.
        • et al.
        Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles.
        Proc Natl Acad Sci U S A. 2005; 102: 15545-15550
        • Whitfield M.L.
        • George L.K.
        • Grant G.D.
        • Perou C.M.
        Common markers of proliferation.
        Nat Rev Cancer. 2006; 6: 99-106
        • Ninomiya K.
        • Arimura H.
        Homological radiomics analysis for prognostic prediction in lung cancer patients.
        Phys Med. 2020; 69: 90-100
        • Ubaldi L.
        • Valenti V.
        • Borgese R.F.
        • Collura G.
        • Fantacci M.E.
        • Ferrera G.
        • et al.
        Strategies to develop radiomics and machine learning models for lung cancer stage and histology prediction using small data samples.
        Phys Med. 2021; 90: 13-22