A Comparative Study of Feature Selection in Machine Learning for Survey-Based Lung Cancer Classification Utilizing SHAP
Downloads
Lung cancer is a fatal malignancy requiring early detection. While machine learning aids in diagnosis using survey data, existing studies frequently suffer from extreme class imbalance and methodological flaws, specifically data leakage during feature selection, leading to optimistically biased performance. This study aims to determine the most robust feature selection and classification methodology for lung cancer prediction while strictly eliminating information leakage and addressing class imbalance. Duplicate entries were removed from a 309-record survey dataset, resulting in 276 unique patients (86.23% positive, 13.77% negative). An unbiased evaluation framework was established using Nested Stratified 10-Fold Cross-Validation. Within this architecture, nine classifiers were optimized across five dynamic feature selection techniques (ANOVA + Chi-Square, Forward Selection, Backward Elimination, Mutual Information Gain, and Recursive Feature Elimination) strictly within the inner training loops. The K-Nearest Neighbors algorithm paired with Recursive Feature Elimination (KNN-RFE) achieved the highest overall predictive accuracy (91.28% ± 4.91%) and F1-score (95.02% ± 2.81%). However, Logistic Regression optimized with Backward Elimination (LR-BE) demonstrated superior stability in the presence of skewed data, achieving the highest balanced accuracy (89.95% ± 8.42%) and precision (98.25% ± 2.85%). Additionally, SHAP (SHapley Additive exPlanations) analysis was integrated, transparently identifying 'Swallowing Difficulty' and 'Coughing' as the most critical predictive biomarkers. In practice, KNN-RFE is recommended for maximizing overall diagnostic discrimination, whereas LR-BE is strictly preferred for handling severe class imbalances to minimize false positives. The rigorous nested validation ensures these metrics reflect realistic clinical generalization rather than algorithmic overfitting. Furthermore, the SHAP integration provides necessary transparency, translating complex predictive pathways into interpretable, evidence-based screening insights for healthcare professionals.
[1] M. Mamun, A. Farjana, M. Al Mamun, and M. S. Ahammed, “Lung cancer prediction model using ensemble learning techniques and a systematic review analysis,” in 2022 IEEE World AI IoT Congress, AIIoT 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 187–193. doi: https://doi.org/10.1109/AIIoT54504.2022.9817326.
[2] C. Li et al., “Global burden and trends of lung cancer incidence and mortality,” Chin. Med. J. (Engl)., vol. 136, no. 13, pp. 1583–1590, Jul. 2023, doi: https://doi.org/10.1097/cm9.0000000000002529.
[3] L. A. G. R. Ries, J. L. Young, G. E. Keel, M. P. Eisner, Y. D. Lin, and M.-J. D. Horner, “Cancer Survival Among Adults: U.S. SEER Program, 1988-2001 (low res),” Bethesda, 2007. [Online]. Available: http://www.seer.cancer.gov
[4] A. Ruano-Raviña et al., “Lung cancer symptoms at diagnosis: Results of a nationwide registry study,” ESMO Open, vol. 5, no. 6, pp. 1–7, Nov. 2020, doi: https://doi.org/10.1136/esmoopen-2020-001021.
[5] J. Kim, H. Lee, and B. W. Huang, “Lung Cancer: Diagnosis, Treatment Principles, and Screening,” Am. Fam. Physician, vol. 105, no. 5, pp. 487–494, Apr. 2022, [Online]. Available: https://www.aafp.org/afp/2022/0500/p487
[6] K. Kourou, T. P. Exarchos, K. P. Exarchos, M. V. Karamouzis, and D. I. Fotiadis, “Machine learning applications in cancer prognosis and prediction,” Comput. Struct. Biotechnol. J., vol. 13, pp. 8–17, 2015, doi: https://doi.org/10.1016/j.csbj.2014.11.005.
[7] C. Zhang and Y. Ma, Ensemble machine learning: Methods and applications, 1st ed., vol. 8. New York: Springer New York, 2012. doi: https://doi.org/10.1007/978-1-4419-9326-7.
[8] S. Pechprasarn, N. Suechoey, N. Pholtrakoolwong, P. Tanedvorapinyo, and Y. Toboonliang, “Optimizing Lung Cancer Diagnosis with Machine Learning and Feature Selection Methods,” J. Curr. Sci. Technol., vol. 14, no. 3, p. 55, Sep. 2024, doi: https://doi.org/10.59796/jcst.V14N3.2024.55.
[9] K. Mohan and B. Thayyil, “Machine Learning Techniques for Lung Cancer Risk Prediction using Text Dataset,” International Journal of Data Informatics and Intelligent Computing, vol. 2, no. 3, pp. 47–56, Sep. 2023, doi: https://doi.org/10.59461/ijdiic.v2i3.73.
[10] E. Vieira, D. Ferreira, C. Neto, A. Abelha, and J. Machado, “Data Mining Approach to Classify Cases of Lung Cancer,” in Trends and Applications in Information Systems and Technologies, Á. Rocha, H. Adeli, G. Dzemyda, F. Moreira, and A. M. Ramalho Correia, Eds., Cham: Springer International Publishing, 2021, pp. 511–521. doi: https://doi.org/10.1007/978-3-030-72657-7_49.
[11] S. Prusty, S. Patnaik, and S. K. Dash, “SKCV: Stratified K-fold cross-validation on ML classifiers for predicting cervical cancer,” Frontiers in Nanotechnology, vol. 4, p. 972421, Aug. 2022, doi: https://doi.org/10.3389/fnano.2022.972421.
[12] Z. Li, “Extracting spatial effects from machine learning model using local interpretation method: An example of SHAP and XGBoost,” Comput. Environ. Urban Syst., vol. 96, p. 101845, Sep. 2022, doi: https://doi.org/10.1016/j.compenvurbsys.2022.101845.
[13] A. M. Ali et al., “Explainable Machine Learning Approach for Hepatitis C Diagnosis Using SFS Feature Selection,” Machines, vol. 11, no. 3, p. 391, Mar. 2023, doi: https://doi.org/10.3390/machines11030391.
[14] J. Lv and L. Wang, “Hybrid modeling of adsorption process using mass transfer and machine learning techniques for concentration prediction,” Journal of Saudi Chemical Society, vol. 29, no. 4, p. 12, Jul. 2025, doi: https://doi.org/10.1007/s44442-025-00016-y.
[15] M. Shantal, Z. Othman, and A. A. Bakar, “A Novel Approach for Data Feature Weighting Using Correlation Coefficients and Min–Max Normalization,” Symmetry (Basel)., vol. 15, no. 12, p. 18, Dec. 2023, doi: https://doi.org/10.3390/sym15122185.
[16] H. Nasiri and S. A. Alavi, “A Novel Framework Based on Deep Learning and ANOVA Feature Selection Method for Diagnosis of COVID-19 Cases from Chest X-Ray Images,” Comput. Intell. Neurosci., vol. 2022, p. 4694567, 2022, doi: https://doi.org/10.1155/2022/4694567.
[17] H. Mamdouh Farghaly and T. Abd El-Hafeez, “A high-quality feature selection method based on frequent and correlated items for text classification,” Soft comput., vol. 27, no. 16, pp. 11259–11274, Aug. 2023, doi: https://doi.org/10.1007/s00500-023-08587-x.
[18] H. Nayl, E. S. Aly, A. Rezk, and M. E. Fares, “Improved chi-square feature selection for robust heart disease data classification,” AIMS Mathematics, vol. 11, no. 1, pp. 2682–2701, Jan. 2026, doi: https://doi.org/10.3934/math.2026108.
[19] S. Williamson, K. Vijayakumar, and V. J. Kadam, “Predicting breast cancer biopsy outcomes from BI-RADS findings using random forests with chi-square and MI features,” Multimed. Tools Appl., vol. 81, no. 26, pp. 36869–36889, Nov. 2022, doi: https://doi.org/10.1007/s11042-021-11114-5.
[20] M. J. Siraj, T. Ahmad, and R. M. Ijtihadie, “Analyzing ANOVA F-test and Sequential Feature Selection for Intrusion Detection Systems,” International Journal of Advances in Soft Computing and its Applications, vol. 14, no. 2, pp. 185–194, 2022, doi: https://doi.org/10.15849/IJASCA.220720.13.
[21] R. Lamba, T. Gulati, H. F. Alharbi, and A. Jain, “A hybrid system for Parkinson’s disease diagnosis using machine learning techniques,” Int. J. Speech Technol., vol. 25, no. 3, pp. 583–593, Sep. 2022, doi: https://doi.org/10.1007/s10772-021-09837-9.
[22] P. Rani, R. Kumar, N. M. O. S. Ahmed, and A. Jain, “A decision support system for heart disease prediction based upon machine learning,” J. Reliab. Intell. Environ., vol. 7, no. 3, pp. 263–275, Sep. 2021, doi: https://doi.org/10.1007/s40860-021-00133-6.
[23] P. Theerthagiri, “Predictive analysis of cardiovascular disease using gradient boosting based learning and recursive feature elimination technique,” Intelligent Systems with Applications, vol. 16, p. 200121, Nov. 2022, doi: https://doi.org/10.1016/j.iswa.2022.200121.
[24] E. Mosca, F. Szigeti, S. Tragianni, D. Gallagher, and G. Groh, “SHAP-Based Explanation Methods: A Review for NLP Interpretability,” in Proceedings of the 29th International Conference on Computational Linguistics, N. Calzolari, C.-R. Huang, H. Kim, J. Pustejovsky, K.-S. Choi, P.-M. Ryu, H.-H. Chen, L. Donatelli, H. Ji, S. Kurohashi, P. Paggio, N. Xue, S. Kim, Y. Hahm, Z. He, T. K. Lee, E. Santus, F. Bond, and S.-H. Na, Eds., Gyeongju, Republic of Korea: International Committee on Computational Linguistics, Oct. 2022, pp. 4593–4603. Accessed: Jul. 10, 2026. [Online]. Available: https://aclanthology.org/2022.coling-1.406/
[25] S. M. Lundberg et al., “From local explanations to global understanding with explainable AI for trees,” Nat. Mach. Intell., vol. 2, no. 1, pp. 56–67, Jan. 2020, doi: https://doi.org/10.1038/s42256-019-0138-9.
[26] F. H. Awad, M. M. Hamad, and L. Alzubaidi, “Robust Classification and Detection of Big Medical Data Using Advanced Parallel K-Means Clustering, YOLOv4, and Logistic Regression,” Life 2023, vol. 13, no. 3, p. 34, Mar. 2023, doi: https://doi.org/10.3390/life13030691.
[27] O. Karasoy and S. Ballı, “Spam SMS Detection for Turkish Language with Deep Text Analysis and Deep Learning Methods,” Arab. J. Sci. Eng., vol. 47, no. 8, pp. 9361–9377, Aug. 2022, doi: https://doi.org/10.1007/s13369-021-06187-1.
[28] D. Dey et al., “The proper application of logistic regression model in complex survey data: a systematic review,” BMC Med. Res. Methodol., vol. 25, no. 1, p. 18, Dec. 2025, doi: https://doi.org/10.1186/s12874-024-02454-5.
[29] R. A. Khan et al., “A novel framework for classification of two-class motor imagery EEG signals using logistic regression classification algorithm,” PLoS One, vol. 18, no. 9 September, p. e0276133, Sep. 2023, doi: https://doi.org/10.1371/journal.pone.0276133.
[30] R. Guo, D. Fu, and G. Sollazzo, “An ensemble learning model for asphalt pavement performance prediction based on gradient boosting decision tree,” International Journal of Pavement Engineering, vol. 23, no. 10, pp. 3633–3646, 2022, doi: https://doi.org/10.1080/10298436.2021.1910825.
[31] K. Gajowniczek and M. Dudziński, “Influence of Explanatory Variable Distributions on the Behavior of the Impurity Measures Used in Classification Tree Learning,” Entropy, vol. 26, no. 12, p. 35, Dec. 2024, doi: https://doi.org/10.3390/e26121020.
[32] S. Uddin, I. Haque, H. Lu, M. A. Moni, and E. Gide, “Comparative performance analysis of K-nearest neighbour (KNN) algorithm and its different variants for disease prediction,” Sci. Rep., vol. 12, no. 1, p. 6256, Dec. 2022, doi: https://doi.org/10.1038/s41598-022-10358-x.
[33] S. Ballı and E. Özdemir, “A novel method for prediction of EuroLeague game results using hybrid feature extraction and machine learning techniques,” Chaos Solitons Fractals, vol. 150, p. 111119, Sep. 2021, doi: https://doi.org/10.1016/j.chaos.2021.111119.
[34] R. Guido, S. Ferrisi, D. Lofaro, and D. Conforti, “An Overview on the Advancements of Support Vector Machine Models in Healthcare Applications: A Review,” Information (Switzerland), vol. 15, no. 4, p. 235, Apr. 2024, doi: https://doi.org/10.3390/info15040235.
[35] H. A. Salman, A. Kalakech, and A. Steiti, “Random Forest Algorithm Overview,” Babylonian Journal of Machine Learning, vol. 2024, pp. 69–79, Dec. 2024, doi: https://doi.org/10.58496/BJML/2024/007.
[36] M. Niazkar et al., “Applications of XGBoost in water resources engineering: A systematic literature review (Dec 2018–May 2023),” Environmental Modelling & Software, vol. 174, p. 105971, 2024, doi: https://doi.org/10.1016/j.envsoft.2024.105971.
[37] S. Gamil, F. Zeng, M. Alrifaey, M. Asim, and N. Ahmad, “An Efficient AdaBoost Algorithm for Enhancing Skin Cancer Detection and Classification,” Algorithms, vol. 17, no. 8, p. 353, Aug. 2024, doi: https://doi.org/10.3390/a17080353.
[38] W. Chang, X. Wang, J. Yang, and T. Qin, “An Improved CatBoost-Based Classification Model for Ecological Suitability of Blueberries,” Sensors, vol. 23, no. 4, p. 1811, Feb. 2023, doi: https://doi.org/10.3390/s23041811.
[39] S. Szeghalmy and A. Fazekas, “A Comparative Study of the Use of Stratified Cross-Validation and Distribution-Balanced Stratified Cross-Validation in Imbalanced Learning,” Sensors, vol. 23, no. 4, p. 2333, Feb. 2023, doi: https://doi.org/10.3390/s23042333.
[40] M. J. Lewis, A. Spiliopoulou, K. Goldmann, C. Pitzalis, P. McKeigue, and M. R. Barnes, “Nestedcv: An R package for fast implementation of nested cross-validation with embedded feature selection designed for transcriptomics and high-dimensional data,” Bioinformatics Advances, vol. 3, no. 1, p. 5, 2023, doi: https://doi.org/10.1093/bioadv/vbad048.
[41] M. Tahkola and Z. Guangrong, “ATSC-NEX: Automated Time Series Classification With Sequential Model-Based Optimization and Nested Cross-Validation,” IEEE Access, vol. 10, pp. 39299–39312, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3166525.
[42] A. Toufah, M. A. Kadim, and M. Y. El Hafidi, “Overcoming SVM limitations in lung cancer classification with a quantum feature map,” BMC Artificial Intelligence, vol. 1, no. 15, p. 13, Dec. 2025, doi: https://doi.org/10.1186/s44398-025-00016-3.
[43] M. Heydarian, T. E. Doyle, and R. Samavi, “MLCM: Multi-Label Confusion Matrix,” IEEE Access, vol. 10, pp. 19083–19095, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3151048.
[44] D. Chicco and G. Jurman, “A statistical comparison between Matthews correlation coefficient (MCC), prevalence threshold, and Fowlkes–Mallows index,” J. Biomed. Inform., vol. 144, p. 7, Aug. 2023, doi: https://doi.org/10.1016/j.jbi.2023.104426.
[45] P. Thölke et al., “Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data,” Neuroimage, vol. 277, p. 14, Aug. 2023, doi: https://doi.org/10.1016/j.neuroimage.2023.120253.
[46] F. M. J. M. Shamrat et al., “AlzheimerNet: An Effective Deep Learning Based Proposition for Alzheimer’s Disease Stages Classification From Functional Brain Changes in Magnetic Resonance Images,” IEEE Access, vol. 11, pp. 16376–16395, 2023, doi: https://doi.org/10.1109/ACCESS.2023.3244952.
[47] E. Richardson, R. Trevizani, J. A. Greenbaum, H. Carter, M. Nielsen, and B. Peters, “The receiver operating characteristic curve accurately assesses imbalanced datasets,” Patterns, vol. 5, no. 6, p. 11, Jun. 2024, doi: https://doi.org/10.1016/j.patter.2024.100994.
[48] O. Ismael, “The Standard Deviation Score: a novel similarity metric for data analysis,” J. Big Data, vol. 12, no. 1, p. 42, Dec. 2025, doi: https://doi.org/10.1186/s40537-025-01091-z.
[49] H. S. Darling, “Are you confident about your confidence in confidence intervals?,” Cancer Research, Statistics, and Treatment, vol. 5, no. 1, pp. 139–144, Jan. 2022, doi: https://doi.org/10.4103/crst.crst_75_22.
[50] D. Rawat, Meenakshi, L. Pawar, G. Bathla, and R. Kant, “Optimized Deep Learning Model for Lung Cancer Prediction Using ANN Algorithm,” in 2022 3rd International Conference on Electronics and Sustainable Communication Systems (ICESC), IEEE, 2022, pp. 889–894. doi: https://doi.org/10.1109/icesc54411.2022.9885607.
[51] J. Viji Cripsy, and T. Divya, “Lung Cancer Disease Prediction and Classification based on Feature Selection method using Bayesian Network, Logistic Regression, J48, Random Forest, and Naïve Bayes Algorithms,” in 2023 3rd International Conference on Smart Data Intelligence (ICSMDI), IEEE, 2023, pp. 335–342. doi: https://doi.org/10.1109/icsmdi57622.2023.00066.
[52] A. Y. Prity, A. Hossain, and S. Sultana, “FT-HELC: A Fine-tuned Hybrid Ensemble Technique for Lung Cancer Classification and Prediction Using a Clinical Dataset,” in 2025 2nd International Conference on Next-Generation Computing, IoT and Machine Learning (NCIM), IEEE, 2025, pp. 1–6. doi: https://doi.org/10.1109/ncim65934.2025.11160004.
[53] S. M. Ganie, P. K. Dutta Pramanik, and Z. Zhao, “Enhanced and Interpretable Prediction of Multiple Cancer Types Using a Stacking Ensemble Approach with SHAP Analysis,” Bioengineering, vol. 12, no. 5, Art. no. 472, 2025. doi: https://doi.org/10.3390/bioengineering12050472.
Copyright (c) 2026 Irwan Budiman, Nanda Putri Rahmawati, Annisa Rizqiana, Rafi'e Rafi'e (Author)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlikel 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).






