Metaheuristic-Based Hyperparameter Optimization Analysis of Deep Neural Network for Cross-Project Defect Prediction in Mobile Applications
Downloads
Software Defect Prediction (SDP) plays a strategic role in identifying software defects during the early stages of development, thereby enabling more efficient allocation of testing resources, particularly in the rapidly evolving mobile application domain characterized by fast release cycles. The commonly used Within-Project Defect Prediction (WPDP) approach is often constrained by the limited availability of historical data, especially in projects at early stages of development. As an alternative, Cross-Project Defect Prediction (CPDP) leverages historical data from other projects as training sources. Moreover, the performance of the Deep Neural Network (DNN) used in SDP is highly dependent on accurate hyperparameter configurations, where manual tuning requires substantial time and computational resources without guaranteeing optimal results. To address this issue, this study analyzes and compares the effectiveness of three metaheuristic algorithms, namely Particle Swarm Optimization (PSO), Genetic Algorithm (GA), and Grey Wolf Optimizer (GWO), in optimizing DNN hyperparameters within a CPDP framework. This study utilizes 14 open-source Android mobile application projects and employs the Leave-One-Out Cross-Validation technique. The performance of each combination is evaluated using ROC-AUC as the primary metric. The Wilcoxon Signed-Rank Test with a Bonferroni correction is used to assess the statistical significance of the observed performance differences. The experimental results demonstrate that GWO-DNN achieves the best performance, with an average ROC-AUC of 0.721, and is the only combination that remains statistically significant after Bonferroni correction, with a small effect size based on Cliff’s delta. Overall, the findings of this study indicate that metaheuristic-based hyperparameter tuning is a sufficiently effective approach for improving the capability of DNN in cross-project software defect prediction within the mobile application domain, although the observed improvements remain moderate.
[1] X. Cai, S. Geng, D. Wu, and J. Chen, “Unified integration of many-objective optimization algorithm based on temporary offspring for software defects prediction,” Swarm Evol. Comput., vol. 63, Jun. 2021, doi: 10.1016/j.swevo.2021.100871.
[2] C. Jin, “Cross-project software defect prediction based on domain adaptation learning and optimization,” Expert Syst. Appl., vol. 171, Jun. 2021, doi: 10.1016/j.eswa.2021.114637.
[3] K. Zhu, S. Ying, N. Zhang, and D. Zhu, “Software defect prediction based on enhanced metaheuristic feature selection optimization and a hybrid deep neural network,” Journal of Systems and Software, vol. 180, Oct. 2021, doi: 10.1016/j.jss.2021.111026.
[4] D. S. Balasubramaniam and D. S. G. Gollagi, “Software defect prediction via optimal trained convolutional neural network,” Advances in Engineering Software, vol. 169, Jul. 2022, doi: 10.1016/j.advengsoft.2022.103138.
[5] P. Y. P. Chan, J. Keung, and Z. Yang, “Identifying inconsistent software defect predictions with symmetry metamorphic relation pattern,” Journal of Systems and Software, vol. 227, Sep. 2025, doi: 10.1016/j.jss.2025.112449.
[6] F. Jiang, X. Yu, Q. Hu, J. Liu, and J. Du, “An ensemble method using neighborhood granular combination entropy for software defect prediction,” Inf. Process. Manag., vol. 63, no. 2, Mar. 2026, doi: 10.1016/j.ipm.2025.104483.
[7] S. S. Nahid, Md. S. Islam, Md. A. Hossain, M. M. Alam, and M. Shoyaib, “OCMTL: Transfer learning by orthogonal core-extraction of a matrix and its application in cross-project defect prediction,” Alexandria Engineering Journal, vol. 133, pp. 498–516, Dec. 2025, doi: 10.1016/j.aej.2025.11.039.
[8] M. Azzeh and M. J. Abdel-Rahman, “Cross-project software defects prediction using fuzzy embedding and deep learning,” Inf. Softw. Technol., vol. 190, Feb. 2026, doi: 10.1016/j.infsof.2025.107968.
[9] Y. Xing, X. Qian, Y. Guan, B. Yang, and Y. Zhang, “Cross-project defect prediction based on G-LSTM model,” Pattern Recognit. Lett., vol. 160, pp. 50–57, Aug. 2022, doi: 10.1016/j.patrec.2022.04.039.
[10] Z. M. Zain, S. Sakri, and N. H. A. Ismail, “Application of Deep Learning in Software Defect Prediction: Systematic Literature Review and Meta-analysis,” Jun. 01, 2023, Elsevier B.V. doi: 10.1016/j.infsof.2023.107175.
[11] Q. Liang, “Research on Software Defect Prediction Model based on Deep Learning,” Highlights in Science, Engineering and Technology, vol. 122, pp. 23–29, Dec. 2024, doi: 10.54097/y0w76b47.
[12] S. R. Goyal, “Effective software defect prediction with deep neural networks,” Results in Engineering, vol. 29, Mar. 2026, doi: 10.1016/j.rineng.2025.108378.
[13] Z. Feng et al., “CodeBERT: A Pre-Trained Model for Programming and Natural Languages,” Sep. 2020, [Online]. Available: http://arxiv.org/abs/2002.08155
[14] C. Pan, M. Lu, and B. Xu, “An empirical study on software defect prediction using codebert model,” Applied Sciences (Switzerland), vol. 11, no. 11, Jun. 2021, doi: 10.3390/app11114793.
[15] Y. Tay, M. Dehghani, D. Bahri, and D. Metzler, “Efficient Transformers: A Survey,” Mar. 2022, [Online]. Available: http://arxiv.org/abs/2009.06732
[16] R. Dharani and K. Danesh, “Optimized deep learning ensemble for accurate oral cancer detection using CNNs and metaheuristic tuning,” Intell. Based. Med., vol. 11, Jan. 2025, doi: 10.1016/j.ibmed.2025.100258.
[17] M. O. Miah, U. Habiba, and M. F. Kabir, “ODL-BCI: Optimal deep learning model for brain-computer interface to classify students confusion via hyperparameter tuning,” Brain Disord., vol. 13, Mar. 2024, doi: 10.1016/j.dscb.2024.100121.
[18] P. I. Kafilla, F. S. Utomo, and G. Karyono, “Enhancing Customer Purchase Behavior Prediction Using PSO-Tuned Ensemble Machine Learning Models,” Jurnal Teknik Informatika (Jutif), vol. 6, pp. 3509–3525, Oct. 2025, doi: 10.52436/1.jutif.2025.6.5.4952.
[19] F. Zito, E. G. Talbi, C. Cavallaro, V. Cutello, and M. Pavone, “Metaheuristics in automated machine learning: Strategies for optimization,” Intelligent Systems with Applications, vol. 26, Jun. 2025, doi: 10.1016/j.iswa.2025.200532.
[20] A. H. Alharbi, E. S. M. El-kenawy, F. H. Rizk, K. S. Gaber, D. S. Khafaga, and M. M. Eid, “Optimized machine learning for building energy prediction: Feature selection and hyperparameter tuning using the Al-Biruni Earth Radius (BER) search optimization algorithm,” Energy Reports, vol. 14, pp. 5505–5538, Dec. 2025, doi: 10.1016/j.egyr.2025.11.104.
[21] T. P. Nguyen, “AIoT-based indoor air quality prediction for building using enhanced metaheuristic algorithm and hybrid deep learning,” Journal of Building Engineering, vol. 105, Jul. 2025, doi: 10.1016/j.jobe.2025.112448.
[22] U. Mumtahina, S. Alahakoon, and P. Wolfs, “Hyperparameter Tuning of Load-Forecasting Models Using Metaheuristic Optimization Algorithms—A Systematic Review,” Nov. 01, 2024, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/math12213353.
[23] T. R. D. Saputri, E. Kurniawan, C. C. Lestari, and T. Antonio, “Nature-based Hyperparameter Tuning of a Multilayer Perceptron Algorithm in Task Classification: A Case Study on Fear of Failure in Entrepreneurship,” Journal of Applied Data Sciences, vol. 6, no. 2, pp. 969–980, May 2025, doi: 10.47738/jads.v6i2.539.
[24] X. Du, X. Yuan, and Z. Qiao, “Just-in-time software defect location for mobile applications based on pre-trained programming encoder model and hierarchical attention network,” Eng. Appl. Artif. Intell., vol. 148, May 2025, doi: 10.1016/j.engappai.2025.110381.
[25] M. Jorayeva, A. Akbulut, C. Catal, and A. Mishra, “Deep Learning-Based Defect Prediction for Mobile Applications,” Sensors, vol. 22, no. 13, Jul. 2022, doi: 10.3390/s22134734.
[26] X. Ju, Y. Cao, X. Chen, L. Gong, V. Chakma, and X. Zhou, “JIT-CF: Integrating contrastive learning with feature fusion for enhanced just-in-time defect prediction,” Inf. Softw. Technol., vol. 182, Jun. 2025, doi: 10.1016/j.infsof.2025.107706.
[27] G. Catolino, D. Di Nucci, and F. Ferrucci, “Cross-project just-in-time bug prediction for mobile apps: An empirical assessment,” in Proceedings - 2019 IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems, MOBILESoft 2019, Institute of Electrical and Electronics Engineers Inc., May 2019, pp. 99–110. doi: 10.1109/MOBILESoft.2019.00023.
[28] Y. Wang and L. Singh, “Analyzing the impact of missing values and selection bias on fairness,” Int. J. Data Sci. Anal., vol. 12, no. 2, pp. 101–119, Aug. 2021, doi: 10.1007/s41060-021-00259-z.
[29] E. D. Prasetyo, B. Rahmat, and A. P. Sari, “Classification Of Cyber Attack And Anomaly In Web Server Using Transformer and Transfer Learning,” Indonesian Journal of Electronics, Electromedical Engineering, and Medical Informatics, vol. 7, no. 4, pp. 713–723, Nov. 2025, doi: 10.35882/ijeeemi.v7i4.119.
[30] T. Emmanuel, T. Maupong, D. Mpoeleng, T. Semong, B. Mphago, and O. Tabona, “A survey on missing data in machine learning,” J. Big Data, vol. 8, no. 1, Dec. 2021, doi: 10.1186/s40537-021-00516-9.
[31] A. M. Sharifnia, D. E. Kpormegbey, D. K. Thapa, and M. Cleary, “A Primer of Data Cleaning in Quantitative Research: Handling Missing Values and Outliers,” J. Adv. Nurs., vol. 82, no. 1, pp. 970–975, Jan. 2026, doi: 10.1111/jan.16908.
[32] D. Kartini, R. A. Badali, M. Muliadi, D. T. Nugrahadi, F. Indriani, and S. W. Saputro, “Dimensionality Reduction Using Principal Component Analysis and Feature Selection Using Genetic Algorithm with Support Vector Machine for Microarray Data Classification,” Indonesian Journal of Electronics, Electromedical Engineering, and Medical Informatics, vol. 7, Mar. 2025, doi: 10.35882/mr7x9713.
[33] H. Henderi, “Comparison of Min-Max normalization and Z-Score Normalization in the K-nearest neighbor (kNN) Algorithm to Test the Accuracy of Types of Breast Cancer,” IJIIS: International Journal of Informatics and Information Systems, vol. 4, pp. 13–20, Mar. 2021, doi: 10.47738/ijiis.v4i1.73.
[34] G. Giray, K. E. Bennin, Ö. Köksal, Ö. Babur, and B. Tekinerdogan, “On the use of deep learning in software defect prediction,” Journal of Systems and Software, vol. 195, Jan. 2023, doi: 10.1016/j.jss.2022.111537.
[35] Z. Allen-Zhu and Y. Li, “Backward Feature Correction: How Deep Learning Performs Deep (Hierarchical) Learning,” Jul. 2023, [Online]. Available: http://arxiv.org/abs/2001.04413
[36] H. Sun, H. Zhang, M. Song, S. Li, and Y. Lu, “Automatic Fresnel zone picking in the dip-angle domain using deep neural networks,” Journal of Geophysics and Engineering, vol. 16, no. 1, pp. 136–145, Feb. 2019, doi: 10.1093/jge/gxy012.
[37] M.-W. Li, Y.-T. Wang, S.-J. Yan, J. Geng, and X.-H. Wang, “An optimization method for tidal current turbine array layout based on particle swarm optimization,” Energy, vol. 348, p. 140583, Apr. 2026, doi: 10.1016/j.energy.2026.140583.
[38] S. Gupta and K. Gautam, “Enhanced cooperative learning and local search integrated particle swarm optimization for global optimization and coverage enhancement in wireless sensor network,” Swarm Evol. Comput., vol. 100, Jan. 2026, doi: 10.1016/j.swevo.2025.102262.
[39] M. A. K. Raiaan et al., “A systematic review of hyperparameter optimization techniques in Convolutional Neural Networks,” Jun. 01, 2024, Elsevier Inc. doi: 10.1016/j.dajour.2024.100470.
[40] A. Yilmaz and İ. Kuş, “General CNN model for biomedical image classification via genetic algorithm-based hyperparameter optimization,” Ain Shams Engineering Journal, vol. 17, no. 1, Jan. 2026, doi: 10.1016/j.asej.2025.103891.
[41] H. Zhi and S. Liu, “Face recognition based on genetic algorithm,” J. Vis. Commun. Image Represent., vol. 58, pp. 495–502, Jan. 2019, doi: 10.1016/j.jvcir.2018.12.012.
[42] A. M. Akbar, R. Herteno, S. W. Saputro, M. R. Faisal, and R. A. Nugroho, “Enhancing Software Defect Prediction through Hybrid Optimization for Feature Selection and Gradient Boosting Classification,” Journal of Electronics, Electromedical Engineering, and Medical Informatics, vol. 6, no. 2, pp. 169–181, Apr. 2024, doi: 10.35882/jeeemi.v6i2.388.
[43] M. Rastgou, Y. He, R. Lou, and Q. Jiang, “A comparison of metaheuristic optimizations with automated hyperparameter tuning methods in support vector machines algorithm for predicting soil water characteristic curve,” Eng. Geol., vol. 353, Jun. 2025, doi: 10.1016/j.enggeo.2025.108121.
[44] D. Veganzones, E. Séverin, and S. Ben Jabeur, “Forecasting corporate bankruptcy in imbalanced datasets using a new hybrid machine learning approach,” Res. Int. Bus. Finance, vol. 81, Jan. 2026, doi: 10.1016/j.ribaf.2025.103200.
[45] P. Nabella, R. Herteno, S. W. Saputro, M. R. Faisal, and F. Abadi, “Impact of a Synthetic Data Vault for Imbalanced Class in Cross-Project Defect Prediction,” Journal of Electronics, Electromedical Engineering, and Medical Informatics, vol. 6, no. 2, pp. 219–230, Apr. 2024, doi: 10.35882/jeeemi.v6i2.409.
[46] S. Dubey, G. Tiwari, S. Singh, S. Goldberg, and E. Pinsky, “Using machine learning for healthcare treatment planning,” Front. Artif. Intell., vol. 6, 2023, doi: 10.3389/frai.2023.1124182.
[47] D. R. Widiana, Syafiuddin, I. Sriwijayasih, I. R. Aju, Y. Praharsi, and E. Novianarenty, “Penerapan Uji Wilcoxon Signed Rank Test Untuk Menganalisis Perbedaan Nilai Test Sebelum Dan Setelah Pelatihan Digital Marketing,” Jurnal Teknologi Maritim, vol. 8, no. 2, pp. 23–32, Oct. 2025, doi: 10.35991/jtm.v8i2.73.
[48] N. Zhang, S. Ying, W. Ding, K. Zhu, and D. Zhu, “WGNCS: A robust hybrid cross-version defect model via multi-objective optimization and deep enhanced feature representation,” Inf. Sci. (N. Y)., vol. 570, pp. 545–576, Sep. 2021, doi: 10.1016/j.ins.2021.05.008.
[49] K. Zhao, Z. Xu, T. Zhang, and Y. Tang, “Simplified Deep Forest Model based Just-In-Time Defect Prediction for Android Mobile Apps,” pp. 222–222, Dec. 2020, doi: 10.1109/QRS51102.2020.00039.
Copyright (c) 2026 Maulana Abdul Rahman, Rudy Herteno, Radityo Adi Nugroho, Friska Abadi, Setyo Wahyu Saputro (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlikel 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
