Optimized Multilayer Perceptron for Early Lung Cancer Diagnosis: Comparative Evaluation and Feature Importance Analysis

Abstract

This study presents an optimized Multilayer Perceptron (MLP) classifier for the early diagnosis of lung cancer using structured clinical data. A dataset of 1,000 patients from the publicly available Kaggle Lung Cancer Data Repository was utilized. After comprehensive preprocessing, including the handling of missing values, encoding of categorical features, and class balancing, the data were used to train and evaluate the proposed MLP model. The model's performance was rigorously compared against both traditional classifiers, such as Support Vector Machine (SVM) and k-Nearest Neighbors (KNN), and state-of-the-art ensemble methods, including Random Forest and XGBoost. Evaluation metrics, including precision, recall, and F1-score, were reported alongside 95% confidence intervals to ensure statistical reliability. While ensemble models achieved near-perfect classification, the optimized MLP also demonstrated exceptional performance with an F1-score of 0.9897, establishing it as a highly competitive deep learning alternative. Furthermore, feature importance was analyzed using SHAP (SHapley Additive Explanations) to enhance model interpretability. The findings demonstrate that the proposed MLP-based approach is a robust, transparent, and powerful tool for classifying the risk of early-stage lung cancer.