SMS Phishing Detection Model with Hyperparameter Optimization in Machine Learning

Abstract

Phishing is one of the growing cybersecurity threats, including through SMS, known as smishing. This research aims to build a model for SMS phishing detection using a machine learning approach optimized through hyperparameter tuning techniques. The data used is obtained from personal SMS messages collected through questionnaires, which are then labeled by information security experts. The SMS text is cleaned using Natural Language Processing (NLP) techniques and represented using the TF-IDF method. Ten classification algorithms are tested in this study: K-NN, Logistic Regression, Support Vector Machine (SVM), Decision Tree, Random Forest, AdaBoost, Bagging, ExtraTree, Gradient Boosting, and XGBoost. Hyperparameter optimization is performed using Grid Search and Optuna, and performance is evaluated using accuracy, F1-score, and ROC-AUC metrics. The results show that the SVM and Logistic Regression models performed the best, achieving accuracy up to 98.5%. Hyperparameter optimization techniques have proven effective in improving the performance of SMS phishing classification models. This research is expected to contribute to the development of accurate and efficient SMS phishing detection systems.