Facial Expression Detection of Autism Children Using ResNet-50 in Convolutional Neural Network Algorithm

Ekawati Prihatini; Selamat Muslimin; Muhammad Rizki Darmawan

doi:10.24014/ijaidm.v8i3.37755

Facial Expression Detection of Autism Children Using ResNet-50 in Convolutional Neural Network Algorithm

Ekawati Prihatini, Selamat Muslimin, Muhammad Rizki Darmawan

Abstract

Facial expression detection in children with autism presents unique challenges due to limitations in verbal communication and social responses. This study develops a Convolutional Neural Network (CNN) model using the ResNet-50 architecture to improve the recognition accuracy of five expression categories: angry, fear, sad, neutral, and happy. A dataset of 3,030 images was divided into training and testing sets (60:40), with data augmentation and hyperparameter tuning applied using the Adam optimizer. The model achieved 89% validation accuracy and 84.49% testing accuracy, along with 86.78% precision and 80.69% recall. Evaluation on 25 test images showed an 84% success rate. These results indicate that ResNet‑50 effectively extracts facial features and classifies expressions with high accuracy, demonstrating potential as a communication aid in autism therapy. Future improvements include adding more diverse training data and optimizing model parameters.

Keywords

Children Autism; Convolutional Neural Network; Facial Expression Detection; ResNet-50

Full Text:

PDF

References

C. Cahyaningtyas, C. Gudiato, M. Sari, T. Informasi, and I. Shanti Bhuana, “Deteksi Ekspresi Wajah Manusia Menggunakan Metode Convolutional Neural Network,” JURNAL FASILKOM, vol. 15, pp. 138–145, 2025.

A. Marcha, “Peran Ekspresi Wajah Dalam Mendukung Komunikasi Verbal Pada Anak-Anak Dengan Autisme,” Jurnal Ilmu Komunikasi, vol. 3, no. 2, 2024.

Y. C. Oktaviani and Y. Wahyuningsih, “Face Expression Recognizer Dengan Convolutional Neural Network Untuk Membantu Penderita Autisme Mengenali Ekspresi Wajah Seseorang,” Jurnal Informatika dan Teknik Elektro Terapan, vol. 11, no. 3, Aug. 2023, doi: 10.23960/jitet.v11i3.3108.

N. P. Novani, D. R. Salsabila, R. Aisuwarya, L. Arief, and N. Afriyeni, “Sistem Pendeteksi Gejala Awal Tantrum Pada Anak Autisme Melalui Ekspresi Wajah Dengan Convolutional Neural Network,” JITCE (Journal of Information Technology and Computer Engineering), vol. 5, no. 02, pp. 93–106, Sep. 2021, doi: 10.25077/jitce.5.02.93-106.2021.

M. Yusqi Alfan Thoriq, K. Eka Permana, I. Agustien Siradjuddin, T. Informatika, U. Trunojoyo Madura, and J. Raya Telang Kamal, “Deteksi Wajah Manusia Berbasis One Stage Detector Menggunakan Metode You Only Look Once (YOLO),” 2023. [Online]. Available: https://ejurnal.teknokrat.ac.id/index.php/teknoinfo/index

A. Wirdiani, I. K. G. Darma Putra, M. Sudarma, R. S. Hartati, and L. S. A. Lofiana, “Real-time Face Recognition System Using Deep Learning Method,” Lontar Komputer : Jurnal Ilmiah Teknologi Informasi, vol. 14, no. 1, p. 62, Oct. 2023, doi: 10.24843/lkjiti.2023.v14.i01.p06.

Syifa Zahrah, Azhar, and Musta’inul Abdi, “Sistem Deteksi Wajah Untuk Pencatatan Kehadiran Mahasiswa Di Kelas Menggunakan Metode Convolutional Neural Network,” JAISE : Journal of Artificial Intelligence and Software Engineering, 2022.

A. Lioga Seandrio, A. Hendrianto Pratomo, and M. Y. Florestiyanto, “Implementation of Convolutional Neural Network (CNN) in Facial Expression Recognition Implementasi Convolutional Neural Network (CNN) Pada Pengenalan Ekspresi Wajah,” Jurnal Informatika dan Teknologi Informasi, vol. 18, no. 2, pp. 211–221, 2021, doi: 10.31515/telematika.v18i2.4823.

R. Rafiif Amaanullah, G. Rizka Pasfica, S. Adi Nugraha, and M. Rifqi Zein, “Implementasi Convolutional Neural Network Untuk Deteksi Emosi Melalui Wajah (Implementation of Convolutional Neural Network for Emotion Detection Through Face),” 2022. [Online]. Available: https://www.kaggle.com/shivambhardwaj0101/emo

S. Bahri, R. Samsinar, and P. S. Denta, “Pengenalan Ekspresi Wajah untuk Identifikasi Psikologis Pengguna dengan Neural Network dan Transformasi Ten Crops,” vol. 5, no. 1, 2022.

T. Elizabeth, “Penerapan Convolutional Neural Network Untuk Klasifikasi Citra Ekspresi Wajah Manusia Pada MMA Facial Expression Dataset,” 2021. [Online]. Available: http://jurnal.mdp.ac.id

C. Nisa and F. Candra, “Klasifikasi Jenis Rempah-Rempah Menggunakan Algoritma Convolutional Neural Network,” MALCOM: Indonesian Journal of Machine Learning and Computer Science, vol. 4, no. 1, pp. 78–84, Dec. 2023, doi: 10.57152/malcom.v4i1.1018.

M. A. Satriawan and W. Widhiarso, “Klasifikasi Pengenalan Wajah Untuk Mengetahui Jenis Kelamin Menggunakan Metode Convolutional Neural Network,” Jurnal Algoritme, vol. 4, no. 1, pp. 43–52, 2023, doi: 10.35957/algoritme.xxxx.

P. Adi Nugroho, I. Fenriana, and R. Arijanto, “Implementasi Deep Learning Menggunakan Convolutional Neural Network ( CNN ) Pada Ekspresi Manusia,” JURNAL ALGOR, vol. 2, no. 1, 2020, [Online]. Available: https://jurnal.buddhidharma.ac.id/index.php/algor/index

A. Hendi Muhammad, “Jurnal Computer Science and Information Technology (CoSciTech) http://ejurnal.umri.ac.id/index.php/coscitech/index Deteksi Dan Klasifikasi Penyakit Daun Tomat Menggunakan ResNet-50,” vol. 6, no. 1, pp. 9–20, 2025, doi: 10.37859/coscitech.v6i1.8501.

Fathoni, R. Muthia Nashiroh, P. Salsa Anindya, R. Ely, R. Fitrah, and I. Ali, “Pengembangan Model Cnn Untuk Klasifikasi Ekspresi Wajah Dan Potensi Penerapannya Dalam Perilaku Kriminal,” JATI (Jurnal Mahasiswa Teknik Informatika), vol. 9, 2025.

W. Zhang, X. Zhang, and Y. Tang, “Facial expression recognition based on improved residual network,” IET Image Process, vol. 17, no. 7, pp. 2005–2014, May 2023, doi: 10.1049/ipr2.12743.

A. Miskow and A. Altahhan, “Emotion Recognition with Facial Attention and Objective Activation Functions,” Oct. 2024, [Online]. Available: http://arxiv.org/abs/2410.17740

Y. Wang, K. Pan, Y. Shao, J. Ma, and X. Li, “Applying a Convolutional Vision Transformer for Emotion Recognition in Children with Autism: Fusion of Facial Expressions and Speech Features,” Applied Sciences (Switzerland), vol. 15, no. 6, Mar. 2025, doi: 10.3390/app15063083.

J. Vicky, F. Ayu, and B. Julianto, “Implementasi Pendeteksi Penyakit pada Daun Alpukat Menggunakan Metode CNN,” 2023.

A. Kusuma Putra, H. Bunyamin, and K. Maranatha Jl drg Surya Sumantri No, “Pengenalan Simbol Matematika dengan Metode Convolutional Neural Network (CNN),” 2020.

Y. N. Fuadah, I. D. Ubaidullah, N. Ibrahim, F. F. Taliningsing, N. K. SY, and M. A. Pramuditho, “Optimasi Convolutional Neural Network dan K-Fold Cross Validation pada Sistem Klasifikasi Glaukoma,” ELKOMIKA: Jurnal Teknik Energi Elektrik, Teknik Telekomunikasi, & Teknik Elektronika, vol. 10, no. 3, p. 728, Jul. 2022, doi: 10.26760/elkomika.v10i3.728.

U. Sri Rahmadhani and N. Lysbetti Marpaung, “Klasifikasi Jamur Berdasarkan Genus Dengan Menggunakan Metode CNN,” vol. 8, no. 2, 2023.

L. Zhang, Y. Bian, P. Jiang, and F. Zhang, “A Transfer Residual Neural Network Based on ResNet-50 for Detection of Steel Surface Defects,” Applied Sciences (Switzerland), vol. 13, no. 9, May 2023, doi: 10.3390/app13095260.

K. L. Kohsasih et al., “Analisis Perbandingan Algoritma Convolutional Neural Network Dan Algoritma Multi-Layer Perceptron Neural Dalam Klasifikasi Citra Sampah,” 2021. [Online]. Available: http://ejournal.stmik-time.ac.id

W. Bismi and M. Qomaruddin, “Jurnal Informatika dan Rekayasa Perangkat Lunak Klasifikasi Citra Genus Panthera Menggunakan Pendekatan Deep learning Berbasis Convolutional Neural Network (CNN),” vol. 5, no. 2, pp. 172–179, 2023.

Sandy Andika Maulana, Shabrina Husna Batubara, Tasya Ade Amelia, and Yohanna Permata Putri Pasaribu, “Penerapan Metode CNN (Convolutional Neural Network) Dalam Mengklasifikasi Jenis Ubur-Ubur,” Jurnal Penelitian Rumpun Ilmu Teknik, vol. 2, no. 4, pp. 122–130, Dec. 2023, doi: 10.55606/juprit.v2i4.3084.

DOI: http://dx.doi.org/10.24014/ijaidm.v8i3.37755

Refbacks

There are currently no refbacks.

Office and Secretariat:

Big Data Research Centre
Puzzle Research Data Technology (Predatech)
Laboratory Building 1st Floor of Faculty of Science and Technology
UIN Sultan Syarif Kasim Riau

Jl. HR. Soebrantas KM. 18.5 No. 155 Pekanbaru Riau – 28293
Website: http://predatech.uin-suska.ac.id/ijaidm
Email: ijaidm@uin-suska.ac.id
e-Journal: http://ejournal.uin-suska.ac.id/index.php/ijaidm
Phone: 085275359942

Journal Indexing:

IJAIDM Stats