Development of a Hand Gesture Detection-Based Robot System with MediaPipe

Selamat Muslimin; Ekawati Prihatini; Tri Martin

doi:10.24014/ijaidm.v8i3.37678

Development of a Hand Gesture Detection-Based Robot System with MediaPipe

Selamat Muslimin, Ekawati Prihatini, Tri Martin

Abstract

This research presents the development of an intelligent robot that can be summoned simply by waving a hand, without the need for physical buttons or voice commands. The system utilizes MediaPipe technology to detect and recognize hand gestures in real time through a camera. When a user waves their hand toward the camera, the system processes the motion and identifies it as a signal to call the robot. Image processing is handled by a Raspberry Pi, while movement control is managed by an Arduino, which regulates the direction and speed of the motors. The robot automatically moves toward the user and stops at a certain point to wait for further confirmation. Test results show that the robot can accurately detect gestures under various lighting conditions and distances. This approach enables more natural and efficient human–robot interaction, making it well-suited for modern contactless service systems

Keywords

Gesture Detection; Hand Waving; Intelligent Robot; MediaPipe; Raspberry Pi

References

P. Purnama, “Superkritis Karbon Dioksida–Cosolvent sebagai Alternatif Media Pemrosesan Biopolimer Polilaktida dalam Menanggulangi Permasalahan Lingkungan,” Arus Lintas Indonesia Proxy untuk Prediksi Perubahan Iklim Regional dan Global (Indonesian Throughflow, ITF: A Proxy for Regional and Global Climate Prediction), pp. 70–71, 2009.

T. Engineering, “Fascicle of Management and Technological Engineering Accuracy Measurement Of The National Instrument Starter Kit,” no. May, 2015.

J. Wirtz, W. H. Kunz, and S. Paluch, “What are the Implications of Service Robots in Hospitality for Consumers ? By Jochen Wirtz , Professor of Marketing , National University of Singapore ; Werner Kunz , Professor of Marketing , University of Massachusetts ; and Stefanie Paluch , Professor o,” no. October, 2022.

S. A. Dewangga, M. Subianto, and W. Swastika, “Implementation of Hand Gesture Recognition as Smart Home Devices Controller,” vol. 06, no. 02, pp. 63–68, 2024, doi: 10.52985/insyst.v6i2.372.

M. Wameed, A. M. ALKAMACHI, and E. Erçelebi, “Tracked Robot Control with Hand Gesture Based on MediaPipe,” Al-Khwarizmi Engineering Journal, vol. 19, no. 3, pp. 56–71, 2023, doi: 10.22153/kej.2023.04.004.

M. Arif et al., “Teknik dan Multimedia Sistem Pendeteksi Tangan Berbasis Mediapipe dan OpenCV untuk Pengenalan Gerakan,” Biner : Jurnal Ilmu Komputer, vol. 2, no. 2, pp. 173–177, 2024, [Online]. Available: https://journal.mediapublikasi.id/index.php/Biner

S. N. Budiman, S. Lestanti, S. M. Evvandri, and R. K. Putri, ‘Pengenalan Gestur Gerakan Jari Untuk Mengontrol Volume Di Komputer Menggunakan Library Opencv Dan Mediapipe’, Antivirus : Jurnal Ilmiah Teknik Informatika, vol. 16, no. 2, Art. no. 2, Nov. 2022, doi: 10.35457/antivirus.v16i2.2508

F. Zhang, V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C.-L. Chang, dan M. Grundmann, "MediaPipe Hands: On-device Real-time Hand Tracking," arXiv preprint arXiv:2006.10214, Jun. 2020.

I. Irfan, M. A. Muthalib, K. Kartika, dan S. Meliala, "Pengiraan pose model manusia pada repetisi kebugaran AI pemograman Python berbasis komputerisasi," INFOTECH Journal, vol. 9, no. 1, pp. 11–19, Jan. 2023.

Pranav, D. (2022, 1 Maret). MediaPipe: The ultimate guide to video processing.LearnOpenCV.https://learnopencv.com/introduction-to-mediapipe/

R. Josyula and S. Ostadabbas, “A Review on Human Pose Estimation,” Oct. 2021, [Online]. Available:

C. Lugaresi et al., “MediaPipe: A Framework for Building Perception Pipelines,” 2019, [Online]. Available: http://arxiv.org/abs/1906.08172

Bazarevsky, V.; Grishchenko, I. Pelacakan Pose Tubuh Real-Time di Perangkat dengan MediaPipe BlazePose, Google Research. Tersedia daring: https://ai.googleblog.com/2020/08/on-device-real-time-body-pose-tracking.html

Sarafianos, S.; Boteanu, B.; Ionescu, B.; Kakadiaris, IA Estimasi pose manusia 3D: Tinjauan pustaka dan analisis kovariat. Comput. Vis. Image Underst. 2016 , 152 , 1–20.

Chen, Y.; Tian, Y.; He, M. Estimasi pose manusia monokuler: Survei metode berbasis pembelajaran mendalam. Comput. Vis. Image Underst. 2020 , 192 , 102897.

Wang, J.; Tan, S.; Zhen, X.; Xu, S.; Zheng, F.; He, Z.; Shao, L. Estimasi pose manusia 3D yang mendalam: Tinjauan. Comput. Vis. Image Underst. 2021 , 210 , 103225.

Yurtsever, MME; Eken, S. BabyPose: Dekode komunikasi non-verbal bayi secara real-time menggunakan estimasi pose berbasis video 2D. IEEE Sens. 2022 , 22 , 13776–13784.

Alam, E.; Sufian, A.; Dutta, P.; Leo, M. Sistem deteksi jatuh pada manusia berbasis penglihatan menggunakan pembelajaran mendalam: Tinjauan. Comput. Biol. Med. 2022 , 146 , 105626.

T. C. A.-S. Zulkhaidi, E. Maria, and Yulianto3, "Pengenalan Pola Bentuk Wajah dengan OpenCV," JURTI (Jurnal Rekayasa Teknologi Informasi), vol. 3, 2019.

M. W. Julius Sembiring, Vara Susilowati, Vinsensius Reinard, “Evaluasi Jarak Deteksi Antara Gestur Tangan Dan Kamera Webcam Dengan Metode Mediapipe,” Informatika dan Teknik Elektro, vol. 2, no. 2, pp. 86–92, 2023.

DOI: http://dx.doi.org/10.24014/ijaidm.v8i3.37678

Refbacks

There are currently no refbacks.

Office and Secretariat:

Big Data Research Centre
Puzzle Research Data Technology (Predatech)
Laboratory Building 1st Floor of Faculty of Science and Technology
UIN Sultan Syarif Kasim Riau

Jl. HR. Soebrantas KM. 18.5 No. 155 Pekanbaru Riau – 28293
Website: http://predatech.uin-suska.ac.id/ijaidm
Email: ijaidm@uin-suska.ac.id
e-Journal: http://ejournal.uin-suska.ac.id/index.php/ijaidm
Phone: 085275359942

Journal Indexing:

IJAIDM Stats