Design and implementation of a deep learning based hand gesture recognition system for Rehabilitation Internet-Of-Things (RIOT) environments using MediaPipe

Mohd Dhuzuki, Nurul Hanis and Zainuddin, Ahmad Anwar and Kamarul Zaman, Nur Anis Sofea and Ahmad Razmi, Alin Nur Maisarah and Kaitane, Wonderful Shammah and Ahmad Puzi, Asmarani and Johar, Mohd Naqiuddin and Yazid, Maslina and Mohd Nordin, Nor Azlin and Sidek, Shahrul Naim and Mohd Zaki, Hasan Firdaus (2025) Design and implementation of a deep learning based hand gesture recognition system for Rehabilitation Internet-Of-Things (RIOT) environments using MediaPipe. IIUM Engineering Journal, 26 (1). pp. 353-372. ISSN 1511-758X E-ISSN 2289-7860

Preview

PDF - Published Version Download (6MB) | Preview

Abstract

Frequent hospital visits for hand rehabilitation exercises, such as strengthening and opposition exercises, present significant challenges, especially for patients in remote areas. This paper addresses this problem by developing a Rehabilitation Internet-of-Things (RIOT) system that utilizes MediaPipe with its pre-trained Deep Learning (DL) to deliver real-time feedback during hand rehabilitation exercises alongside Web Assembly (WASM) for efficient processing. The system's objective is to provide precise, real-time tracking of hand movements, enabling patients to perform exercises at home by maintaining an optimal distance between the camera and hand placement, ensuring ideal room lighting conditions across IoT devices such as mobile phones' front cameras and webcams, while healthcare professionals remotely monitor their progress. The methodology involves the integration of MediaPipe for detecting hand landmarks and adaptive sensitivity algorithms to ensure reliable recognition across different environments, such as varying lighting and hand positions. Future work could incorporate additional deep-learning models like CNNs and RNNs to enhance gesture classification accuracy. Several limitations, including latency and distance sensitivity, are addressed in this system with edge computing alongside adaptive algorithms. The key contributions of this research are as follows: First, developing a real-time and cost-effective solution for remote stroke rehabilitation. Second, accuracy is improved by integrating MediaPipe with deep learning techniques. Lastly, latency issues and accuracy challenges at extended distances are alleviated by employing innovative calibration methods and adaptive adjustments. Initial trials demonstrate promising results, though further testing is required under real-world conditions to validate the system's effectiveness fully

Item Type:	Article (Journal)
Uncontrolled Keywords:	Rehabilitation Internet-of-Things (RIOT), MediaPipe, Deep Learning (DL), hand gesture recognition, Artificial Intelligence (AI)
Subjects:	T Technology > T Technology (General) > T173.2 Technological change
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science
Depositing User:	Ts.Dr. Ahmad Anwar Zainuddin
Date Deposited:	23 Jan 2025 16:45
Last Modified:	23 Jan 2025 16:45
URI:	http://irep.iium.edu.my/id/eprint/118584

Actions (login required)

View Item