Lightweight hand gesture recognition using MediaPipe landmark features and LSTM for embedded human–machine interaction

Khan, Toqi Tahmid and Rukmana, Ade and Sediono, Wahju (2026) Lightweight hand gesture recognition using MediaPipe landmark features and LSTM for embedded human–machine interaction. In: 2026 10th International Conference on Mechatronics Engineering (ICOM), 8-9 September 2026, Kuala Lumpur.

PDF - Published Version Restricted to Repository staff only Download (2MB) | Request a copy

Abstract

Hand gesture recognition has emerged as an effective approach for enabling natural and contactless human–machine interaction (HMI). However, many existing visionbased approaches rely on computationally intensive image-based deep learning models or require specialized sensing hardware, limiting their deployment on low-cost real-time systems. This paper presents a lightweight vision-based hand gesture recognition framework that integrates MediaPipe hand landmark detection, geometric feature extraction, long short-term memory (LSTM) classification, and Arduino-based device control. Instead of directly processing RGB images, the proposed approach represents hand gestures using a compact feature vector consisting of normalized landmark coordinates, inter-landmark distances, and joint angles. Temporal dependencies between consecutive hand movements are modeled using an LSTM network to recognize both static and dynamic gestures. A prototype system was developed using a standard RGB webcam and an Arduino microcontroller for real-time control of electronic devices and environmental monitoring. Experimental results demonstrate an overall recognition accuracy of 92.5% with inference latency below 100 ms. The model contains only 24.6K parameters, occupies 0.3 MB of storage, and achieves 12 ms inference time on a standard CPU, enabling responsive interaction suitable for real-time HMI applications. The proposed framework provides a low-cost and computationally efficient solution for gesture-based control systems in smart environments, assistive technologies, and embedded applications.

Item Type:	Proceeding Paper (Invited Papers)
Uncontrolled Keywords:	Hand gesture recognition, human–machine interaction, MediaPipe, long short-term memory (LSTM), computer vision, deep learning, Arduino, real-time control
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics. Computer engineering. Computer hardware. Photoelectronic devices T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics. Computer engineering. Computer hardware. Photoelectronic devices > TK7885 Computer engineering
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering > Department of Mechatronics Engineering Kulliyyah of Engineering
Depositing User:	Dr.-Ing. Wahju Sediono
Date Deposited:	27 Oct 2026 10:57
Last Update:	27 Oct 2026 10:57
Queue Number:	2026-10-Q5819
URI:	http://irep.iium.edu.my/id/eprint/132236
Indexed In:	Google Scholar

Actions (login required)

View Item