Hands and fingers tracking for tactile graphics reading assistive device

Shamsul Bahrin, Muhammad Ikmal Hakim and Md Yusof, Hazlina and Sidek, Shahrul Na'im (2022) Hands and fingers tracking for tactile graphics reading assistive device. In: Innovative Manufacturing, Mechatronics and Materials Forum, iM3F 2021, Gambang, Pahang.

	PDF (Full Paper) - Published Version Restricted to Registered users only Download (393kB) | Request a copy
Preview	PDF (Scopus) - Supplemental Material Download (111kB) | Preview

Abstract

Awareness on the difficulties faced by the blind and visually impaired (BVI) people to read tactile graphics has caught the attention of many researchers to develop assistive devices using machine vision approach. At the moment, most techniques that have been used to detect and track hands and fingers are insufficient to support the active and complex behavior of the tactile graphics reading and exploration. This paper presents a system to track both hands and multiple fingers using MediaPipe Hands solution. A benchmark experiment has shown the accuracy of the system to detect and track for single and both hands with 93.1 and 99.9% respectively at an average speed of 20 FPS. Besides that, a preliminary design of tactile graphics reading assistive device has been introduced in this paper. By using the xy-coordinates that have been extracted from the hand landmarks detection, a specific hand gesture and conditions have been configured to allow reader to interact with the tactile graphics. The works presented have shown a promising result to be further explored for future development of tactile graphics reading assistive devicewhich support two hands and multiple fingers exploration.

Item Type:	Conference or Workshop Item (Slide Presentation)
Uncontrolled Keywords:	Visual impairment · Blindness · Tactile graphics · Tactile exploration · Hand and finger tracking system · MediaPipe
Subjects:	T Technology > T Technology (General)
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering Kulliyyah of Engineering > Department of Mechatronics Engineering
Depositing User:	Dr. Hazlina Md Yusof
Date Deposited:	06 Apr 2023 10:04
Last Modified:	06 Apr 2023 10:06
URI:	http://irep.iium.edu.my/id/eprint/104300

Actions (login required)

View Item