YOLO-based scene recognition via object co-occurrence for the visually impaired

Marzuki, Muhammad Farhan and Samad, Rosdiyana and Noordin, Nurul Hazlina and Abdullah, Nor Rul Hasma and Mustafa, Mahfuzah and Pebrianti, Dwi (2026) YOLO-based scene recognition via object co-occurrence for the visually impaired. In: 2026 IEEE International Conference on Mechatronics, 8-9 September 2026, Kulliyyah of Engineering, IIUM.

PDF - Published Version Restricted to Repository staff only Download (1MB) | Request a copy

Abstract

Individuals with visual impairments face persistent difficulties in perceiving the context of their surroundings, since most assistive tools report isolated objects rather than the overall scene. This paper presents an image-based scene recognition framework, evaluated offline as a laptop simulation on static images, that combines deep learning-based object detection with a transparent, rule-based classification stage. YOLOv8m and YOLOv11m detectors were trained to recognize 16 everyday object classes associated with three scene categories: kitchen, classroom, and roadside. Detected objects are mapped to scene labels using an object co-occurrence table and predefined keyword sets, which makes the classification stage interpretable and easily extensible without retraining. Two datasets were used to quantify the effect of data quality: an initial set of 920 images with basic augmentation and an enhanced set of 6,170 images with targeted class balancing and augmentation covering flipping, rotation, hue, brightness, and blur. Across all hyperparameter configurations, YOLOv11m outperformed YOLOv8m, achieving 0.882 accuracy, 0.838 precision, 0.843 recall, and 0.8997 mAP@50 at 60 epochs with a batch size of 16 and a learning rate of 0.001, compared with 0.856 accuracy and 0.8405 mAP@50 for the best YOLOv8m configuration. The enhanced dataset, evaluated with the highercapacity medium-variant models, improved accuracy by approximately 9 percentage points over the small-variant baseline on the initial dataset; because dataset size and model capacity were increased together, this gain reflects their combined effect rather than dataset enhancement in isolation. On the initial dataset, a manual scene-level verification of all 317 images of the independent evaluation set confirmed end-to-end pipeline accuracy of 0.78–0.79. The results indicate that pairing a modern single-stage detector with lightweight co-occurrence reasoning provides accurate and explainable scene-level feedback suitable for assistive applications.

Item Type:	Proceeding Paper (Invited Papers)
Uncontrolled Keywords:	scene recognition, object detection, YOLOv8, YOLOv11, object co-occurrence, visually impaired, assistive technology
Subjects:	T Technology > T Technology (General) T Technology > T Technology (General) > T10.5 Communication of technical information
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering > Department of Mechanical Engineering Kulliyyah of Engineering
Depositing User:	Dr Dwi Pebrianti
Date Deposited:	30 Oct 2026 17:03
Last Update:	30 Oct 2026 17:03
Queue Number:	2026-10-Q5924
URI:	http://irep.iium.edu.my/id/eprint/132352
Indexed In:	Google Scholar

Actions (login required)

View Item