Hybrid vision system for enhanced situational awareness in Unmanned Surface Vehicles: Decision-Level Camera-LiDAR fusion with supervised and unsupervised approaches

Abd Alhattab, Yousef and Mohd Zaki, Hasan Firdaus and Embong, Abd Halim and Zainal Abidin, Zulkifli and Faizabadi, Ahmed Rimaz and Ibrahim, Ahmad Imran (2025) Hybrid vision system for enhanced situational awareness in Unmanned Surface Vehicles: Decision-Level Camera-LiDAR fusion with supervised and unsupervised approaches. IEEE Access, 13. pp. 75618-75634. E-ISSN 2169-3536

	PDF - Published Version Restricted to Registered users only Download (5MB) | Request a copy
Preview	PDF - Supplemental Material Download (177kB) | Preview

Abstract

The integration of LiDAR and cameras with an efficient data fusion approach significantly improves Enhanced Situational Awareness (ESA) for small-sized Unmanned Surface Vehicles (USVs). This is critical for early obstacle detection, particularly when maritime obstacles are only 5 to 10 seconds away. This is important to ensure effective collision avoidance in regions without reliable Global Positioning System (GPS) and Automatic Identification System (AIS), while maintaining low computational requirements and real-time performance. This paper presents a system aimed at ESA using an unsupervised approach and a trained deep learning model to detect multiple maritime obstacles in denied GPS zones. The proposed system incorporates a technical mechanism that uses a compact hardware system with an integrated advanced computing module and sensors for small-sized USVs. It addresses the challenges of achieving peak performance on an edge machine learning computer integration to reduce computational overhead. Further, it minimizes temporal detection variations using sophisticated filters and clustering in dynamic maritime environments with synchronized LiDAR and camera data fusion. The detection model, trained using Maritime Federated Large Dataset (MFLD2), achieved over 99% operational accuracy with the proposed data fusion approach. The system’s capacity to precisely identify obstacle location and distance is validated by experimental findings, enabling real-time situational awareness.

Item Type:	Article (Journal)
Uncontrolled Keywords:	Obstacle detection, collision avoidance, fusion system, situational awareness, unmanned surface vehicles.
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics. Computer engineering. Computer hardware. Photoelectronic devices V Naval Science > VM Naval architecture. Shipbuilding. Marine engineering
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering > Department of Mechatronics Engineering Kulliyyah of Engineering
Depositing User:	Dr. Hasan Firdaus Mohd Zaki
Date Deposited:	05 Jun 2025 14:51
Last Modified:	05 Jun 2025 14:51
URI:	http://irep.iium.edu.my/id/eprint/121357

Actions (login required)

View Item