Enhanced automated vehicle classification with an optimised auto-labelling pipeline

Tashfin, Muqtasid and Hanizam, Syahirah and Mohd Zaki, Hasan Firdaus and Faizabadi, Ahmed Rimaz and Abdul Rahman, Hasbullah (2026) Enhanced automated vehicle classification with an optimised auto-labelling pipeline. In: 2026 10th International Conference on Mechatronics Engineering (ICOM), 8-9 September 2026, Kuala Lumpur.

PDF - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

Modern Intelligent Transportation Systems (ITS) are hindered by the laborious and costly manual labelling required for deep learning-based traffic analysis. This study presents an automated vehicle classification pipeline that combines a custom YOLOv5 detector with a dual vision-language model framework (Qwen2.5 VL 3B and Gemma 3 4B) guided by a 16-class vehicle taxonomy, producing structured JSON outputs. Evaluation in this study covers the category assignment only; the additional attribute fields are produced but not assessed. Both models are adapted by QLoRA fine-tuning on 6,500 images annotated through knowledge distillation from a frontier teacher model. A weighted similarity score between the two model outputs acts as a confidence signal, allowing low-agreement cases to be routed to human review. Evaluated on 1,300 real-world highway images annotated to the Malaysian Highway’s six-class scheme, the fine-tuned pipeline raised mAP from 0.450 to 0.726 at full coverage, a 61% relative gain over the single-VLM baseline. Treating the agreement score as a selective prediction mechanism raises mAP further to 0.815, at the cost of deferring 65% of cases to human review. The detection stage achieved an F1-score of 0.9620. Fine-tuning also improved structural compliance, raising the number of parseable JSON outputs from 1,177 to 1,258. Future work may integrate a VLM judge and RAG, optimize detection of distant vehicles, and carry out end-to-end model training.

Item Type:	Proceeding Paper (Plenary Papers)
Uncontrolled Keywords:	Automated Vehicle Classification , Intelligent Transportation Systems , Deep Learning , Vision-Language Models , Fine-Grained Classification , Parameter-Efficient Fine-Tuning
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering > Department of Mechatronics Engineering
Depositing User:	Dr. Hasan Firdaus Mohd Zaki
Date Deposited:	03 Nov 2026 17:00
Last Update:	03 Nov 2026 17:00
Queue Number:	2026-10-Q5957
URI:	http://irep.iium.edu.my/id/eprint/132406
Indexed In:	Google Scholar

Actions (login required)

View Item