Comparative evaluation of anomaly-based controller area network IDS

Sharmin, Shaila and Mansor, Hafizah and Abdul Kadir, Andi Fitriah and Abdul Aziz, Normaziah (2023) Comparative evaluation of anomaly-based controller area network IDS. In: ICSCA 2023: 2023 12th International Conference on Software and Computer Applications, 23rd - 25th February 2023, Kuantan, Malaysia.

	PDF (Full Paper) - Published Version Restricted to Registered users only Download (834kB) | Request a copy
Preview	PDF (Scopus) - Supplemental Material Download (404kB) | Preview

Abstract

The vulnerability of in-vehicle networks, particularly those based on the Controller Area Network (CAN) protocol, has prompted the development of numerous techniques for intrusion detection on the CAN bus. However, these CAN IDS are often evaluated in disparate experimental settings, with different datasets and evaluation metrics, which hinder direct comparison. This has given rise to efforts at benchmarking and comparative evaluation of CAN IDS under similar experimental conditions to provide an understanding of the relative performance of these CAN IDS. This work contributes to these efforts by reporting results of the comparative evaluation of four statistical and two machine learning-based CAN intrusion detection algorithm, against the Real ORNL Automotive Dynamometer (ROAD) CAN intrusion dataset. The ROAD dataset differs from datasets used in previous work in that it includes the stealthiest possible version of targeted ID fabrication attacks which are more difficult to detect. It also consists of masquerade attacks, which have not been commonly used in comparative evaluation studies. Furthermore, in addition to metrics such as accuracy, precision, recall, and F1-score, we report balanced accuracy, informedness, markedness, and Matthews correlation coefficient, which place equal important on positive and negative classes and are better measures of detection capability, especially for imbalanced datasets. We also report training and testing times for each CAN IDS as an indicator of real-time intrusion detection performance. Results of experiments were found to be generally concordant with previous work, in terms of accuracy, precision, recall, and F1-score. Entropy and frequency-based CAN IDS were found to be relatively better at detecting attacks, particularly fabrication attacks; while other algorithms did not perform well, as indicated by low MCC scores.

Item Type:	Conference or Workshop Item (Invited Papers)
Uncontrolled Keywords:	Security and privacy→Network security; Intrusion detection systems; Embedded systems security
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology Kulliyyah of Information and Communication Technology Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science
Depositing User:	Hafizah Mansor
Date Deposited:	04 Jul 2023 15:07
Last Modified:	21 Jul 2023 15:06
URI:	http://irep.iium.edu.my/id/eprint/105306

Actions (login required)

View Item