Behavioral insights of SARSA-based reinforcement learning for IPv6 intrusion classification

Firman Daru, April and Maulana Hirzan, Alauddin and Attarbashi, Zainab and Fanani, Fajriannoor (2026) Behavioral insights of SARSA-based reinforcement learning for IPv6 intrusion classification. Revista de Informática Teórica e Aplicada, 33 (4). pp. 68-81. ISSN 2175-2745

PDF - Published Version Restricted to Repository staff only Download (271kB) | Request a copy

Abstract

Intrusion Detection Systems (IDS) are crucial for network security, but traditional models like signature-based and supervised learning approaches require frequent updates or retraining. This study introduces a reinforcement learning-based IDS using SARSA for IPv6 intrusion detection. The SARSA agent with epsilon 0.1 (E-0.1) performed best: optimal detection at step 11, 193,453 correct and 25,014 incorrect actions, total reward 842,195, average reward 3.855 per epoch, outperforming E-0.5 and E-0.9. Furthermore, a comparison of Q-Table exploitation between Q-Learning and SARSA indicates that both algorithms yield similar results. However, SARSA is preferable in security contexts because its more realistic approach results in a lower false alarm rate. This behavioral distinction is demonstrated by a -1.2109 value difference and a substantially lower mean state value for SARSA (4.903) compared to the more optimistic Q-Learning (16.310). To ensure robustness, the results were validated using different random seeds, and E-0.1 consistently produced stable total rewards with minimal variance. Additionally, Shannon entropy confirmed the agent's stable decision-making behavior, where E-0.1 showed faster convergence, indicating efficient learning. Altogether, these findings suggest that the SARSA-based IDS offers a more adaptive and reliable solution for intrusion detection, particularly in dynamic IPv6 environments.

Item Type:	Article (Journal)
Uncontrolled Keywords:	epsilon greedy, IPv6, intrusion classification, policy behavior analysis, SARSA algorithm
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics. Computer engineering. Computer hardware. Photoelectronic devices > TK7885 Computer engineering
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology Kulliyyah of Information and Communication Technology
Depositing User:	Dr Zainab Senan Mahmod
Date Deposited:	16 Aug 2026 22:29
Last Update:	16 Aug 2026 22:29
Queue Number:	2026-08-Q4756
URI:	http://irep.iium.edu.my/id/eprint/130811

Actions (login required)

View Item