An empirical analysis of multi-hop retrieval and reasoning failures in Agentic RAG

Afiqa, Nurul and Ismail, Amelia Ritahani and Andi, Tri (2026) An empirical analysis of multi-hop retrieval and reasoning failures in Agentic RAG. In: 4th International Conference on Information Technology and Computing (ICITCOM 2026), 5 August 2026, Yogyakarta, Indonesia.

PDF - Published Version Restricted to Repository staff only Download (1MB) | Request a copy

Abstract

Retrieval-Augmented Generation (RAG) has become a widely adopted approach for improving Large Language Models (LLMs) by grounding responses in external knowledge sources. However, traditional RAG systems often struggle with multi-hop questions answering where information must be retrieved and combined across multiple documents. In addressing this limitation, Agentic Retrieval-Augmented Generation (Agentic RAG) introduces iterative retrieval and reasoning mechanisms that allow autonomous evidence gathering over multiple retrieval cycles, yet the underlying causes of its remaining failures are not well understood. This paper presents an automated diagnostic pipeline for the empirical analysis of retrieval and reasoning failures in Agentic RAG and evaluates the performance of Agentic RAG on the MuSiQue multi-hop question answering benchmark and compares its performance against a Traditional RAG baseline. Experimental results show that traditional RAG is dominated by retrieval failures with 44% of evaluated samples whereas Agentic RAG reduces retrieval failures to 6% but exhibits a higher proportion of reasoning-chain failures after successfully retrieving supporting evidence. Qualitative case studies demonstrate that many remaining errors arise from difficulties in identifying intermediate entities and integrating evidence across reasoning hops rather than from missing information. These findings suggest that Agentic RAG effectively mitigates retrieval bottlenecks in multi-hop question answering, though reasoning and evidence integration remain major challenges.

Item Type:	Proceeding Paper (Plenary Papers)
Uncontrolled Keywords:	Multi-hop question answering, Agentic Retrieval-Augmented Generation (RAG), reasoning failures
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology Kulliyyah of Information and Communication Technology
Depositing User:	Amelia Ritahani Ismail
Date Deposited:	19 Oct 2026 15:26
Last Update:	19 Oct 2026 15:26
Queue Number:	2026-10-Q5690
URI:	http://irep.iium.edu.my/id/eprint/132045
Indexed In:	UNSPECIFIED

Actions (login required)

View Item