Evaluating Adan vs. Adam: an analysis of optimizer performance in deep learning

Ismail, Amelia Ritahani and Azhary, Muhammad Zulhazmi Rafiqi and Hitam, Nor Azizah (2025) Evaluating Adan vs. Adam: an analysis of optimizer performance in deep learning. In: International Symposium on Intelligent Computing Systems, November 6–7, 2024, Sharjah, United Arab Emirates, November 6–7, 2024, Proceedings Conference proceedings.

PDF - Published Version Restricted to Repository staff only Download (898kB) | Request a copy

Abstract

Choosing a suitable optimization algorithm in deep learning is essential for effective model development as it significantly influences convergence speed, model performance, and the success of the train- ing process. Optimizers play an essential role in adjusting the model’s parameters to minimize errors, assisting the learning process during the model development. With various optimization algorithms available, choosing the one that best suits the deep learning model and dataset can make a substantial difference in achieving optimal results. Adaptive Moment Estimation (Adam) and Adaptive Nesterov Accelerated Gradi- ent (Adan), two well-known optimizers, are widely used in deep learning for their ability to handle large-scale data and complex models efficiently. While Adam is known for its balance between speed and reliability, Adan builds on this by incorporating momentum and lookahead mechanisms to enhance the model’s performance. However, choosing the right opti- mizer for different tasks can be challenging, as each optimizer offers var- ious advantages and disadvantages. This paper, therefore, explores the comparative effectiveness of Adam and Adan optimizers, analyzing their impact on convergence speed, model performance, and overall training success on different classifications tasks, which are image and text classifi- cations. The results show that Adam performs better initially, but prone to overfitting. On the other hand, for image classification tasks, Adan provides more consistent optimisation across extended training periods. Based on these results, this paper aims to provide insights into the strengths and limitations of each optimizer, highlighting the importance of considering task-specific requirements when selecting an optimization algorithm for deep learning models.

Item Type:	Proceeding Paper (Plenary Papers)
Uncontrolled Keywords:	Adam algorithm · Adan algorithm · Adaptive optimizer · Deep Learning
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science
Depositing User:	Amelia Ritahani Ismail
Date Deposited:	15 Apr 2025 10:38
Last Modified:	15 Apr 2025 10:38
URI:	http://irep.iium.edu.my/id/eprint/120526

Actions (login required)

View Item