Entropy learning and relevance criteria for neural network pruning

Geok, See Ng and Abdul Rahman, Abdul Wahab and Shi, Daming (2003) Entropy learning and relevance criteria for neural network pruning. Internation Journal of Neural Systems, 13 (5). pp. 291-305. ISSN 0129-0657 (P), 1793-6462 (O)

PDF - Published Version Restricted to Repository staff only Download (1MB) | Request a copy

Abstract

In this paper, entropy is a term used in the learning phase of a neural network. As learning progresses, more hidden nodes get into saturation. The early creation of such hidden nodes may impair generalisation. Hence an entropy approach is proposed to dampen the early creation of such nodes by using a new computation called entropy cycle. Entropy learning also helps to increase the importance of relevant nodes while dampening the less important nodes. At the end of learning, the less important nodes can then be pruned to reduce the memory requirements of the neural network.

Item Type:	Article (Journal)
Additional Information:	6145/38198
Uncontrolled Keywords:	Entropy; entropy learning; relevance criteria; neural network pruning
Subjects:	T Technology > T Technology (General)
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science
Depositing User:	Ahmad Nazreen Mohd Shamsuri (PT)
Date Deposited:	12 Sep 2014 09:37
Last Modified:	12 Sep 2014 09:37
URI:	http://irep.iium.edu.my/id/eprint/38198

Actions (login required)

View Item