An efficient parallel clustering algorithm on big data using Spark

Mallik, Moksud Alam and Zulkurnain, Nurul Fariza and Nizamuddin, Mohammed Khaja and Sarkar, Rashel and Ahmed, S K Jamil (2022) An efficient parallel clustering algorithm on big data using Spark. Journal of East China University of Science and Technology, 65 (2). pp. 535-547. ISSN 1006-3080

PDF - Published Version Restricted to Repository staff only Download (553kB) | Request a copy

Abstract

Clustering is a useful tool for dealing with large amounts of data. When dealing with larger datasets, typical algorithms become inefficient. The main reason for this is that most algorithms do not support large data sets or dimensionality. Furthermore, they are only capable of handling organized data. Every second, data from numerous streams such as log files, social media, and YouTube is poured in. Because of the increasing number and variety of data on the internet, we need to refine a parallel clustering algorithm that is both efficient and effective for Big Data. There are mainly two frameworks to process big data: MapReduce and Spark. Spark is the future of the big data platform. It is 100 times faster than Map Reduce. Here we are proposing a new parallel fuzzy clustering algorithm called "An efficient parallel clustering algorithm on big data using spark" which deals with real-time processing. Proposed algorithm gives the fast and iterative data processing and eliminates the effect of batch processing.

Item Type:	Article (Journal)
Uncontrolled Keywords:	Data Clustering, Big Data, Parallel Computing, Apache Spark
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800 Electronics. Computer engineering. Computer hardware. Photoelectronic devices > TK7885 Computer engineering
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Engineering > Department of Electrical and Computer Engineering
Depositing User:	DR Nurul Fariza Zulkurnain
Date Deposited:	15 Sep 2022 13:40
Last Modified:	15 Sep 2022 13:40
URI:	http://irep.iium.edu.my/id/eprint/99923

Actions (login required)

View Item