Big data processing using Hadoop HDFS and map-reduce for Public Open Data (POD)

Muhamad Ibrahim, Najhan and Idris, Norbik Bashah and Hassan, Mohd Khairul Azmi and Breathnach, Ciara and Amir Hussin, Amir 'Aatieff (2021) Big data processing using Hadoop HDFS and map-reduce for Public Open Data (POD). Journal of Engineering Science and Technology, 16 (Special Issue on ACSAT, December 2021). pp. 1-11. ISSN 1823-4690 (In Press)

PDF - Published Version Restricted to Registered users only Download (846kB) | Request a copy

Abstract

Information acquired and processed by government agencies and made available to the public via a web portal is known as Public Open Data. Many governments have created Public Open Data (POD) to promote government transparency by making data available to the public. Malaysia is no exception, having opened public-access data since 2014. Processing large amounts of data may become increasingly complex as the availability of data sources, data types, data volume, speed, and complexity grow. As a result, it's important to consolidate and find a new method for handling and optimizing massive amounts of POD. With the development of Big Data, computer system overflow has raised some problems. As a result, industry, researchers, and government have entered into a number of comprehensive research and development cooperative agreements. While the large volume of data and variety of data formats can make Big Data processing challenges, it also has the potential to produce innovative solutions with a wider range of applications. There's also a lot of debate over whether or not Big Data can supplant traditional data records. This study aims to investigate the potentials of big data processing utilizing Apache Hadoop and Java map/reduces in the future to discover valuable patterns, correlations, trend preferences, and other important information. With improved the future prediction and prevention mechanisms in place by authorities, the overall cost of government for public service would be reduced.

Item Type:	Article (Journal)
Uncontrolled Keywords:	Apache Hadoop, Big data processing, HDFS, Map-reduce, Public open data.
Subjects:	T Technology > T Technology (General) T Technology > T Technology (General) > T10.5 Communication of technical information
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Information System Kulliyyah of Information and Communication Technology > Department of Information System
Depositing User:	Dr. Najhan Muhamad Ibrahim
Date Deposited:	24 Dec 2021 04:53
Last Modified:	24 Dec 2021 04:53
URI:	http://irep.iium.edu.my/id/eprint/95010

Actions (login required)

View Item