Understanding the capabilities of data quality measurement and monitoring

Awang Abu Bakar, Normi Sham and Mohd Sabri, Alea Syaffa and Hazlan, Aisya Sufiah (2025) Understanding the capabilities of data quality measurement and monitoring. Journal of Sustainable Software Engineering and Information Systems, 1 (1). pp. 30-44. E-ISSN 0000-0000

Preview

PDF - Published Version Download (308kB) | Preview

Abstract

Background of Study: Ensuring high-quality data is essential for organizations that depend on analytics, automation, and regulatory compliance. Aims and Scope of Paper: This paper explores the foundational concepts and evolving practices of two interrelated capabilities: data quality measurement and data quality monitoring. While measurement focuses on quantifying attributes such as accuracy, completeness, consistency, and timeliness, monitoring emphasizes the continuous detection and alerting of anomalies during data operations. Methods: This paper examines the application of frameworks like Total Data Quality Management (TDQM), ISO 8000, and Data Management Association Data Management Body of Knowledge (DAMA DMBOK), alongside emerging tools such as rule-based engines, metadata-driven platforms, and AI-driven anomaly detection systems. Results: Findings reveal a persistent gap in systems that integrate both measurement and monitoring effectively, hindering long-term data governance. This paper discusses a case study of the Data Quality framework implementation in the Healthcare sector. It was found that the healthcare organization implemented the Total Data Quality Management (TDQM) framework and Apache Griffin to ensure the accuracy, completeness, consistency, timeliness, and validity of clinical and IoT data through continuous monitoring, automated validation, and anomaly detection. Governance mechanisms aligned with ISO 8000 and HIPAA standards ensured full compliance, traceability, and accountability across all data quality and auditing processes. This study contributes to a deeper understanding of how integrated data quality practices can support digital transformation and operational resilience across industries. Conclusion: The paper concludes by recommending the adoption of continuous quality measurement practices aligned with governance policies and supported by both human expertise and automation, arguing that data quality must be embedded as a dynamic and strategic function within the digital enterprise. While measurement emphasizes the quantification of data attributes such as accuracy, completeness, consistency, and timeliness, monitoring focuses on the continuous detection and alerting of data anomalies during data operations.

Item Type:	Article (Journal)
Uncontrolled Keywords:	Anomaly Detection Data Governance Data Quality Data Quality Measurement Data Quality Monitoring
Subjects:	T Technology > T Technology (General)
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology > Department of Computer Science Kulliyyah of Information and Communication Technology Kulliyyah of Information and Communication Technology
Depositing User:	Dr. Normi Sham Awang Abu Bakar
Date Deposited:	28 Oct 2025 09:10
Last Modified:	28 Oct 2025 09:10
URI:	http://irep.iium.edu.my/id/eprint/123866

Actions (login required)

View Item