Rater reliability of a localised CEFR descriptor for assessing writing at Malaysian Higher Education

Abu Sufi, Mohd Khairul and Abu Kassim, Noor Lide and Mohd Bakri, Siti Khairunnisa (2023) Rater reliability of a localised CEFR descriptor for assessing writing at Malaysian Higher Education. In: ICEHE-2023, 6-7 November 2023, IIUM. (Unpublished)

PDF Restricted to Registered users only Download (6MB) | Request a copy

Abstract

The ability to master the English language writing has been recognised as a paramount importance at the Malaysian tertiary level education. Consequently, the Malaysian Ministry of Education introduced the CEFR-aligned English language curriculum in 2017 and the application of CEFR standards in assessing higher education students’ English language writing skill has become a ubiquitous practice in Malaysia. However, the CEFR Can Do statements only serve as a guiding principle towards a development of an assessment rubric that would conform to local Malaysian standards, whilst preserving the standards of the CEFR. This study explicates reliability of a localised CEFR assessment descriptor for writing. Using a Multi-facet Rasch Model (MFRM), a comparison between 2 sets of examiners are compared based on the raters’ severity level using the localised CEFR descriptor.10 experienced academics, as well as 3 CEFR experts from Malaysian higher educational institutions participated in semi-structured interviews, following a four-cyclical process of the language management theory. Qualitative findings revealed a positive reception of CEFR among participating academics. Despite positive feedback, the study underscores the necessity of localising CEFR within the Malaysian higher education system to align with the objectives of academics aiming to cultivate English-proficient students, leading towards the development of the localised CEFR writing descriptor. Additionally, quantitative findings of the study illuminated that as a group, there is no significance difference between the two groups of raters when assessing writing scripts using the localised CEFR descriptor. However, several raters from the pool of academics displayed a high severity level. Overall, it can be summarised that the localised CEFR descriptor for writing could be used to assess writing at higher education level, provided that rater training is conducted.

Item Type:	Proceeding Paper (Slide Presentation)
Uncontrolled Keywords:	CEFR Multi-facet Rasch Model writing assessment higher education localisation
Subjects:	L Education > LB Theory and practice of education > LB2300 Higher Education
Kulliyyahs/Centres/Divisions/Institutes (Can select more than one option. Press CONTROL button):	Centre for Languages and Pre-University Academic Development (CELPAD) > English Language Division Kulliyyah of Education
Depositing User:	Mohd Khairul Abu SUfi
Date Deposited:	07 Dec 2023 14:31
Last Modified:	07 Dec 2023 14:31
URI:	http://irep.iium.edu.my/id/eprint/108538

Actions (login required)

View Item