Psychometric Characteristics of MCQs used in Assessing Phase-II Undergraduate Medical Students of Universiti Sains Malaysia

ABSTRACT

Multiple choice questions (MCQs) are used in assessing both undergraduate and postgraduate medical students of the School of Medical Sciences (SMS), Universiti Sains Malaysia (USM). Questions that are generated by the subject specialists are vetted at departmental and central levels. MCQs used in assessing Phase-II MD students are analysed in terms of its reliability, validity and difficulty and discriminating indices. For reliability in terms of internal consistency both Spearman-Brown formula and Cronbach’s alpha were used. Difficulty and discriminating indices for the MCQs were collected from the computer generated marking sheets. Alpha reliability coefficient for internal consistency is 0.91 for both MCQ1 and MCQ2 while corrected reliability (Spearman-Brown prophecy) for MCQ1 is 0.88 and MCQ2 is 0.91. The process of generating and vetting of questions judges the face and content validity of both these MCQs. The concurrent validity for MCQ1 and MCQ2 are r=0.55, p<0.01 and r=0.69, p<0.01 respectively. Sixty percent of both MCQ1 and MCQ2 are within the difficulty index of 20% to 80%, while 34% of MCQ1 and 37% of MCQ2 have discriminating indices of ≥0.2. The MCQs have satisfactory levels of reliability and validity. A majority of the MCQs are within the acceptable level of difficulty index. A well-structured and strict central vetting process in the SMS ensures an acceptable standard of MCQs.

Keywords: