Day 2 : Oral Presentations (Theme: Curriculum-Assessment)
Reliability of OSTE in the health professions education exit examination of College of Physicians and Surgeons -a psychometric analysis
Location
AKU Auditorium
Start Date
27-1-2013 9:00 AM
Abstract
Background: This study aims to assess the reliability and validity of OSTE (Objective Structured Teaching Encounters) as a performance assessment tool for the membership exit examination of HPE and to determine the difference in the examination scores of two consecutive Health Professions Education batches.
Methods: The OSTEs were designed on the basis of exam blueprints at the Department of Medical Education, College of Physicians and Surgeons Pakistan. Scores of participants of two consecutive years were selected through convenience sampling with the approval of the gatekeepers by employing correlational and comparative cross-sectional study designs. Scores of examinees (group I n= 14, group II n=15) were analyzed by using SPSS version 16. Mean + SD was computed and overall means were compared by t-test. Reliability was determined by Cronbach’s α coefficient. Standard error of measurement (SEM) was computed. OSTE stations were not uni-dimensional, hence factor analysis including varimax rotation was performed. For convergent validity (Pearson correlations) was determined between scores of OSTE, lectures delivered and workshops conducted.
Results: In-group I, overall mean score was 7.64 + 0.65 and in-group II 7.33 + 1.03, (p=0.429). In group I SEM =0.35 and group II SEM= 0.48. Factor analysis yielded four factors in group I and five factors in group II with eigen values >1. The Pearson’s correlation of average scores of OSTE and lecture was significant for group I (r = 0.61, p 0.019), and insignificant for group II (r=0.46, p= 0.08). Correlation of OSTE and workshop conducted was insignificant for group I (r= 0.51, p = 0.057) and significant in group II(r = 0.53, p =0.04).
Conclusions: OSTE is a reliable, valid assessment tool for discerning competency among learners of HPE program.
Key words: Validity, reliability, objectively structured teaching examination
Reliability of OSTE in the health professions education exit examination of College of Physicians and Surgeons -a psychometric analysis
AKU Auditorium
Background: This study aims to assess the reliability and validity of OSTE (Objective Structured Teaching Encounters) as a performance assessment tool for the membership exit examination of HPE and to determine the difference in the examination scores of two consecutive Health Professions Education batches.
Methods: The OSTEs were designed on the basis of exam blueprints at the Department of Medical Education, College of Physicians and Surgeons Pakistan. Scores of participants of two consecutive years were selected through convenience sampling with the approval of the gatekeepers by employing correlational and comparative cross-sectional study designs. Scores of examinees (group I n= 14, group II n=15) were analyzed by using SPSS version 16. Mean + SD was computed and overall means were compared by t-test. Reliability was determined by Cronbach’s α coefficient. Standard error of measurement (SEM) was computed. OSTE stations were not uni-dimensional, hence factor analysis including varimax rotation was performed. For convergent validity (Pearson correlations) was determined between scores of OSTE, lectures delivered and workshops conducted.
Results: In-group I, overall mean score was 7.64 + 0.65 and in-group II 7.33 + 1.03, (p=0.429). In group I SEM =0.35 and group II SEM= 0.48. Factor analysis yielded four factors in group I and five factors in group II with eigen values >1. The Pearson’s correlation of average scores of OSTE and lecture was significant for group I (r = 0.61, p 0.019), and insignificant for group II (r=0.46, p= 0.08). Correlation of OSTE and workshop conducted was insignificant for group I (r= 0.51, p = 0.057) and significant in group II(r = 0.53, p =0.04).
Conclusions: OSTE is a reliable, valid assessment tool for discerning competency among learners of HPE program.
Key words: Validity, reliability, objectively structured teaching examination