Day 2 : Oral Presentations (Theme: Curriculum-Assessment)
Determining the construct validity of assessment of clinical competence in internal medicine clerkship (at AgaKhan University)
Location
AKU Auditorium
Start Date
27-1-2013 8:50 AM
Abstract
Background: With modern curricular planning methods, the curricular content, the instructional methods and assessment strategies are continually monitored to ensure that quality standards are maintained. Quiet often a robust measure and analysis of the validity of clerkship assessments is overlooked. This study was done to explore the construct validity related evidence of the multi-method assessments used by the department of medicine for assessing student performance in the undergraduate medicine clerkship program at Aga Khan University.
Methods: This was an ex-post facto study of the assessments scores of 167 students of two consecutive cohorts who rotated in the clinical clerkships of the department of medicine from 2009-2011. Analysis consisted of reliability, factor and regression analyses of end of clerkship scores in 3-5 years. The multi-trait multi-method matrix (MTMM) was developed to assess the construct validity.
Findings: It was found that the reliability of the assessment instruments used were moderate to high (0.62 - 0.94). Factor analysis isolated 2 attributes which were identified as clinical reasoning ability and professional behaviour. Regression analysis found that the third year scores were predictive of final year score. Construct validity of the multi-method assessments was determined by application of the Campbell and Fiscke criteria to the MTMM developed.
Conclusions The undergraduate assessment of clinical competence conducted by the department of medicine at Aga Khan University predominantly assesses students’ clinical reasoning and professional behaviour; both traits are essential for physicians to be certified as clinically competent.
Key words: Assessment, Construct Validity, Psychometrics, Clinical competence
Determining the construct validity of assessment of clinical competence in internal medicine clerkship (at AgaKhan University)
AKU Auditorium
Background: With modern curricular planning methods, the curricular content, the instructional methods and assessment strategies are continually monitored to ensure that quality standards are maintained. Quiet often a robust measure and analysis of the validity of clerkship assessments is overlooked. This study was done to explore the construct validity related evidence of the multi-method assessments used by the department of medicine for assessing student performance in the undergraduate medicine clerkship program at Aga Khan University.
Methods: This was an ex-post facto study of the assessments scores of 167 students of two consecutive cohorts who rotated in the clinical clerkships of the department of medicine from 2009-2011. Analysis consisted of reliability, factor and regression analyses of end of clerkship scores in 3-5 years. The multi-trait multi-method matrix (MTMM) was developed to assess the construct validity.
Findings: It was found that the reliability of the assessment instruments used were moderate to high (0.62 - 0.94). Factor analysis isolated 2 attributes which were identified as clinical reasoning ability and professional behaviour. Regression analysis found that the third year scores were predictive of final year score. Construct validity of the multi-method assessments was determined by application of the Campbell and Fiscke criteria to the MTMM developed.
Conclusions The undergraduate assessment of clinical competence conducted by the department of medicine at Aga Khan University predominantly assesses students’ clinical reasoning and professional behaviour; both traits are essential for physicians to be certified as clinically competent.
Key words: Assessment, Construct Validity, Psychometrics, Clinical competence