Validity evidence of multiple mini interviews for selection into an undergradudate medical education program
Date of Award
2018
Document Type
Thesis
Degree Name
Master of Science in Health Policy & Management (MSc Health Policy & Mgmt)
Department
Community Health Sciences
Abstract
Considering the high stakes involved for students, faculty, institutions and the society, rigorous mechanisms underlying assessment should be employed to student selection throughout the continuum of medical education. Healthcare professionals require many personal attributes in addition to cognitive abilities and psychomotor skills for competent practice. The multiple mini-interview (MMIs) are considered to be valid and reliable method for assessing these attributes. Objectives: This study reports the evaluation of MMIs conducted at Al-Nafees Medical College & Hospital under the contemporary unitary concept of 'Construct Validity' especially in relation to content related evidence, response process, internal structure evidence and relation to other variables. It also reports the validity evidence on MMI stations' ability to measure the intended attributes and reports on the stability of its results by analysing another cohort two-years apart. Methods: An a priori nine-factor-model was hypothesized for the attributes considered most essential for a graduating student of Al-Nafees Medical College and Hospital. After operationally defining the attributes, scenarios were constructed for each of the attributes. A 5-point rating scale was used to rate each item on the station. The mean scores, standard deviation and number of items for each station, the reliability coefficient using Cronbach's alpha, standard error of measurement and item-total correlation of the scores on each station were determined. Confirmatory Factor Analysis (CFA) using principal component analysis (PCA) with varimax rotation following Kaiser Rule (i.e. eigenvalues > 1.0) was done to determine the number of factors being assessed on MMI examination as a whole and to assess if the stations were assessing what they were supposed to assess for 2013 and 2015 students' cohorts. MMI scores of 2015 students' cohort were correlated with their secondary and higher secondary certificate scores for evidence of divergent validity. Results: For the 2013 cohort the mean scores on each MMI station ranged from 27.4% to 80.0%. CFA using PCA was performed for 37 items which loaded onto ten factors with all items loading on their own stations except one station on empathy which loaded onto two factors. For the 2015 cohort CFA was done for 26 items and revealed the seven-factor model. The reliability of stations using Cronbach's alpha calculated for each station separately ranged from 0.64 to 0.98 for 2013 cohort and 0.73 to 0.94 for 2015 cohort. For 2015 cohort the mean scores ranged from 58.13% to 71.88%. The item-total correlations ranged from 0.30 to .75 For the 2013 cohort and 0.30 to .75 for the 2015 cohort. Conclusion: MMI can be used to make reliable and valid decisions to select students with desired attributes contextualized to an institutions' mission and vision. eywords: Multiple mini interviews, Personal attributes, Medical school selection, Exploratory factor Analysis
Recommended Citation
Ayub, R. (2018). Validity evidence of multiple mini interviews for selection into an undergradudate medical education program (Unpublished doctoral thesis). Aga Khan University, Karachi, Pakistan.