Date of Award

12-18-2024

Degree Type

Thesis

Degree Name

Master of Health Professions Education

First Advisor

Dr. Naveed Yousuf

Second Advisor

Dr. Sara Shakil

Third Advisor

Dr. Azam S. Afzal

Department

Educational Development

Abstract

Introduction: Workplace based assessment is a well-established formative assessment modality in postgraduate medical teaching, yet its role in undergraduate teaching is still under exploration. Male urinary catheterization is one of the basic procedural skills every medical student must learn. Till now, a formative assessment tool for this is not available. So, this study will develop and investigate the validity and reliability of the Assessment of Male Urinary Catheterization Skill tool (AMUCS).
Research Objectives: To measure the content validity, internal consistency, interrater reliability and the ability of the newly developed AMUCS tool to differentiate between the different levels of performance. 1. To investigate the CVI and CVR of the newly developed AMUCS tool. 2. To measure the reliability of scores (internal consistency) using AMUCS on third year and final year undergraduate medical students. 3. To calculate the inter-rater reliability of scores by two independent raters using AMUCS. 4. To investigate if there is a difference in scores between third year and final year students assessed on urinary catheterization skills using AMUCS.
Methodology: This is a cross-sectional validation study conducted on third year and final year MBBS students (n=185) at HITEC Institute of Medical Sciences Taxila after receiving ethical approval from HITEC Institute of Medical Sciences Taxila and AKU. The tool was developed by utilizing the Delphi method. After development of the tool it was discussed with 3 subject specialists and their suggestions and corrections were incorporated in the assessment tool. The content validity was calculated by CVI and CVR. A pilot study was conducted on 15 students, in which they had to catheterize mannequin and were assessed by assessors according to AMUCS. The second phase study was conducted on all third and final year students in a three-day workshop, in which the students catheterized mannequins and were assessed according to AMUCS., Descriptive statistics were calculated and internal consistency reliability was calculated by Cronbach alpha. For inter-rater reliability, the Intraclass correlation coefficient (ICC) and Kappa statistics were used. To investigate the ability of AMUCS to differentiate the different levels of performance (Year 3 versus Year 5 students) independent t-test was performed.
Results: Content validity analysis revealed that all 17 items had a CVR of 0.83. 64.7% of the items achieved a high CVI (≥0.9). The descriptive analysis showed that the mean score ( percentage) of final year students (68.14 and 68.16) was better as compared to third year (54.61 and 54.68) by both assessors. The reliability of scores ( Cronbach's Alpha ) was 0.810.The Inter-rater reliability calculated by ICC was 0.999, and a Kappa coefficient of 0.963 was found, indicating an almost perfect agreement, far exceeding what would be expected by chance. The differences in performance between third year and fifth year students were analyzed using Independent T test which demonstrated statistically significant differences with fifth-year students scoring significantly higher than third-year students across both supervisors' ratings (p < 0.001).
Conclusion: This study developed the AMUCS tool, which showed content validity and high reliability of scores. It was observed that this tool can differentiate between learners of different levels.

First Page

1

Last Page

89

Download

Share

COinS