Date of Award

12-18-2024

Degree Type

Thesis

Degree Name

Master of Health Professions Education

First Advisor

Dr Naveed Yousuf

Second Advisor

Dr Javeria Rehman

Third Advisor

Dr Vaqar Bari

Department

Community Health Sciences

Abstract

Introduction: The evaluation of procedural skills is a cornerstone of competency-based dental education. Traditional assessment methods often lack objectivity and consistency, compromising the reliability of skill evaluations. The Objective Structured Assessment of Technical Skills (OSATS) was developed to address these gaps by providing structured, criterion-based, and reliable assessment of cavity preparation skills in undergraduate dental students. This study aims to develop and validate an OSATS tool tailored for cavity preparation to ensure its reliability, validity, and educational impact.
Methodology: This psychometric validation study was conducted at Bhitai Dental and Medical College, Mirpurkhas, involving 160 second- and third-year BDS students. A newly developed OSATS tool was evaluated for internal consistency, inter-rater reliability, and content validity. Assessments were conducted pre- and post-training by two calibrated assessors using a standardized scoring rubric. Descriptive statistics, Cronbach’s Alpha, and intra-class correlation coefficients (ICC) were used to assess the overall construct validity which includes all validity. Data collection was done after ethical approval from BDMC and AKU ERC.
Results: Cronbach’s Alpha values demonstrated excellent internal consistency for the OSATS tool, improving from 0.769 pre-training to 0.950 post-training. Intraclass coeffiecient (ICC) values ranged from 0.772 pre-training to 0.785 post-training, reflecting good inter-rater reliability. Content Validity Index (CVI) scores were consistently 1.0, affirming the tool's clarity, relevance, and comprehensiveness. Mean scores for students improved significantly post-training, indicating the intervention’s effectiveness in enhancing technical skills. No statistically significant difference was found between the scores of the undergraduate dental students of year 2 and year 3 on the OSATS tool. However, the difference between assessment 1 and assessment 2 (post feedback) by both the assessors was highly significant. (p value< 0.001)
Conclusion: The OSATS tool demonstrated robust psychometric properties, including high reliability and validity. Feedback significantly enhanced student performance, while the standardized evaluation criteria ensured consistency across assessors. These findings support OSATS as an effective assessment tool for procedural skills, promoting competency development and patient safety.

First Page

1

Last Page

94

Download

Share

COinS