Validation of an assessment tool for direct observation performance skills at triage (DOPS-T) for the health care professionals in the emergency department of tertiary care hospital - a work place based assessment
Date of Award
2018
Document Type
Thesis
Degree Name
Master of Health Professions Education (MPHE)
Department
Educational Development
Abstract
Workplace based assessment is an effective way to measure the knowledge translation into clinical practice and has an important place in residency education with emphasis on skill acquisition and enhancement of learning through effective feedback. One of the goals of academic programmes in emergency medicine is to develop expertise in triage. Triage in the emergency department (ED) is the first contact point of a patient with the health care where patients are categorized as per their disease acuity which in turn can determine how fast the patient would be provided care. No instrument for workplace-based assessment of triage skills in the ED could be found. This study was aimed to develop and validate a tool for Direct Observation of Performance Skills at Triage (DOPS-T) for health care professions (HCPs) in the ED. Method The study was conducted at the emergency department of Sultan Qaboos University Hospital (SQUH) after ethics approval. SQUH is a tertiary care hospital at Muscat, Oman. Fifty HCPs (25 nurses and 25 physicians) were included in the study after informed consent. All HCP's underwent a Canadian Triage and Acuity Scale (CTAS) certification course. The change in knowledge was assessed by pre-test and post-test. After three months all FICP's were observed for their triage performance skills during the morning, evening and night shifts by two independent assessors in real-time clinical setting. Nine — point DOPS-T scale was utilized to rate the performance. SPSS version 22 and Stata version 12 were used for data analysis. Spearman correlation and interclass correlation test were used to calculate construct validity and inter-rater reliability. Effect size was calculated using Cohen's d. Learner's satisfaction and feedback were recorded. Feasibility was assessed by professionals' satisfaction and time spent in observation and providing feedback. Results Significant improvement in knowledge pertaining to triage was noticed in post-test (88.214.0) as compared to pre-test (42.2+9.0). Three hundred items were recorded using the direct observation of performance skills at triage (DOPS-T) tool for nurses and physicians. DOPS-T overall mean score on the 9-point Likert scale ±1 SD was 76.7112.44 (minimum-maximum: 57.78 - 96.67). Score was highest (8.4811.22) for 'taking the vital signs', followed by 'communication skills' (8.2110.96) while lowest score (6.32+1.58) was observed for 'reassessment done appropriately on separate items on the scale'. Summed scores were high in all the three shifts that is morning, evening and night for HCPs with more than 5 years of experience (79.12110.86 vs 74.03+8.29). Inter-rater reliability was high for assessors (ICC = 0.918 (95%0: 0.89-0.94) as well as for DOPS-T score (95%CI: 0.89-0.94). Inter-item correlation matrix showed moderate correlation. Internal consistency calculated by Cronbach's alpha was 0.911. The assessment process was rated as "very satisfied" (7-8 on a 9-point Likert scale).Mean time (±1SD) to complete the DOPS-T tool by the assessors was 15.4514.76 minutes while giving feedback took 5.77±1 .22 minutes. Conclusion The study demonstrated that recently developed DOPS-T instrument showed construct validity and reliability for direct observation of performance skills at triage for nurses and physicians. DOPS-T is feasible to be used despite of distinctive ED work hours.
Recommended Citation
Fayyaz, J. (2018). Validation of an assessment tool for direct observation performance skills at triage (DOPS-T) for the health care professionals in the emergency department of tertiary care hospital - a work place based assessment (Unpublished doctoral thesis). Aga Khan University, Karachi, Pakistan.