The revival of essay-type questions in medical education: Harnessing artificial intelligence and machine learning
Document Type
Article
Department
Educational Development
Abstract
Objective: To analyse and compare the assessment and grading of human-written and machine-written formative essays.
Study design: Quasi-experimental, qualitative cross-sectional study. Place and Duration of the Study: Department of Science of Dental Materials, Hamdard College of Medicine & Dentistry, Hamdard University, Karachi, from February to April 2023.
Methodology: Ten short formative essays of final-year dental students were manually assessed and graded. These essays were then graded using ChatGPT version 3.5. The chatbot responses and prompts were recorded and matched with manually graded essays. Qualitative analysis of the chatbot responses was then performed.
Results: Four different prompts were given to the artificial intelligence (AI) driven platform of ChatGPT to grade the summative essays. These were the chatbot's initial responses without grading, the chatbot's response to grading against criteria, the chatbot's response to criteria-wise grading, and the chatbot's response to questions for the difference in grading. Based on the results, four innovative ways of using AI and machine learning (ML) have been proposed for medical educators: Automated grading, content analysis, plagiarism detection, and formative assessment. ChatGPT provided a comprehensive report with feedback on writing skills, as opposed to manual grading of essays.
Conclusion: The chatbot's responses were fascinating and thought-provoking. AI and ML technologies can potentially supplement human grading in the assessment of essays. Medical educators need to embrace AI and ML technology to enhance the standards and quality of medical education, particularly when assessing long and short essay-type questions. Further empirical research and evaluation are needed to confirm their effectiveness.
Publication (Name of Journal)
Journal of the College of Physicians and Surgeons Pakistan
DOI
10.29271/jcpsp.2024.05.595
Recommended Citation
Shamim, M. S.,
Zaidi, S. A.,
Rehman, A.
(2024). The revival of essay-type questions in medical education: Harnessing artificial intelligence and machine learning. Journal of the College of Physicians and Surgeons Pakistan, 34(5), 595-599.
Available at:
https://ecommons.aku.edu/pakistan_fhs_mc_ded/63