Is Feedback In The Eye Of The Beholder? Chat Gpt Vs. Google Gemini In Giving Feedback On Students’ Oral Presentation Scripts
DOI:
https://doi.org/10.15503/emet2024.115.124Abstract
Aim. The paper aims to demonstrate how integrating innovative technologies like Chat GPT and Google Gemini can offer a transformative approach to giving feedback to students.
Methods. This qualitative research explores the effectiveness of Chat GPT and Google Gemini in assessing students’ oral presentation scripts. The sample consists of 12 oral presentation scripts written by medical students in years 5 and 6 for their Medical English class. The scripts are ordinarily evaluated based on the rubrics including clarity, cohe- sion, medical terminology, grammar, and syntax. Also, the Medical case report bench- marks are considered in the evaluation. The identical assessment criteria were used as prompts for both Chat GPT and Google Gemini and the feedback produced by each tool was then compared manually. The feedbacks were analysed for comprehensiveness, sug- gestions, problems and tone to determine which tool was more suitable for this task.
Results. The results indicate differences in the comprehensiveness and the focus of the assessment. Whereas Chat GPT relies for the most part on medical case report benchmarks and how they were/were not met by the students, Google Gemini analyses strengths, areas for improvement (clarity, concision, flow and transitions) and provides specific feedback by slide.
Conclusions. The results can contribute to a better understanding of the capabilities and limitations of Chat GPT and Google Gemini in providing feedback on students’ oral presentation scripts. Furthermore, the findings can help educators select the most ben- eficial tool, with which they can provide personalised and immediate insights and thus support student learning and growth in medical English. Nevertheless, what is necessary for that potential to be adequately harvested with the lowest probability of error, bias and hallucination is quality-based prompt engineering and coaching on the part of the user.
Downloads
References
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment and Evaluation in Higher Education, 38(6). 698-712.
Dubac Nemet, L. (2021). The importance of assessment in task-based Medical English courses for the evolution of students’ presentation skills. 5. eđunarodna konferencija “Suvremeni izazovi u poučavanju jezika struke”, 65-77.
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A. & Chartash, D. (2023). How does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Medical Education, 9, e45312. doi: 10.2196/45312.
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1). 81-112. Haughney, K., Wakeman, S., & Hart, L. (2020). Quality of Feedback in Higher Education: A Review
of Literature. Educational Sciences, 10(3). 60.
McGee, S. (n.d.) Oral Case Presentation. Retrieved from https://uw.pressbooks.pub/fcmtextbook/
part/oral-case-presentation
OpenAI. (2022). Introducing Chat GPT. Retrieved from https://openai.com/index/chatgpt/.
Pichai, S., & Hassabis, D. (2023). Introducing Gemini: our largest and most capable AI model. Retrieved
from https://blog.google/technology/ai/google- gemini-ai/#capabilities.
Siegel, M. D. (2018). The Yale Way. Retrieved from https://files- profile.medicine.yale.edu/
documents/eccf7de4-4c16-4f56-9a91-59b8015e48da
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., Voss, C., Radford, A., Amodei, D., & Chris-
tiano, P. F. (2020). Learning to summarize with human feedback. In: Proceedings of the 34th Inter- national Conference on Neural Information Processing Systems (NIPS’20). Curran Associated Inc., Red Hook, NY, USA, Article 253: 3008-3021. Retrieved from: https://proceedings.neurips.cc/ paper/2020/file/1f89885d556929e98d3ef9b86448f951-Paper.pdf
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Ksenija Benčina, Lorna Dubac Nemet
This work is licensed under a Creative Commons Attribution 4.0 International License.
