← Back to Paola Mejia-Domenzain

AI or Human? Evaluating Student Feedback Perceptions in Higher Education

Tanya Nazaretsky, Paola Mejia-Domenzain, Vinitra Swamy, Jibril Frej, Tanja Käser

European Conference on Technology Enhanced Learning (ECTEL 2024) · 2024

Study design: identical feedback is labelled as human-written or AI-generated and rated blind, then with the provider revealed, to measure shifts in perceived objectivity, usefulness and genuineness.
Study design: identical feedback is labelled as human-written or AI-generated and rated blind, then with the provider revealed, to measure shifts in perceived objectivity, usefulness and genuineness. · Diagram drawn for this page from the paper’s method.

FindingStudents who could not tell who wrote the feedback rated the AI's higher; those who could tell preferred the human's.

Abstract

Feedback plays a crucial role in learning by helping individuals understand and improve their performance. Yet, providing timely, personalized feedback in higher education presents a challenge due to the large and diverse student population, often resulting in delayed and generic feedback. Recent advances in generative Artificial Intelligence (AI) offer a solution for delivering timely and scalable feedback. However, little is known about students' perceptions of AI feedback. In this paper, we investigate how the identity of the feedback provider affects students' perception, focusing on the comparison between AI-generated and human-created feedback. Our approach involves students evaluating feedback in authentic educational settings both before and after disclosing the feedback provider's identity, aiming to assess the influence of this knowledge on their perception. Our study with 457 students across diverse academic programs and levels reveals that students' ability to differentiate between AI and human feedback depends on the task at hand. Disclosing the identity of the feedback provider affects students' preferences, leading to a greater preference for human-created feedback and a decreased evaluation of AI-generated feedback. Moreover, students who failed to identify the feedback provider correctly tended to rate AI feedback higher, whereas those who succeeded preferred human feedback. These tendencies are similar across academic levels, genders, and fields of study. Our results highlight the complexity of integrating AI into educational feedback systems and underline the importance of considering student perceptions in AI-generated feedback adoption in higher education.

Venue
European Conference on Technology Enhanced Learning (ECTEL 2024)
Year
2024
DOI
10.1007/978-3-031-72315-5_20
Topics
Online Learning and Analytics

Cite this paper

Tanya Nazaretsky, Paola Mejia-Domenzain, Vinitra Swamy, Jibril Frej & Tanja Käser (2024) AI or Human? Evaluating Student Feedback Perceptions in Higher Education. European Conference on Technology Enhanced Learning (ECTEL 2024). https://doi.org/10.1007/978-3-031-72315-5_20

BibTeX
@inproceedings{nazaretsky2024ai,
  title         = {{AI or Human? Evaluating Student Feedback Perceptions in Higher Education}},
  author        = {Tanya Nazaretsky and Paola Mejia-Domenzain and Vinitra Swamy and Jibril Frej and Tanja Käser},
  year          = {2024},
  booktitle     = {European Conference on Technology Enhanced Learning (ECTEL 2024)},
  publisher     = {Springer Nature},
  doi           = {10.1007/978-3-031-72315-5_20},
  url           = {https://paola-md.github.io/papers/ai-or-human-evaluating-student-feedback-perceptions-in-higher-education.html}
}