AI-driven Feedback Loops for Student Evaluation 

Project members

Judy Irving, Daniel Long, Kate Forrester, Matthew Hurst (MSD).

Project summary

Using AI-driven feedback loops to streamline the collection, analysis, and response to student feedback, enhancing teaching and learning while reducing the time and resources required for academic and administrative processes. 

View final project report (PDF)

AI in Teaching and Learning at Oxford Knowledge Exchange Forum, 9 July 2025

Findings from projects supported by the AI Teaching and Learning Exploratory Fund in 2024–25 were presented at the AI in Teaching and Learning at Oxford Knowledge Exchange Forum at Saïd Business School on Wednesday, 9 July 2025.

Project team members each presented a lightning talk to all event participants, and hosted a series of small group discussions.

Follow the links below to view the lightning talk recording and presentation slides for this project.

Click the image above to play lighting talk recording

View presentation slides (PDF)

Project case study

The Nuffield Department of Primary Care Health Sciences piloted the use of ChatGPT to streamline feedback collection and analysis within its postgraduate programmes. The initiative aimed to reduce administrative burden, enhance responsiveness to student input, and build AI literacy among staff. Over a six-month period, the project team explored ChatGPT's capacity to automate feedback processes, focusing on tasks such as drafting surveys and analysing qualitative and quantitative data. 

The primary rationale for using AI was to achieve greater efficiency and consistency across programmes. However, while ChatGPT showed potential for simplifying repetitive writing tasks and generating summarised text, its performance in analysing raw feedback data proved problematic. The tool struggled with complex multi-layered inputs, often producing oversimplified or inaccurate analyses, and required extensive human oversight. Instead of reducing workload, it frequently increased it. 

Despite these challenges, the project generated several key benefits. Staff developed AI prompt engineering skills and became more confident using AI for ancillary tasks like proofreading, summarising, and refining survey design. Hands-on experimentation and interdisciplinary collaboration with pedagogical and technical teams enhanced team learning, providing valuable insights into AI’s current capabilities and limitations in educational contexts. 

Challenges included inconsistent AI outputs, and limitations in functionality of ‘chat’ and ‘custom GPT’ modalities in terms of sharing workflows across teams. The individual licensing approach also poses problems for retaining continuity of practice on individual programmes in the long term. Organisational issues—such as uneven engagement by project members and lack of early project structure—further impacted progress. Feedback from staff highlighted the need for clearer early scoping, more structured training, and better tools for collaboration and a structured method for recording accuracy of results from different prompt iterations. 

The project’s most significant learning was a recalibration of expectations around AI. ChatGPT was best suited to simple, well-defined tasks and not ready for standalone use in feedback analysis. Future work will likely focus on integrating AI into hybrid workflows where it supports, but does not replace, human judgment. The team also emphasised the importance of continued upskilling, framework development for AI use, and sharing findings to support institutional learning as AI tools evolve.