Computer Science Department, MS Thesis Presentation Eamon Worden "Few-shot is All you Need, Providing Intelligent Feedback in Mathematics Education"

Eamon Worden 

MS Student

WPI – Computer Science Department 

Wednesday, April 23, 2025 

Time: 10:30 AM – 11:30 AM 

Location:  Unity Hall 320 

Advisor: Prof. Neil Heffernan

Reader: Prof. Fabricio Murai

Abstract : 

Automated feedback systems are important in mathematics education for providing timely and scalable support to students. While pretrained Large Language Models (LLMs) such as GPT have shown promise in generating feedback, fine-tuning LLMs to improve their performance is costly and resource-intensive. In this work, we explore cost-efficient alternatives, focusing on Retrieval-Augmented Generation (RAG) and prompt engineering techniques to enhance LLMs for feedback generation for open-ended math answers. We evaluate the effectiveness and practicality of RAG and few-shot prompting in generating both descriptive feedback and numeric scores for middle school math open responses.

Our results show that RAG based approach aligns most closely with teachers-generated feedback, offering a cost-effective solution without the need for fine-tuning. Further to answer questions about the validity and effectiveness of these methods, we conduct a study with expert teachers who evaluate feedback methods based on their quality and acceptability to students. We find that teachers prefer the RAG-based approach, although few-shot prompting performs slightly better in some areas. We further examine why various methods outperform others when generating feedback and find length and correctness are major factors for why certain feedback is preferred. These results highlight the potential of RAG and prompt-engineering-based approaches as efficient methods for improving LLM-generated feedback, offering effective scalable solutions in educational applications.