WPI - Computer Science Department , PhD Proposal Defense, Aaron Haim " On the Effects of On-Demand Assistance in Online Learning Platforms""
12:00 pm to 1:00 pm
Aaron Haim
PhD Candidate
WPI – Computer Science
Monday, August 5, 2024
Time: 12:00 p.m. – 1:00 p.m.
Location : Unity Hall 320 H
Zoom: https://wpi.zoom.us/j/5088315569
Committee Members:
Advisor : Prof. Neil Heffernan, WPI – Computer Science Department
Prof. Jacob Whitehill, WPI – Computer Science Department
Prof. Lane Harrison, WPI – Computer Science Department
Prof. Stacy Shaw, WPI – Social Science & Policy Studies
Abstract:
Obtaining on-demand assistance on a given problem has been shown to improve student performance on either the subsequent problem or a set of problems making up the post-test. This effect has been replicated at-scale within online learning platforms, even when crowdsourced by experts. However, while on-demand assistance is helpful in general, there are many unknowns as to what features of a piece of assistance attributes to the benefit of the student. Additionally, existing research is sometimes non-reproducible, making it difficult to determine whether the underlying results are reliable.
This proposal presents research on experiments with on-demand assistance using an at-scale experiment manager while also discussing the reproducibility of results in education technology. It includes studies on student perception of the effects of on-demand assistance on performance, comparing the effectiveness between educators on their created assistance, understanding the effects of feedback based upon student engagement with the system, and the current state of open science practices and reproducible research in the field. There are no significant findings towards student perception and educator effectiveness.
For low effort students, although there is no significant difference, students receiving feedback are more likely to answer the next problem correctly. Towards the state of open and reproducible research in the field of education technology, there is approximately a 10% adoption rate between certain aspects of open research, while less than 1% produced exact or similar results to those reported.