Modeling Student Performance on Auto-Graded, Randomized Homework Questions

The Internet of Things is creating large data sets for many devices from watches to thermostats. Expanding upon 'big data' generated by users are interactive textbooks used in chemical engineering courses. Thus, new research can answer many questions about student engagement and learning. Auto-graded homework questions with randomized numbers and content can explore students’ proficiency on new course concepts. Over 800 questions and 170,000 attempts were studied from three cohorts of students (n=284) using a fully interactive online textbook, Material and Energy Balances zyBook. Problems consist of multiple parts with the intention of each part increasing in difficulty. A variety of problem types were used including multiple choice as well as single and multiple numerical response. When students were allowed an unlimited number of attempts, a median success of 94% was measured. Success correlated with final course grade with some statistically significant differences. Additionally, a metric was developed to model student performance. The metric incorporated overall success rate, modified success rate that accounts for only the students who attempted each level, and attempts before correct. Based on a point system, questions were separated into easy, medium, and hard categories and cross correlated between question type and question order.