Beyond Right and Wrong: Performance and Persistence in Solving Scaffolded, Randomized, and Auto-Graded Homework Problems

Positive life outcomes and learning are more strongly related to grit – combining passion and perseverance – than raw IQ. Measuring grit in chemical engineering education is addressed here. Auto-graded homework questions with randomized numbers and content provide frequent, formative assessments for students that created big data for instructors and researchers. Here, analytics are examined, and a model is constructed that combines student performance and persistence into a single score. This difficulty score shows potential to be used by faculty for creating assignments that are scaffolded, which has been shown to lead to greater learning. Over 800 questions and 170,000 attempts were studied from three cohorts of students (n=284) using a fully interactive online textbook, Material and Energy Balances zyBook. Problem sets consisted of multiple, scaffolded parts where easier levels preceded harder levels. Problem types included multiple choice as well as single and multiple numerical response. When students were allowed an unlimited number of attempts before a fixed due date, a median correct of 94% was measured. A difficulty score was established using both performance and persistence measures. Performance included both raw fraction correct and modified fraction correct, which accounted for only the students who attempted each level. Persistence was measured via attempts before correct. Based on a three-tiered point system, auto-graded questions were designated easy, medium, or hard, and cross correlation by question type and question order begins to validate the model.