(48c) Replacing Mimicry with Mastery: Homework Wrappers and Effort-Based Grading
AIChE Annual Meeting
Monday, November 14, 2022 - 8:38am to 8:56am
In a problem-solving course, students often use a mimicry approach to solve homework problems. Their approach is to find a solved problem that appears to be similar and to mimic the steps taken in that solution to solve the assigned problem. The mimicry approach is maladaptive, inefficient (in terms of learning) and often ineffective. In this study, the solution to each assigned homework problem was provided to the students as a part of the homework assignment. Students were instructed to attempt to solve the problem as if they were taking an exam. They were told that the purpose of the homework was for them to gauge their mastery for solving that type of problem, and that their submission would be graded exclusively on the basis of their effort to master the solution process and not on whether they obtained the correct answer. They were told to refer to the provided solution only if they "got stuck" while solving the problem, and then to use it after they had completed their solution to determine whether they had made any mistakes. Each assignment included a homework wrapper question. This question asked them to reflect upon their mastery for solving this type of problem, to identify any misunderstandings that had resulted in mistakes, and to determine what they should do to remove misunderstandings and further develop their mastery for solving this type of problem. The student responses to the wrapper questions, together with observations of their help-seeking activities, suggested reduced reliance on pure mimicry during problem solving. In cases where significant mimicry was observed, there still were indications that their achievement goals may have shifted to include understanding how to solve the problem and not just getting the correct answer.
The context of this study was a required, upper-division kinetics and reaction engineering course. Between 2005 and 2015, the course was revised in a number of ways. It was converted from didactic lecture to a flipped classroom format. Explicit instruction was added in the areas of identifying the problem type and enumerating a general approach for solving each problem type. Instead of jumping directly to what the instructor did to solve each problem, the instructor explained how they knew what to do. "Mastery coaching," that is messaging from the instructor stressing that the goal is to know how to solve a problem, not just getting the answer, was introduced throughout class meetings. This also included the instructor noting that getting stuck on a problem was good because it showed a point where the student's understanding was not yet complete. Scaffolding was added to the in-class problem-solving activities each time a new reactor type or problem type was encountered. Finally effort-based grading of homework and the inclusion of homework wrapper questions in assignments was introduced.
As noted, a flipped classroom format was used to teach the course. Prior to class, students were expected to read text and/or watch videos that presented theory, derived and simplified equations and demonstrated how to use that theory and equations to solve a variety of different reaction engineering problem types. The problem types included parameter estimation (fitting models to reactor data), reactor response, process optimization and system design. During class, the pre-class information was briefly reviewed and students were afforded an opportunity to ask questions. Most of the time in class the students were engaged in problem-solving activities. A single homework problem (including the solution and a wrapper question) was due before the start of the next class. Some homework problems included a "thought question" that was intended to make the students interpret or qualitatively explain the answer to a problem. The homework submissions were awarded one point for an attempt to set up the equations need to solve the problem, one point for attempting to perform the calculations and obtain the answer, one point for answering the wrapper question and one point for answering the thought question; the correctness of the answers were not checked. At the end of the course, these homework scores were scaled to represent 20% of the students' final course grade. Importantly, 70% of the students' final course grade was based upon evaluations where they were graded on correctly setting up the necessary equations and performing the calculations to obtain a correct answer, but the students had always been afforded ample opportunity to practice prior to these evaluations.
When wrappers were first introduced in the course, they were more focused upon whether the student had correctly identified the problem type and, if they made a mistake, where in the general approach for solving that problem type they had made the mistake. In addition, the solution to the problem was made available and the wrapper question was assigned after the students submitted their solutions. There were some encouraging results (the recurrence rate of mistakes that were identified using wrappers was 4.6% compared to 29.3% for those that were not identified). However, three observations prompted the change to include the solution and wrapper with the assignment. First, 31% of the wrapper responses were deemed to be insincere (the students did not take them seriously). Second, the wrapper submission rate was only 61%. Finally, there was no evidence to suggest that the use of mimicry had diminished.
The fall 2021 offering of the course (it is only offered in the fall) was the first time the problem, wrapper and solution were all provided at the time of assignment. The wording of the wrappers was also changed to emphasize mastery over identifying solution process mistakes. As an example, here is a wrapper used with a parameter estimation problem: "The second learning outcome for [course number] is the ability to estimate values for parameters in a postulated rate expression and assess its suitability using kinetics data from an ideal stirred tank or plug-flow reactor. This is the first [course number] kinetics data analysis problem. After you have completed your solution, take a few minutes to reflect upon your current ability to satisfy this learning outcome. Consider whether you would easily recognize problems of this type on an exam, whether you know how to proceed without consulting solved examples of this type of problem, your overall confidence in your abilities for this type of problem, what you might do to increase your understanding and confidence, etc. Then write a short paragraph summarizing your reflections."
The wrapper submission rate increased slightly to 69%, but far fewer insincere responses were submitted. More importantly, the responses indicated students were focusing on problem-solving mastery and not only on getting the correct answer. In particular, the responses frequently assessed whether or not the respondent felt they could solve a similar problem under exam conditions (e. g. without access to a solved problem). Responses to successive wrappers on problems of the same type typically evidenced increasing mastery. Similarly, students often came to office hours to discuss their approach to solving a problem even though they had been provided with the solution.
At this point, the wording of the wrappers requires some minor revision, but they appear to be eliciting the desired student response. One focus in the next offering of the course will be on using the wrappers to initiate a dialog between the students and the instructor. The other focus will be measuring the students' achievement goal orientation and how it changes over the course of the semester. The present study suggests that students are moving toward a mastery orientation and away from a performance orientation. At this point, the combined use of effort-based grading and assignments that include both the problem solution and an appropriate wrapper question appears to be a promising approach to reducing student use of mimicry when solving homework problems and increasing their focus on the development of their problem-solving mastery.