The value of using different types of physics problems to help students become proficient problem-solvers

During a professional development course for graduate student teaching assistants (TAs), a question was asked–would you use a context-rich problem (presented via a narrative set in a real-world context, not broken into parts and sometimes even without an explicit question) in an introductory physics class, and if so, how? One graduate TA’s sentiment appeared to resonate with most of the other TAs in the class, ‘I will not use this at all.’. Though more experienced in teaching, physics faculty often share this point of view–less than half of interviewed faculty members indicated they appreciated yet would not use such a problem. But if we want students to become proficient problem-solvers, should we not employ a comprehensive toolbox containing a wide variety of problem-types? Is there not a place for a context-rich problem in our teaching or are we overlooking the benefits and utility of problems such as these? Evidence continues to suggest that TAs and faculty alike both appear to be reluctant to adopt some types of physics problems in their teaching, while relying almost completely on just a couple types of problems. Based upon education research data, we believe it is time to broaden those horizons.


Introduction
Helping students become proficient problem solvers is an important goal of many physics courses at all levels, e.g.see [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16].However, is the whole the sum of its parts?The behaviourist paradigm [17] of learning and teaching perceives the difference between a novice and an expert merely as quantitative-experts simply know more than novices.Instruction involves an expert breaking down the knowledge or skills to be learned into a sequence of smaller steps.In the context of problem solving, it involves training students in bits of procedures that together consist of the solution to a problem.The role of the student is to master each step by practice and repetition.The role of the teacher is to model each step, provide opportunities for students to practice, and provide appropriate reinforcements to encourage success.In contrast, the cognitive apprenticeship paradigm views the difference between a novice and an expert as qualitative-their culture of practice.The role of the student is to engage in 'whole' activities that simulate those of the field they are learning (e.g.solving real-world problems).During these activities, they interact with their peers and with their instructor to reflect on their existing ideas and compare them to the ideas and procedures that more closely characterize the field, resolving possible conflicts, actively constructing new ideas and insights.The role of the teacher is to (a) model or explicate the intellectual skills and processes of the discipline, (b) coach students while they use the procedures of the discipline.The explication of thought processes serves as scaffolds.Finally, the teacher (c) gradually decreases this support (scaffolding) until students are independent.Thus, student learning is a gradual process of developing self-reliance [18].
For example, a piano teacher would not expect that simply practicing simple chords would be an effective way for an aspiring pianist to learn to play masterpieces.Instead, a piano teacher would likely share with their students the considerations underlying the choices she made when demonstrating playing, provide immediate feedback on the choices and underlying considerations the student made while listening to the piano student playing, then ask the student to bear those in mind when practicing playing independently.Similarly, from the point of view of a physics instructor, this guided learning process should manifest through the following: (1) we model choices of fruitful problem-solving and share with our students the considerations underlying these choices.(2) We provide coaching and immediate feedback during problem-solving activities.
(3) We gradually fade the scaffolds while our students develop self-reliance [18].If we adhere to this framework, then the same underlying physics problem posed in different ways can facilitate different aspects of the guided process.
As illustrated in figures 1-4, providing students with different types of problems can help with the development of problem-solving autonomy, due to the different advantages in the way in which a problem is posed.
Here are some examples to illustrate this point:  whole problems [19,20].See an example in figure 2. • A traditional textbook style problem may be helpful when students are ready for less support, and can practice problem-solving independence.Traditional textbook style problems could also be used during the modelling stage, as part of an in-class example.See an example in figure 3. • A multiple-choice problem could be beneficial when we need to provide immediate feedback after students have thought about a problem, such as during an in-class clicker question and/or when we wish to allow students to work together with the instructor coaching and scaffolding their learning.See conceptual and quantitative examples in figure 4.
One could, in fact, create different problem types like those in figures 1-4 for the same underlying physics scenario.For instance, because context-rich problems offer important advantages, it may be constructive to consider how to effectively turn a traditional textbook problem into a context rich problem.To do so, one could frame the textbook problem as a real-life scenario, rich with detail, perhaps adding extraneous information that students would need to reflect upon and filter, and requiring the students to construct what the problem is asking them (which is often done by omitting an explicit question from the problem as in figure 1).This would yield a problem that would require a careful conceptual analysis, systematic problem-solving, and reflection.

Preparing graduate students to teach reveals a narrow scope of problems they prefer to use in teaching
Physics graduate students may one day become faculty, so their views about what sort of problems to use in teaching are informative, since knowing their views allows for the possibility of intervention during professional development programs that often take place as part of a graduate student's preparation to serve as a teaching assistant (TA).To investigate this, we conducted a longitudinal iterative study of graduate students enrolled in a TA professional development course at a researchbased university in the US.The two-hour-long class meets once per week for an entire semester; it is a lengthy in-depth professional development course for first-year graduate students who are concurrently embarking on their first semester of serving as a recitation or lab TA.These students received instruction on research-based instructional strategies and completed different in-class and homework activities.The activities related to problem types took place in iterative phases that were informed by what was uncovered in the  previous phase.All phases were conducted as a homework activity in the middle of the semester, after they had some teaching experience, to elicit their ideas about different problem types.In phase 1, TAs were given several example problem types (see figures 1-4) covering the same underlying introductory physics problem.They were asked to complete worksheets related to these problems under the assumption that they had complete control over the introductory physics class, including control over problem types chosen for various purposes.The worksheet asked them to identify the features of each example problem (e.g.does it contain multiple choices?Is it broken into parts?etc), to list pros and cons of the features in the problems, and to rank the problem types based on how much they like them, how often they would use them, how instructionally beneficial they are, and how challenging they are.In addition, TAs were asked (as four separate questions) whether they would use the problem type for an exam, a quiz, a homework assignment, or a group work assignment.As a follow up, a subsample of TAs was interviewed after completing the training course to more deeply probe their sentiments at the end of the semester.Based on the results of phase 1, we decided to make the question of how to use various problems more open-ended, and in phase 2 we asked TAs to create their own problem for a quiz, and describe if they would give the quiz as a group or individual activity, and to further explain how they would change the problem for use in a homework assignment, exam, clicker question, and for a situation where they needed to assess students remotely.
In both phases, TAs had to implicitly or explicitly convey the types of problems they would use in their teaching for various purposes (see figures 1-4 for examples).We found that whether given examples of problem types (as was done in Phases 1) or whether asked to create their own problem (as was done in phase 2), the variety of types of problems that TAs prefer to use is narrow in scope.
Figure 5 shows the average phase 1 rankings on a Likert scale from 1 to 5 of how likely TAs would be to use various types of problems (CR = context-rich, MC = multiple-choice, BP = broken into parts, and Text = textbookstyle).It is clear from the rankings that TAs had a strong preference for using problems that are broken into parts and were least likely to use context-rich problems.
In phase 2, when given freedom to create a problem of their own design, 72% of TAs incorporated subparts into the design of their problem in some way, while only 27% created a contextrich style problem for any of the purposes they were asked to think about (quiz, exam, homework, or remote learning activities).In both phases, it appears that standard textbook style problems are the next most favoured after broken-into-parts problems, and multiple-choice problems are not at all popular.Though in phase 1, multiple-choice problems were ranked higher for likelihood of using them, compared with context-rich problems, majority of TAs reported that they would use them only for ease of grading on quizzes or exams.In phase 2, 67% of TAs created a textbook-style problem for at least one of the purposes (quiz, exam, homework, or remote learning).Since we explicitly asked about clicker questions in phase 2, it was clear from written responses that many TAs implicitly had assumed a clicker question would have to be multiple-choice.However, when prompted to consider what they would change about their problem to turn it into a clicker question, no one explicitly described adding multiplechoices that could help provide meaningful feedback or conveyed positive sentiments about the usefulness of multiple-choices for quick feedback in clicker questions.In fact, one TA offered the following critique, 'I hate clicker questions.' Physics faculty typically have more teaching experience than graduate students, so one might expect they would choose a wider variety of problems to incorporate into their instruction and assessment.However, like TAs, the problem feature most used by faculty members has been shown to be problems that are broken into parts, while multiple-choice and context-rich problems were least likely to be used [2].These results suggest that TAs and faculty alike do not typically 'think outside the box' about how different types of problems could be used in different ways.They prefer to stick with familiar kinds of problems that are broken into parts and/or are like the ones you would find in a textbook.The utility of using a variety of problems to support all phases of the cognitive apprenticeship model does not appear to be a motivation to widen the scope of problems chosen.

Reasons behind graduate student and faculty problem preferences
Graduate students' reasoning for preferring to use the types of problems they did is well-illustrated in figure 6.Here we see the phase 1 data which show the average rankings for how challenging and instructionally beneficial TAs judge the problems to be.These rankings reveal that problems which are broken into parts are regarded as the least challenging type of problem but having the most instructional benefit.This suggests that TAs do not believe there is instructional merit in challenging students too much, which is in line with results of prior research into TA expectations of undergraduate students [21].In phase 2, we see suggestions of a similar trend.We found that TAs also appeared to want to avoid challenging students, since the higher the stakes, the more likely they were to create broken-into-parts problems, with more TAs choosing broken-into-parts problems for exam situations compared with quiz situations.This suggests that TAs valued providing support for students over and above challenging students to solve problems more autonomously by offering them opportunities to solve problems which do not have the steps already broken-down for them.
One might suppose that TAs awareness of the need to challenge students would grow over time, so that, when we compare their sentiments to those of physics faculty, we may see a difference of opinion.However, research has shown that, while faculty show some recognition of the importance of helping students build problemsolving autonomy, they may not necessarily make choices to use problems that support that goal.Indeed, past work has shown that faculty often identify problem features that align with both their own instructional goals and with research-based methods to develop good problem-solving, but Even though the majority of faculty were aware that over-reliance on broken-into-parts problems can hinder students in developing problem-solving autonomy, 67% of faculty reported relying heavily on these problems anyway, and the most common reason for this use was to avoid stressful situations for students [2].Similarly, faculty did not report wide use of context-rich problems, even though they recognized them as potentially beneficial, and again, the most common reason was avoiding student stress [2].Another possible reason is to avoid stressful situations for the instructor by requiring more time to be devoted to interacting with confused students.It is encouraging that faculty recognize that, building autonomy is an important learning goal, but if we always seek to avoid the possibility of student stress from challenging students to solve problems with less support (or avoid instructor stress), then we undermine our instructional goals.Moreover, we ignore the crucial phase of weaning that the cognitive apprenticeship model would suggest is needed for optimal progress.

Thinking outside the box-how to make use of a variety of problem types to support student learning
Our research suggests that copious use is being made (by both faculty and graduate student TAs) of problems that are broken into parts and problems that are of a standard textbook style, to the near exclusion of other types of problems.Below, we summarize some possibilities for incorporating a variety of problem types for supporting students using the cognitive apprenticeship model as a guide: • Group work with context-rich problems.
Context-rich physics problems are complex, lacking in structure, often provide redundant information and have real-life contexts [22][23][24][25][26][27].How could these be useful to support student learning?Research has answered this question!Studies suggest that students who engage with context-rich problems are more likely to think about the concepts first and have a more positive attitude about problem solving [22][23][24][25][26][27].As students become more experienced in solving context-rich problems, they show progress towards more expert-like problem solving.These positive results are powerfully harnessed when students work in groups.
Students who work in groups are more likely to use effective problem-solving strategies and show positive inter-dependence when working on context-rich problems than when working on analogous traditional textbook problems [22,23].• Guided problem-solving tutorials with multiple-choice problems.Multiple-choice problems are often thought of by physics instructors only as means to easily grade a high-stakes assessment like an exam.However, multiple-choice questions can be used in a variety of other instructionally beneficial ways that can facilitate the coaching and feedback aspects of the cognitive apprenticeship model.One such way is by utilizing paper or web-based tutorials that contain multiple-choice questions that students can interactively engage with to receive immediate feedback and guide them through the problem-solving process.Tutorials can be used as self-paced assignments that allow for some scaffolding support while the student develops problem-solving independence and learns how to correct missteps along the way.A multitude of studies supporting the usefulness of such tutorials abounds [28][29][30][31][32]. Modern online versions of such tutorials can even be used for remote instruction [33], an application of a multiple-choice question that was overlooked when TAs in phase 2 were asked about how they might change a problem for use with remote instruction.• In-class multiple-choice clicker questions.
Another way of providing coaching and feedback involves supporting students' conceptual learning.Multiple-choice questions given in class as clicker questions allow for an opportunity to monitor students' conceptual understanding and offer feedback accordingly.Used as a 'Think-Pair-Share' activity, multiple-choice clicker questions can promote both positive inter-dependence among students as well as individual accountability [34].These formative assessment opportunities serve to engage students in discussion with peers to improve learning.Students benefit from the support of their peers and from the immediate feedback they receive when the answer is shared.
In addition to these possibilities, other uses of a variety of problem types exist when employing research-based instructional strategies, such as interactive lecture demonstrations, pre-lecture quizzes for a flipped classroom, 'Just-in-Time' teaching, and more.Rather than constraining ourselves to a narrow set of standard problems, we could enhance our teaching by making use of a diverse palette of imaginative types of problems that support students through the entire process of learning effective problem-solving skills and engage them meaningfully in the process.

Opportunities to construct different problem types for a given situation
Some physics faculty and TAs may have considered using the different types of problems for a given situation discussed here, however, getting used to developing and utilizing these different types of problems to facilitate student learning may take practice and exposure.Moreover, context-rich problems may be less familiar to faculty and TAs since that problem type is not typically in textbooks.For TAs and instructors who are new to context-rich problems, a database of examples, implementation suggestions, and additional information is available through the University of Minnesota ( [35]).
In the TA professional development class, students were given illustrative examples of different problem types for a given situation, to demonstrate how that situation could be posed using different problem types (see figures 1-4 for the examples provided to TAs).These example problem types included multiple-choice, contextrich, broken into parts, and a traditional textbook problem.However, graduate students were not given the opportunity to construct different problem types themselves for situations they chose, so they did not have ownership of the process of constructing different problem types for a given situation.Giving them the opportunity to construct different problem types would also have given them an opportunity to reflect upon the benefits of different problem types.Thus, in future implementations, our TA professional development course will provide graduate students opportunities to construct different problem types from some situations provided to them and other situations of their own liking.By doing this, we hope to model the coaching and scaffolding aspect of the cognitive apprenticeship model in their learning about different problem types to develop autonomy in this area.Moreover, such professional development efforts, which target graduate students who may one day become faculty can serve as a model and potentially influence how future faculty approach the use of different problem types.

Figure 1 .
Figure 1.An introductory physics problem posed in a context-rich format.

Figure 2 .
Figure 2. The same problem scenario as in figure 1, but posed as a broken-into-parts problem.

Figure 3 .
Figure 3.The same problem scenario as in figure 1, but posed as a textbook-style problem.

Figure 4 .
Figure 4.The same problem scenario as in figure 1, but posed as (a) a quantitative multiple choice problem, and (b) a conceptual multiple choice problem.

Figure 5 .
Figure 5. Average rankings for how likely TAs are to make use of various types of problems.CR = contextrich, MC = multiple choice, BP = broken into parts, and Text = textbook-style.

Figure 6 .
Figure 6.Average rankings for use of various problems.CR = context-rich, MC = multiple-choice, BP = broken into parts, and Text = textbook-style.
Phys.Rev. Phys.Educ.Res. 6 020108 [3] Bolton J and Ross S 1997 Developing students' physics problem-solving skills Phys.Educ.32 176 [4] Harper K 2006 Student problem solving behaviors Phys.Educ.44 250 [5] Yap K and Wong C 2007 Assessing conceptual learning from quantitative problem solving of a plane mirror problem Phys.Educ.42 50 [6] Marusic M, Erceg N and Slisko J 2011 Partially specified physics problems: university students' attitudes and performance Eur.J. Phys.32 711 [7] Mashood K and Singh V 2013 Large-scale studies on the transferability of general problem-solving skills and the pedagogic potential of physics Phys.Educ.48 629 [8] Mason A and Singh C 2016 Using categorization of problems as an instructional tool to help introductory students learn physics Phys.