Investigating the impact of a tutorial approach in physics teacher education

Recently, teacher education in Austria has undergone major changes. Among other factors, profound content knowledge is essential for the professionalisation of teachers and the later quality of their instruction at school. First evaluation findings of our physics teacher education program revealed year two students’ deficits in the conceptual understanding of introductory mechanics. Since conceptual difficulties hinder meaningful learning of advanced topics and the development of pedagogical content knowledge, we adapted courses by integrating parts of tutorials of the University of Washington into the existing courses without changing the entire teaching mode. We used a pre-post-test design to investigate the impact of these short phases of the tutorial approach. The results indicate that this kind of learning opportunities supports students’ conceptual understanding in the field of mechanics. It shows a positive impact on content knowledge and self-efficacy. Students’ feedback on the tutorial approach shows that most students of the intervention groups appreciated this approach and its usefulness. In addition, all students pleaded for continuing the tutorial approach in the calculus-based mechanic course.


Introduction
International competitive students' performance tests like PISA, TIMSS and PEARLS revealed deficits in the Austrian education system. As a reaction, a paradigm shift from input-to output-orientation was initiated by the implementation of educational standards on the level of secondary education. Simultaneously, the education of future teachers was restructured. Now, Austrian teacher education programs follow a Bologna conform bachelor -master structure.
So far, the quality of science teacher education in Austria has only been partly researched in terms of the three professional domains first mentioned by Shulman [1]: content knowledge (CK), pedagogical knowledge (PK) and pedagogical content knowledge (PCK). The new bachelor-master curriculum for future physics teachers at the University of Graz was developed based on research findings on professional knowledge. Educational formats were created to intertwine the development of CK and PCK: Physics teacher students have to attend physics lectures e.g. on mechanics in parallel to calculusbased mechanics courses, mechanics labs, and didactics courses covering the educational reconstruction and implementation of these topic for secondary students.
While teacher students attend lectures and labs together with physics majors, the calculus-based mechanics courses and the didactics courses are sui generis. So far, research on this implementation showed that students appreciate this intertwined structure [2]. However, the output lags behind our expectations. In our previously conducted case studies, we identified a lack of substantial CK: typical students' conceptions were reproduced by teacher students while solving FCI items in think aloud studies [3]. Therefore, it is not surprising that certain PCK facets, especially the identification of students' conceptions and knowledge about instructional practices triggering conceptual change, are poorly developed. These results also go along with the findings of Sorge et al. [4] who showed that deep conceptual understanding of CK is a prerequisite for the development of PCK. Among other factors, profound CK is essential for the professionalization of teachers and the quality of their instruction in school.
These findings of previous research indicate the need of more learning opportunities that are tailored to the needs of our teacher students with the aim of promoting a deep conceptual understanding. Our findings concerning the level of conceptual understanding of the teacher-students is not unique, seen in an international context. In many countries, the teaching methods have not been adapted to findings of educational research during the last 5 decades. Many teachers, on the secondary as well as on the tertiary level, are averse from using new methods. A frequent reason is that they were not prepared to use them in their own education. So, it is essential to find ways, which can lead future teachers to change their teaching methods. So, we restructured the design of a calculus-based mechanics course for teacher students in the 5th semester. We tried to include a tutorial approach into the traditional structure of this course and evaluated the intervention with a pre-post-test design.

Theoretical background and research aim
Many empirical studies [5,6] show the efficiency of the "Tutorials in Introductory Physics" [7] developed by McDermott and Schaffer at the University of Washington. Although, there are findings about many different topics that show the positive impact of the tutorial approach, we want to give an example that fits in the thematic scope of our study. Table 1 contrasts students' performance in terms of post-test cores on the level of domain-specific topics. It compares students' performance in a traditional course structure with the tutorial approach. The results of Table 1 clearly show the superiority of the tutorial approach.
Above all, tutorials were found to promote the understanding of basic physical concepts and scientific thinking. In addition, they provide a structure that supports the students in their active engagement with the subject areas in terms of active learning. The aim of the presented study is to identify simple and thus by our faculty members accepted modifications in the calculus-based mechanics course for teacher students (course duration: 14 weeks, 90 min per week) that improve their conceptual understanding. In doing so, our focus is not to change the teaching modus fully, but rather to integrate small adaptions. In our experience, this makes it easier to find faculty staff who is ready to integrate such changes in their courses. The modifications in the calculus-based mechanics course are outlined in section 3. . The evaluation of this newly designed teaching and learning environment was based on the following research questions:  Do different types of intervention result in differences in CK, self-assessment of the own performance, self-efficacy or beliefs about teacher education?  Does the use of elements from the Washington Tutorials improve the performance in mechanics and/or the self-assessment of students' performance?  What is the subjective perception of the students concerning the usefulness of the tutorials, the way they were implemented in the course and a possible continuation of this approach?

Implementation of a tutorial approach
The curriculum for physics teacher students has a focus on mechanics in the third semester. Three different types of courses on mechanics are scheduled in the third semester: two subject-specific courses (lecture and calculus-based) and one physics-didactics course (seminar). As we wanted to improve physics teacher students' conceptual understanding of mechanics, we implemented minimal changes into the structure of the didactics course (3EC) and the calculus-based mechanics course (2 EC). Both courses are sui generis for teacher students, however, there are similar calculus-based mechanics courses for physics majors. These courses are taught by a group of instructors who prepare the courses together and they are used to identical educational settings (same exercises, same exams) in the courses for both teacher students and majors. This makes the preparation of the course tasks and of the exams easier.
And as a second argument they use that both groups, future teachers and future physicists have to learn the same physics anyway.
In the traditional calculus-based mechanics course, the students get conventional tasks (Figure 1), which they have to solve at home. In the next class, some randomly selected students have to present their solutions which are then graded. In addition, there are several written intermediate examinations during a semester, which are also graded. A traditional course task can be seen in Figure 1.
The focus of this traditional course setting is put rather on the output demonstrated in students' performances during sessions than on their learning processes. Without changing the content and the general structure of the course, we implemented a tutorial approach for some topics, using tasks from the Washington Tutorials [7]. A comparison of both types of tasks (see Figure 1 and Figure 2) reveals, the different focus of the tasks. While in the traditional task the students only have to do certain calculations, in the tutorial approach several more cognitive operations have to be carried out. The modified intervention in the calculus-based teacher student course consists of elements of the Washington Tutorials taken from "Tutorials in Introductory Physics" [7]. We selected the tutorial tasks in that way that they have as much overlap as possible with the content of the original tasks. So, we examined assignments of the original course and of the Washington tutorials for their central concepts in mechanics. We exchanged conventional assignments with assignments of the Washington tutorials, when they covered the same central concepts. These "new tasks" were used in short sequences (15-20 minutes), which are subsequently called "tutorials". The tutorials we integrated as a part of the traditional course were based on the "interactive tutorial lectures" [8]. The tutorial phases were structured as followed:  Handing out the work sheets and giving the instructions  Groupwork on worksheets  Discussion in plenary The beginning of a tutorial phase can be described as followed: At the beginning of each tutorial assignment, the worksheet is distributed to the students. Then the instructor (lecturer of the course) explains new terms and tasks (for example: free body pictures). The main part of the tutorials phase consists of an interactive group work of two to three students who are working together on the assignments. The cooperation and exchange between students are actively supported during this phase. The role of the instructor is a supportive but passive one in this setting. He does not show solutions, but supports students' learning processes by asking targeted questions, with the result that students can find a solution on their own. Finally, there is a plenary discussion in which students from different groups articulate their most important findings, basic ideas, and problems. In doing so, the instructor acts as a moderator and tries to stimulate students to think through arguments together and to steer the arguments in the right direction. However, he does not reveal the solution.
This type of tutorial exercises was used in three subject-specific courses (one per week) in a row and two physics-didactics courses in the same period of the term. In each of these two independent series three different tutorials were implemented. The tutorials were used in subsequent classes of the course. In Figure 3 the content of these series is displayed. In addition to the main topic, Newton's second and third laws, there were also tasks with topics momentum and energy used.

Research design of the evaluation
There were different types of interventions which differed in the intensity of the tutorial approach. In other words, they differed in the number of tutorial phases. Depending on the courses in which the students were enrolled, they could encounter tutorial phases in the traditional calculus-based course or in the related physics-didactics course or in both courses. To create these differences and furthermore to get a control group with just classic courses without tutorial phases, we implemented in only one of two courses the tutorial approach.
The intervention was evaluated by a pre-post-test design. The pre-test was administered in the class directly before the series of tutorials. The post-test was administered directly in the class after the final tutorial. An overview of the total research design is given in Figure 4. The content of the pre-and posttest is also presented in Figure 4.
As already mentioned, the participants of this intervention can be divided in several sub-groups ( Figure 5). The sub-groups are based on the different types of courses and the integration of the tutorial approach. Students themselves register for courses at the beginning of the semester. Usually they choose those courses that fit best into their time-table. At the moment, when the students registered for a course, it was not clear to them that some of the courses were to use a tutorial approach. We used tasks from the Washington tutorials for all courses with a tutorial approach. The tasks for the subject-specific course as well as for the physics-didactics course were on the same topics but differed in the assignment (see Figure 3).
Depending on the selected courses, the intensity of the tutorial intervention differed. Overall, the sample consisted of N = 155 students with 43 teacher students out of it. The intervention group consisted of three sub-groups: those students who worked with tutorials either in three sessions of the calculusbased course (I1; N = 13) or the didactics course (I2; N = 9), or both (I3; N = 7). In this sample we have two groups with an intensity of three tutorials, however in different courses: I1 in the subject specific course and I2 in the physics didactics course. As they show similar results, we merged them to one group as shown in Figure 5. The control group consisted of teacher students (C1; N = 14) and physics majors (C2; N = 112) who were not instructed with a tutorial approach in any course.  The concept test on mechanics focuses on school knowledge and in-depth knowledge, which is particularly important for physics teacher students. The pre-and the post-test consisted of CK-items and different scales on beliefs and self-efficacy. Additionally, we asked students about their certainty of response on the CK-items. For the evaluation of the tutorial approach, we used a pre-post design in an intervention and control group setting. The majority of used test items was taken from Riese [9], Woitkowski [10] and Hestenes [11] and expanded with feedback as a qualitative supplement. The results were analyzed quantitatively.

Findings
Caused by the small number of participants in the different sub-groups we could not get significant results. Nevertheless, the results mirror some impact of the tutorials phases on the students' conceptual understanding and their self-efficacy.

Content knowledge
All groups have improved on average. However, the increase for the matched pre-and post-tests is slightly larger in the tutorial group. Although the normalized gain g indicates a small change [12] for all subgroups, the average Cohen's d for dependent variables show medium effect sizes. Especially for the group with six tutorial phases it is much higher, nearly large in effect size [13]. For the control group there is just a small effect size. (Table 2) The results of the CK items show an improvement of the intervention group. The effect increases with the intensity of the tutorial approach. However, this effect could not be observed for all tasks. Figure 6. Frequencies of normalized change c [14] in CK for different course-groups and intensities of tutorial approach for physics teacher students. A negative value means a decrease and a positive means an increase in performance (range = 0.66 -(-0.34) = 1.00).
As far as students' certainty of response is concerned, we found that certainty values vary from item to item. For some items we get positive, for other items negative correlations between performance and certainty of response. The results for self-assessment of the own performance are generally in good accordance with the test results. Misjudgments show a strong tendency to overestimate. This overestimation was somewhat reduced in the intervention group.

Beliefs and self-efficacy
As far as self-efficacy is concerned, we found the tendency of higher gains (Figure 7Hiba! A hivatkozási forrás nem található.) in the intervention group without being statistically significant. Physics teacher students self-efficacy increased more when tutorials were part of the course. The items on beliefs about teacher education were part of the questionnaire of the physics teacher students only. The results show a recognizable influence of the tutorials on their answers. The higher the intensity of tutorials, the more the answers in the post test differed to those of the control group. For example, students had totally different opinions about the increase of courses on subject-specific topics 8 in contrast to didactics or pedagogy. The majority of students with a high number of tutorials is in favor of an expansion. These results are exactly the opposite to the results of the control group.

Feedback
Feedback on the tutorial approach showed that all students of the intervention group appreciated this approach. About 70% of the participants confirm its usefulness, especially for learning physic concepts and for one's own explanatory skills. According to the students, the tutorials helped to overcome misconceptions and uncertainties. All students advocate a further implementation of the tutorials. The majority of them (65.4%) wanted to stick to the current form implemented and the rest (34.6%) pleaded for an extension of the tutorial phases. Some (7.7%) even suggested to convert the entire course to the tutorials-format. At the same time, some recognize the calculus-based exercises as important, as they train slightly different competences.

Conclusion
The results of the present study indicate an improvement in the learning opportunities in the physics teacher education program in the field of mechanics. It shows a similar impact on CK and self-efficacy. However, most of the results in favor of the tutorials are not statistically significant, although they indicate a better performance of the students in the tutorial approach. This result is however not very surprising for us, as the time on task in the tutorial tasks was very short compared to the full course. In addition, the tutorial approach posed a totally new method to the students. So, it also took them some time to get used to it. Finally, the grading system was not adapted to the tutorial approach. Due to all these circumstances, we did not expect that the intervention using tutorials would lead right a way to statistically significant changes in performance. Although the tests did not yield any significant results, the further use of the tutorials or even their expansion to the entire duration of the course are wanted. So, in a next step the ratio of tutorial mode to the traditional course structure has to be changed and effects have to be evaluated.