DBR based Redesign cycle of a TLS on Dynamics for High School

In this paper, the process of the design a Teaching Learning Sequence (TLS) on Newton’s Law for high school is presented. The design of the TLS follows Design Based Research (DBR) methodology First, the grasp of the design is going to be presented to later, present the data obtained after the application of instruments designed to evaluate the TLS regarding the students achievement and the quality of the TLS. How we involved in-service teachers to participate in the design and redesign process and the implementation of the TLS is also explained.


Introduction
Design Based Research (DBR) is a very well-known methodology for de design, implementation, evaluation and redesign of the Teaching Learning Sequences (TLS).DBR not only aims to empirically demonstrate what works and what not but also aims to develop theories of classroom intervention.These are "humble theories" in the sense that they are conclusions from the analysis of specific teachinglearning processes in a particular curriculum domain or topic [1].Beyond simply creating designs that are effective, a design theory explains why TLS designs work and suggests how they can be adapted to new circumstances.Thus, like other methodologies, DBR is a crucible for generating and testing general theories [2].
Although there are different approaches to apply DBR methodology, there is a consensus among different authors to say that DBR should follow a design, implementation and evaluation and redesign cycle [3,4].Following this process, there are several works explaining how to apply DBR methodology to design a TLS for students in general (including pre-service teachers) and for in-service teachers where all of them are in the role of "students" for the designed TLSs [5,6,7,8].Although there are a lot of DBR based works for in-service teachers, they are presented as learners or it is not clear how they are part of the process design, implementation and evaluation and redesign.
In this article, we will apply the DBR methodology to design, implement and evaluate and redesign a TLS on Newton's laws for secondary school and explain how we included in-service teachers in the process.The results of the implementation are analysed using different measurement tools and these results are supplemented with input from teachers.All these results will feed into the redesign of the TLS.Previous work based on the DBR methodology of our research group was designed for university level, which means that the researchers, designers and teachers were the same people.In this case, the TLS designed is for secondary level and only one of the researchers is dedicated to secondary education.Our aim was to include different schools and teachers, which means that we need to include teachers from outside the research group.To face this challenge, different questions arise: How are we going to involve teachers in the process?How are we going to adapt the DBR methodology keeping the cycle and including teachers?
In the following sections, we will describe the process we have followed to build a community of teachers working according to the DBR methodology, explaining how and when they participated in the process.We will then present details on the inclusion of teachers and the implementation of a pilot study and a first full cycle of DBR to continue presenting the results obtained in terms of the quality of the sequence and the achievement of the learning objectives by the students where we captured the basis of the redesign.In the final section, we will present conclusions and future work.

Building a teachers community into DBR methodology
The DBR process is well established in the physical education research community, it is a cycle that includes three general phases: Design, implementation and evaluation and redesign.In the design phase, design tools such as "epistemological analysis" and "learning demands" [4] are applied.These tools are applied taking into account the context (including the academic level) where the TLS on Newton's laws will be applied, key ideas, learning demands and required teaching strategies are defined.Based on the results obtained after applying these design tools (learning objectives, learning demands related to students' difficulties), the activities and the teacher's guide specifying among others the teaching techniques and other practical aspects are designed and the TLS is ready to be implemented.The next step is the teaching experiment, i.e. the implementation.The implementation is the application in class of the activities designed by the researchers and/or teachers following the teaching guide generated as part of the TLS.For the evaluation of the TLS different evaluation tools are created.The teacher diary and teacher interviews provide us with information about the quality of the TLS.The pre-and post-tests give information about the students' achievements in relation to the learning objectives.
As mentioned in the introduction, one of the objectives of this work is to include teachers from outside the research group in the DBR process.The aim is to make them feel listened to and part of the whole process.Given the teaching load they have to cope with, we know that it is unrealistic to ask them to be involved in the project from the beginning and to be part of every little detail.It is difficult to ask them to be involved in tasks such as a literature review to establish the state of the art.In this paper we are proposing the involvement of teachers in the three phases of the DBR methodology, listening to them, taking into account their tacit knowledge about the context, but avoiding the theoretical part of the process, which can be very challenging for most of them.The involvement is done in the following way.In the design phase, we present the TLS to the teachers and explain the results of the application of the design tools, so that we first present the key ideas and learning demands of the TLS and finally define the learning objectives of the TLS and discuss their appropriateness and accuracy.We give teachers time to analyse the proposal and ask them if they agree or disagree with the proposal.If they do not agree, we give them the opportunity to propose and discuss changes.In this way, we take into account the teachers' point of view on the basis of the TLS design.
After discussing the basis of the design, the complete TLS (including activities, student's book and teacher's guide) is shared with the teachers.The teachers who are going to implement the TLS take some time to study the proposal.After a couple of weeks, we ask them to suggest changes to the TLS.Changes can be very diverse, ranging from a wording issue or a suggestion to change a picture to proposing to remove, change or add a specific activity.Any suggested changes must be argued by the teachers and accepted by the other members of the community.It is not mandatory to implement exactly the same TLS by all teachers.Some activities could be presented as optional and others could be mandatory but different wordings could be presented but the learning objectives should be the same for all final versions to be implemented.
The involvement of teachers for the implementation phase is essential, i.e. it is important to make the teacher feel comfortable and confident for the implementation.To this end, in the last TLS discussion we included a self-assessment activity for the teachers involved in the community.We ask them to study the teacher's guide and to analyse and list their needs in order to implement the proposal properly following the proposed teaching techniques.The teachers list their needs and some seminars and/or workshops are organised to train the teachers in relation to their needs for the implementation of the TLS.Despite all this training, they are offered continuous and priority mentoring and accompaniment during the implementation.
In the evaluation phase, teachers participate in the process by giving pre-and post-tests to their students to measure the achievement of the learning objectives.The analysis of these open-ended questionnaires provides us with data on the effectiveness of the TLS.This information is one of the bases for the redesign phase.In addition to this, and as far as the quality of the TLS is concerned, the qualitative evaluation of teachers is essential.Their diary gives us information on other key aspects to be taken into account such as timing, appropriateness of writing, comprehension of questionnaires, appropriateness of activities to the proposed learning objectives, appropriateness of teaching techniques....All this information is cross-checked in an interview with the teachers.In this interview, the researchers have the opportunity to share with the teachers in depth all the information recorded in their diaries.The information collected through the different instruments is analysed by the researchers and elements of redesign are proposed and finally shared and discussed with the teachers.

Applying DBR for the design of a TLS on Newton's Law for high school
Following the application of the design tools Epistemological Analysis and Learning Demands, 5 epistemological elements of the Newtonian model of force and motion were defined.These 5 key ideas were rewritten as 4 learning objectives.Once the learning objectives were defined, they were confronted with the learning difficulties of the students applying Learning Demands tool.These learning difficulties are well defined in the literature by means of the well-known Force Concept Inventory questionnaire.To define the learning demand, 3 aspects are taken into account: 1) The degree of inconsistency of the difficulty presented by the students with the key idea to which it relates; 2) The degree of connection or relevance of this inconsistency to other related areas of the curriculum and 3) The degree of commonality or how common the difficulty is among the students.As a result of the application of this tool, learning demands are defined as low, medium or high.These results are presented in Table 1.
We started working on this TLS with the teaching community in the academic year 2019-2020.Our idea was that this cycle would be the first one but as we are going to explain, finally the cycle was not completed due to covid19 and this cycle is going to be considered as a pilot implementation.5 teachers came to the first meeting where we explained the basics of the TLS design that has been presented in table 1.After the discussion sessions explained in the previous section, 4 teachers came to the second meeting.All 4 teachers agreed to propose an adjustment in the learning objective related to vectors.In addition, 5 minor changes were proposed in the activities mainly related to writing, and two activities contextualised in electromagnetism became optional.As a result of the self-evaluation of the teachers present, the need for training in active methodologies was detected, so training courses on different teaching techniques were offered.Finally 3 teachers implemented the TLS in 6 groups in three different schools, but due to covid19 some steps of the implementation did not go as designed so the evaluation phase was not completed and there was no data as a basis for a redesign.
In the academic year 2020-2021 we started the whole process again but having included in the TLS the changes that were proposed in the teachers' community in the design phase in the academic year 2019-2020.On this occasion, we also started with 5 teachers and for different reasons, only one of them was part of the community for the 2019-2020 academic year.After the first meeting, the teachers proposed a problem as a bridge between the introductory problems and other more complex ones.In this cycle, the need for training in active methodologies was also detected, so it was organised and delivered.The 5 teachers implemented the TLS in 8 experimental groups (two of them without posttest, which is why it they are not included in the results).In addition, the same pre-post-test was given to 4 other control groups that received regular teaching.are related to the first learning objective, question 4 to the second learning objective and question 5 to the last one.These questionnaires were administered to 6 experimental and 4 control groups.A phenomenographic analysis was carried out and a categorisation of the answers was made taking into account the students' reasoning.Regarding the phenomenographic analysis [9], although the analysis was carried out on individual responses, the focus was on the responses given as a group and the results were presented as percentages.In addition, each category was defined to give us a different line of reasoning and they could be ordered hierarchically.The process was repeated for each of the five questions.To construct the categorisation, the students' answers were subjected to rigorous analysis [10] and the analysis was not only focused on correct or incorrect answers, but also on identifying the students' understanding of the defined learning objectives.For this purpose, a set of preliminary categories was defined by a member of the research group.The categorisation was refined with other researchers in the group in an iterative process and a final set of categories was proposed.The students' responses were categorised by different researchers and Cohen's kappa was measured for reliability with values ranging from 0.87 to 1, revealing a very good consistency.
Teachers made notes in their diaries, looking at aspects such as the comprehension of the activities, the timing or the engagement of the students.They added comments if they decided to exclude or include something outside the TLS, explaining why.As part of the evaluation phase, teachers were interviewed in the post-implementation interview following a semi-structured structure.The basis of the interview was their diary and the researchers had the opportunity to analyse in depth some of the aspects mentioned in their diaries.Three teachers (teaching 4 groups in the same school) were interviewed at the same time and in a parallel session another teacher (who was also a member of the community in the 2019-2020 school year) in charge of 2 other groups was interviewed [11].

Results of the implementation of TLS1 and re-design of TLS2
As described in the previous sections, several instruments were used to assess TLS1.Questionnaires related to the assessment of students' achievement of the learning objectives were used to measure the effectiveness of the TLS.As for the quality of the TLS, the proposed instruments were the teachers' diary and post-implementation interviews with teachers.The analysis of the data on the effectiveness of the TLS1 was measured from two different points of view.The first analysis took into account the correctness of the responses and control and experimental groups results were compared.The pre-test answers were analysed and the two-tailed Fisher's test was applied to the percentages of correct answers in each group.The p-values obtained are between 0.57 and 0.79, all of them much higher than 0.05, which means that there are no statistical differences between the two populations.The same comparison was made for the post-test and in this case the results were different (see table 1).Table 1 shows statistically large differences in questions 1 and 4, but not so large in question 3. Question 5 is very close to the limit of statistical significance and question 2 is below the limit, which means that for question 2 both populations behaved the same.A second analysis was performed regarding the correctness of the responses of both populations, Hake's gain was measured for each population and each question (see table 2).As shown in table 2, the results are in line of the results shown in table 1.The gains for questions 2 and 5 are low and better gains are obtained for questions 1, 3 and 4, although there is room for improvement especially in question 4.However, the gains obtained by the control group are rather poorer or even negative in question 1.From the phenomenographic analysis we obtain a set of categories by analysing the students' answers.From the answers not categorised as correct we obtain the difficulties of the students and the percentage of the difficulties that persist after the experimental application of the TLS1.The data are presented in table 3. We can see that the students know very well the formulation of Newton's third law (Q3) but have significant difficulties in identifying these forces in a free body diagram of a book on a table (Q2).We can also see that the idea of impetus remains in a considerable percentage.More than a third of the students do not eliminate this idea after the application of the experimental sequence (Q1 and Q4).Another major difficulty that remains in the students' ideas is the lack of recognition of an acceleration and a force when the direction of the velocity of a body changes without changing its modulus; half of the students have this difficulty in their reasoning (Q5).On the other hand, we can see that they improved in the identification of forces after the instruction, they do not draw or mention any extra force or forget to include any (Q1 and Q2).The direct relationship of force with other quantities such as velocity or energy also decreases, although there are still non-negligible percentages who base their answer on this idea (Q3).Information on the quality of the TLS was collected from teachers' diaries and interviews with teachers.Teachers mentioned some positive aspects and some negative aspects or aspects that needed improvement.As positive aspects they mentioned increased student participation in class and interesting group discussions.They also said that the timing and the organisation of activities were realistic.On the negative side, they mentioned that there was some resistance to the methodology, especially from students who usually get good marks.According to the teachers, good students were afraid of lowering their grades and did not feel comfortable with the uncertainty of the new teaching approach.They also mentioned common problems related to the working group that may have been exacerbated by the covid19 protocols that were still in place in 2020-21.Some teachers also mentioned that some simulation-based activities were repetitive.In addition, many of the teachers felt the need for more "endof-chapter" problems.
Based on all these data, a redesign proposal is made, so TLS2 is designed.In relation to the low gain in question 2 and the persistence of difficulty B, in one of the activities (activity 4), in addition to the explanations of the action-reaction forces, a drawing of these forces is required.In addition, students are asked to identify which of them are action-reaction forces.Regarding difficulty C, the teacher's guide is changed, making this difficulty more explicit and asking teachers to emphasise it in every feedback they have the opportunity to do so.In relation to the low performance in question 5 and difficulty F, a new activity (activity 11) is included in TLS2.This activity deals with circular motion where the main objective is to work on the vector nature of velocity and to emphasise that although the modulus remains constant, a change in direction is due to a force.In relation to the data obtained from the diaries and interviews, a reformulation was proposed in one activity (activity 9) and two activities (activities 12 and 15) were modified to make parts of these simulation-based activities optional.

Conclusion and future works
This work has shown a complete application of the DBR methodology for the design of a TLS on Newton's laws for secondary schools.The methodology is completed following the three main steps: design, implementation and evaluation and redesign.The application of the design tools to define key ideas, learning objectives and Learning Demands is described.Based on the Learning Demands, the TLS1 activities (including the teacher's guide) were designed, implemented and evaluated using quantitative and qualitative tools in relation to the students' performance in relation to the learning objectives and the quality of TLS1.Based on the results obtained after the evaluation and taking into account the improvement points detected, a redesign is proposed and TLS2 is designed.This TLS2 will be implemented in the following courses.
This proposal also describes a new approach to include non-research teachers in the TLS design process.The number of teaching hours per week that secondary school teachers have to teach does not allow them time to design new innovative research-based teaching materials.On the other hand, it is difficult to implement any new materials that do not take into account the context or that teachers are not comfortable with.We believe that the approach proposed here is an opportunity to include teachers in the design process taking into account their time availability.In this way we aim to make them feel heard and part of the design team and thus encourage the implementation of a DBR-based TLS that they can implement feeling confident and comfortable.
We will continue with future implementations to measure the effectiveness of the TLS but also the quality of the sequence with respect to the teachers' proposals.

K4. 3 Figure 1 .
Figure 1.Scheme of the interactions of the Teachers Community for discussion with DBR methodology.

Table 1 .
Results of the application of Epistemologial Analysis and Learning Demands design tool within DBR methodology for the design of a TLS on Newton's law for high school

Table 2 .
5 question open-ended pre-test and post-test questionnaires p value for two tailed Fisher test applied to control and experimental groups

Table 3 .
5 question open-ended pre-test and post-test questionnaires g Hake gain for experimental and control groups.

Table 4 .
Percentages of students' difficulties among questions after TLS1 implementation (note that only more relevant difficulties are shown, so the sum of percentages of each question