Using the Design-Based Research Approach to Develop and Evaluate a New Study Module about Knowledge Acquisition in Science

The research approach design-based research has become more popular in recent years, also in the field of science didactics. Representatives of this field usually explicitly distinguish themselves from evaluation. In contrast, we have formed a synthesis of both research approaches for our development and evaluation project. By conducting the development and evaluation according to a design-based research process, we are able to take advantage of the added value of the design-based research approach. This article describes the philosophy of our research with the image of the didactic engineer and gives an insight into the implementation through the concrete description of our project, the development and evaluation of a new study module on knowledge acquisition in science for pre-service teachers. Finally, we discuss the added value of the elements of design-based Research compared to a pure evaluation and where the limits lie regarding the claims of design-based research.


Introduction
"My goal is to make learning and teaching as good as possible.Thereby I do not only stick to aspects that have already been proven to be successful, but also constantly refine teaching, linking it to current innovations."This saying could come from an educational researcher at university as well as a practicing teacher at school or university.Even if the goal is the same, the way to reach it is controversial.While researchers will probably rely primarily on scientific principles, teachers will mainly use their experience.This often leads to both sides confronting each other with claims.Practicing usually accuse the researchers that their ideas cannot be implemented in practice because their results only apply under laboratory conditions.Conversely, in the case of successful individual examples from practice, the researchers usually criticize that the factors of success were not systematically investigated, so that a transfer to further cases beyond the individual case is problematic [1].This dilemma is often summarized under the term theory-practice gap [2].Being aware of this gap, there are now more and more research approaches that attempt to build a bridge between the two extremes [2].One approach that has become increasingly popular in recent years is the design-based research approach [3,4].In this article, we would like to give an example of how we used design-based research to innovate a teaching concept.
In particular, we would like to highlight how we have differentiated design-based research for our project and what our understanding of this approach is.Central to this is our image of the subject didactic engineer, which is presented in chapter 2. After the theoretical introduction to our research approach, we give an insight into the real implementation of our idea of design-based research in a project in Chapter 3. The new teaching innovation that was developed in this project is a module for student

The Design-Based Research Approach in Educational Research
Design-based research is a relatively new approach in educational research.In their 2012 article, Anderson and Shattuck collected research projects that use design-based research [5].They come to the conclusion that this research approach has entered educational research primarily since the turn of the millennium.The predominant reason for this is to overcome the theory-practice gap mentioned in the introduction.The authors already warn there that the promise of completely closing this gap is an illusion [5].
The term design-based research was coined in the context of educational research by the designbased research Collective in 2003.This term refers to a process that develops theory-oriented solutions to concrete problems from practice [6].Such processes can be set up in very different ways; therefore, the concrete implementation of design-based research projects is also very different.In a comparative analysis, however, Reimann identifies four central aspects that are common to all research that follows the design-based research approach [1].The first commonality is motivation: researchers who choose design-based research want to make a difference in practice.Therefore, the second important commonality is that they want their research to provide an immediate answer to a problem in practice.This answer is given in the form of a "design", i.e., the researchers see it as their essential task to design and develop something new, an innovation.The high value or the central position of design is the third common feature.Despite this "design belief" [1], empirical research and the theoretical foundation are equally important, which is the fourth common feature.Researchers of design-based research are thus characterized by the fact that they assign a high value to practice innovation as well as to theoretical work.Whether this automatically bridges the theory-practice gap remains to be discussed.In addition to the common features listed by Reimann, the literature also mentions the cyclical approach as a common feature [7,8], i.e., the process of testing and revising the design in several cycles.
The basic idea of design-based research is reminiscent of evaluation research.Evaluation research is also about developing, testing, and refining a new concept.In addition, this research approach also uses multiple methods (mixed method) to be able to evaluate the effectiveness of the new concept.Most authors from the design-based research community specifically distinguish themselves from evaluation research by pointing out that, in contrast to design-based research, evaluation research is not based on several development cycles and, above all, is less theory-based.Design-based research places a high value on theory-based development on the one hand and on the other hand on the fact that results also have an additional value for basic research -and do not only make a statement about the individual case as in evaluation research.In this paper, we argue that this clear demarcation is not absolutely necessary but show how we apply a synthesis of both approaches for our project.The innovation we are presenting relates to a general problem from practice in the sense of design-based research but is implemented in a very institution-specific way, so that it is rather reminiscent of the individual case from evaluation research in this point.In accordance with the design-based research approach, the development work takes place in a very theory-based manner.The optimization also exhibits typical design-based research cycles.The extent to which the results are helpful for basic research will be discussed at the end.First, we present our approach below by illustrating our research philosophy through the image of the didactic engineer that we have developed.

The Didactic Engineer
In our procedure, we find the basic characteristics that Reimann assigns to the design-based research approach.We put the design in the center, however at the same time we attach importance to accompanying research, both in the preparation phase and in the optimization phase.In order to make our working method transparent for colleagues from the STEM field, we have developed the image of the didactic engineer for our research approach (Figures 1, 2).By comparing this image with that of an engineer whose main goal is to develop a high-performance machine that is oriented towards the customer's needs, we would like to make the basic procedure and our research philosophy understandable within the framework of the design-based research approach.
The research philosophy of the didactic engineer (Figure 2) is in principle comparable to that of the classical engineer (Figure 1): For both, the focus is on an innovation that is to be designed -for the classical engineer, the innovation is a machine; for the didactic engineer, it is a teaching concept.The need for this innovation is based on a motivation, a concrete practical problem.In addition to this comparable central goal, the innovation, the processes that contribute to the development and improvement of this innovation are also comparable.In both mechanical and didactic engineering, there is a preparation phase and one or more optimization phases.In the preparation phase, the innovation is theoretically prepared by reviewing existing research.While the classical engineer looks at basic scientific literature and preliminary developments of the machine, the didactic engineer looks at basic literature from the area of anticipated teaching content, didactics and pedagogy and looks at already existing didactic development projects.However, since both engineers are working on a concrete practical problem, it is also necessary to ask the future customers for whom the innovation is to be the answer to the practical problem -in the case of the didactic engineer, these are not customers but future learners.In the optimization phase, which usually consists of several optimization cycles, the engineer then usually asks himself two questions: firstly completely objectively about the performance of the new machine, and secondly about the satisfaction of the customer.Transferred to our didactic engineer, he asks the questions about the performance increase and the satisfaction of the learners.
The working method and intention of the didactic engineer are thus comparable to the work of the classical engineer.Through this comparison, however, one can above all also illustrate the philosophy and the interplay with basic research.The basic research that (mechanical) engineers work with is physics.It provides all the necessary theories and laws obtained in the laboratory under strict variable control.For every engineer it is indispensable to know these and to deal with the corresponding findings from basic research during the development of an innovation.The optimization process of the innovation does not take place under variable control; usually several aspects are changed simultaneously to solve the practical problem.Thus, the results cannot be directly traced back to basic research.However, questions arise in the research process that can be investigated with the help of basic research.So, the user not only uses theories from basic research, but also returns questions and hypotheses based on the results of engineering research.This interplay between (mechanical) engineering and physics is comparable to the didactic engineer and didactic basic research.The philosophy of our approach is expressed here.The researcher is a didactic engineer, a user whose heart's desire is an innovation, which he prepares in a well-founded manner and optimizes with the help of known research and measurement methods.The measurement methods originate from basic research, just as theoretical preliminary work, i.e., basic research is held in high esteem.However, the results of the didactic engineer's research do not contribute to basic research but focus on optimizing the innovation.Only questions are returned to basic research.Our central questions, our approach and our relationship to basic research can be illustrated by this comparison.The next section shows in concrete terms how this research approach, which basically differentiates the general design-based research approach for our research, is implemented.

Procedure of a Design-Based Research Project
The preparation and optimization process of the didactic engineer presented in the previous section can be integrated almost seamlessly into a typical design-based research cycle [8].Such a cycle is illustrated in Figure 3 to explain our basic approach.
The preparatory phase leads into the first design consists of two parts, theory-led development and research-led development.The theory-led development is based on a literature review of basic research on findings relevant to the design.The research-led development consists of a survey of future learners.This first design is then realized and analyzed with different evaluation methods.Based on the results, a revision takes place, which leads to an improved design.This cycle of design revision, implementation and analysis is repeated several times until the result is so satisfactory that one can speak of a final design -even though it goes without saying that this also has to be evaluated and adapted if necessary.Based on evaluation, we have not only considered and evaluated the individual implementation groups separately but have also combined them into an overall group and ascertained which results are common to all groups and become significant due to the larger number (overall evaluation).In these results, the individual cycles are no longer visible.The diagram in Figure 3 as well as all the previous explanations in this chapter should point out how we have understood design-based research and differentiated it for our needs.In the following third chapter, the implementation of our research approach will be concretized through the explanation of the project.

Development of the New Study Module as a Design-Based Research Project
In this chapter we want to show how our theoretical considerations on the design-based research approach have been practically applied in a project.The project involves the development of a new study module for pre-service teacher students.We do not focus on individual aspects of the project, but rather on the process.On this basis, our idea of design-based research can be discussed in the concluding chapter.

Motivation
As described in Chapter 2, design-based research projects are a reaction to a concrete problem from practice.The practical problem that the MINTplus 2 (engl.STEMplus2) project wants to respond to is the fact that networked thinking is hardly promoted in schools due to the strict division into school subjects.Yet this interdisciplinary thinking is urgently needed to meet the current challenges of our time (climate change, scarcity of resources, ...).In order to promote interdisciplinary thinking and STEMorientation more strongly in schools, the MINTplus 2 project has set itself the goal of promoting networked thinking and conveying related teaching ideas already in the pre-service teacher study curriculum according to the motto "make future teachers better in order to make future classes better".To provide space for such events, the pre-service teacher study curriculum at the TU Darmstadt has been restructured.The essential element of this restructuring is the networking area, in which students learn about and discuss didactic ideas and teaching concepts across their subject boundaries.This idea represents an innovation in the German pre-service teacher education and led directly to the next practical problem.After the idea of the networking area with its competence goals was created [9], there was a lack of suitable events to implement the goals in concrete terms.Therefore, the MINTplus2 project consists of several sub-projects in which new modules are designed.Our design-based research project is one of these sub-projects.
For the design of a completely new module, the first question is the choice of concrete topic.In consultation with the other module developers of the new networking area, we were given the task of developing a module that addresses networking in the three natural sciences of biology, chemistry, and physics.These three sciences are taught separately in German schools.Teachers have usually studied only one, sometimes two of the subjects.In contrast, many natural phenomena can be better understood from a holistic perspective, e.g., vision can only be understood comprehensively through the interaction of biology and physics.For this reason, joint science teaching is also being discussed repeatedly in Germany [10].But it is not only the subject content that is linked, the process of working in the natural sciences is also fundamentally the same.Therefore, teaching concepts from this area of competence (e.g., on the topic of Nature of Science) can be implemented well in all three subjects [11,12].This fact led us to choose the topic of knowledge acquisition in the natural sciences and its implementation in the classroom as the topic of the new module.The following is an overview of the conception of the new module.

Overview of the Entire Project
The timeline of our project can be described in Figure 4 in analogy to the theoretically described process in Figure 3 in Section 2.3.The theoretical foundation is based on a literature review and the preliminary study in summer 2019.The initial design was analysed and revised a total of three times, i.e., a total of three trial cycles took place.Unfortunately, the first two were affected limitations due to the covid19 pandemic (summer 2020 to summer 2021), i.e., they were conducted digitally.Only the last cycle could take place entirely in presence.All sub-cycles of the project are discussed in the following in an overview.The focus is not on the individual details, but rather on the research process as a whole.

Theoretical and Empirical Foundation of the Design
The preparatory phase covered approximately the first year of the project in 2019.In addition to an extensive literature review, a preliminary study was conducted with pre-service teacher students in the summer semester of 2019.

Literature Research.
Based on the literature, the principle orientation of the seminar first was specified.Thus, from the four teaching concepts according to Sommer et.al. ( 2018), the concept "Interdisciplinary Approaches to Science" was selected as the central one, just as from the four competences of the Ministry of Education, the competence area "Acquisition of Knowledge" was determined as the central one.This decision based on the literature not only reflects the basic orientation of the seminar but also provided two important scales for the preliminary study [13].After determining the basic theoretical orientation, the literature was used in a survey-like manner to identify as many topics as possible related to the module topic of Acquisition of Knowledge in Science, for example current discussed concepts such as "Nature of Science" [13] or "Inquiry-Based-Learning" [14].The topics selected in this way were then arranged in a module plan.Here, too, literature was consulted in order to be able to logically justify the sequence of topics, for example.For example, Höttecke's (2008) approach of researching-discovering lessons is classified as a teaching concept for integrating Nature of Science into lessons [12].Therefore, it seems to be a logical sequence to first discuss what "Nature of Science" is and what teaching ideas there are for it, to go into more detail about the explorativediscovering teaching in the following session.In addition to the sequence of topics, the subdivision of topics into several seminar sessions can also be meaningfully justified based on literature.An example of this is the differentiation of the competence of gaining knowledge into "scientific investigation", "scientific modelling" and "reflection on the theory of science" by Wellnitz et al. (2012) [14], which was used to theoretically work through the competence area of gaining knowledge in three seminar sessions.

Preliminary Study.
For the preliminary study, a paper-pencil test was conducted with N=43 student teachers.The detailed results [13] will only be briefly summarised here.The basic orientation of the seminar could be confirmed: The results of the preliminary study showed that student teachers feel significantly less prepared for interdisciplinary teaching concepts than for subject-specific ones (r=1.28,Wilcoxon test).They also estimate their prior knowledge regarding the teaching of subject competence significantly better than that regarding the competence "gaining knowledge"" (r=0.59,Wilcoxon test).In addition, the planned topics were suggested to the students in the questionnaire and evaluated by the students in terms of perceived prior knowledge and their interest.As a result, the topics "safety" and "electronic data acquisition" were sorted out, and the topics "assessing experimental competence in everyday school life" and "experimenting in heterogeneous learning groups" were scheduled as complete sessions.In the open question about wishes, the students stated that practical teaching ideas should already be integrated in the basic seminar.They also offered the wish the project to take place in a real school context if possible.

First Design
In the first design, the basic sequence of the module was determined.The module essentially consists of three parts: firstly, a weekly seminar in which the contents of the module topic "Knowledge Acquisition in Science" are developed theoretically and with practical examples; secondly, a school project in which the student teachers implement what they have learned in a project and try it out under real conditions with pupils in school; and thirdly, a reflection phase, which is made up of an accompanying Mahara portfolio [15] and a concluding reflection discussion.The topics selected for the seminar according to the preliminary work in chapter 3.3.are shown in figure 5.They were implemented digitally in the first seminar run due to the covid19 pandemic, but this only meant a methodological change, not a change in content.The originally planned school project had to be discarded for the first implementation, as it was not permitted to carry out university projects in school during the covid19 pandemic.Instead, the project was only developed and reflected theoretically.

Implementations
The following table gives an overview of the outcarried implementations.It is to be noted that the differences seen in the table are mostly caused by the conditions during covid 19 pandemic rather than revisions based on evaluation.Sections 3.6 and 3.7 provide insight into how the seminar was further developed in the philosophy of design-based research.

Insight into an Exemplary Evaluation
Step.We first want to give an insight into how we exemplarily developed the seminar further in the first cycle based on the evaluation results.Since we wanted to record both the improvement in performance and the satisfaction of the learners with the new module according to our image of the didactic engineer, we used different methods.A major learning goal of the seminar is the expansion of conceptual knowledge regarding the module topic "Knowledge Acquisition in Science".We chose concept maps as an evaluation tool to record the increase in knowledge, as these can depict conceptual knowledge particularly well [16].We were able to determine that conceptual knowledge increased significantly over the course of the semester (r=0.89,Wilcoxon test) [17].However, when assigning the terms to the topics, we also found that the topic "Nature of Science" was hardly recorded by the students in the concept maps.In the evaluation forms after each session, the topic was also rated as less relevant than the other topics.This observation in the evaluation results inevitably led us to the consequence of fundamentally revising the session with the topic "Nature of Science".New exercises were integrated and a greater practical relevance to school teaching was demonstrated.When multiple evaluation methods were used, questionnaires in the pre-post design were also used.However, due to the small samples of the intervention groups (see Table 1), these never became significant within the individual cycles.However, across the whole group, results observed in the small samples can also be observed in the larger group with significances.For example, a questionnaire with three scales out of the PISA questionnaire was used in the pre-post design.The three scales used were Epistemological Beliefs, Interest in Science and Value of Science [18].While no changes were observed in the last two scales, the student teachers improved significantly in the Epistemological Beliefs scale [r=0.37,Wilcoxon test].This fits our hypotheses, as we explicitly addressed the epistemological properties of the natural sciences in the seminar through topics such as models of knowledge acquisition or Nature of Science.

Final Design
Due to the cycles characteristic of the design-based research approach, the seminar plan shown in Figure 4 was revised several times.The revision of the seminar plan that can be seen here is exemplary in this article for further revisions of the original design (e.g., also the design of the individual sessions).It can be seen that the basic structure and sequence of the seminar plan has changed.The example that the general introduction to subject didactic models of knowledge acquisition should be arranged directly before the more detailed aspects of the competence "Knowledge Acquisition in Science" and should not be interrupted by the topic "Integrated Science Lessons" is easily recognisable.The need for this revision arose primarily from feedback from students who did not recognise the common thread in the original variant.In summary, in the final design the basic structure of seminar, project and reflection was retained, whereby the seminar was revised in three cycles by means of design-based research.Unfortunately, these cycles could not be used as revision cycles for the project, as the project could not be carried out in school in the first two due to the covid19 pandemic.

Discussion
Our illustrations show that a simple evaluation project can be usefully expanded into design-based research.Through elements such as a stringent theoretical foundation and preliminary studies as well as a cyclically structured optimisation process, the evaluation process can be upgraded in terms of research methodology.However, it remains to be noted that in this project a complete closing of the theorypractice gap was not achieved even through this research design.The results are not so far-reaching that they advance basic didactic research; only research questions (e.g., on the conceptual understanding of knowledge acquisition in the natural sciences) can be fed back to basic research.In terms of our research philosophy of the didactic engineer, this is satisfactory; the latter 3rd World Conference on Physics Education Journal of Physics: Conference Series 2727 (2024) 012007 IOP Publishing doi:10.1088/1742-6596/2727/1/01200710 also does not contribute any basic physical knowledge through research on its innovative machine.Nevertheless, we have to conclude that our approach does not meet all the criteria of the design-based approach due to the low transferability of the results to other contexts and thus represents a weakening of the approach towards evaluation.In the future, it should be further discussed whether such an opening of the approach versus the strict demarcation between the two approaches makes sense in order to accompany and evaluate innovative teaching concepts like ours in a research-methodologically valuable way in the future.

Figure 1 .
Figure 1.Innovative development (e.g., of a machine) by an engineer with relevant questions of the preparation and optimisation process.

Figure 2 .
Figure 2. Innovative development of a teaching concept by a didactic with relevant questions of the preparation and optimisation process.

Figure 3 .
Figure 3. Procedure of a Design-Based Research Project

Figure 4 .
Figure 4. Procedure of our Design-Based Research Project

3rd 8 Figure 5 .
Figure 5. Structure of the Seminar Sessions in the First Design

3. 6
. Evaluations A brief insight into our revisional work should clarify how we have further developed the seminar based on the evaluation results of the individual cycles and what the final overall evaluation should look like.

Figure 6 .
Figure 6.Structure of the Seminar Sessions in the Final Design

Table 1 :
overview of the implementations Insight into an Exemplary Preliminary Overall Evaluation Result.