Project-based Approach to Implement a STEM-Focused Engineering Curriculum

In recent decades, integrated science, technology, engineering and mathematics (STEM) projects are occupying a central place in science and engineering curricula. The interest of this problem-based teaching approach in a scientific-technical context lies in finding ways to develop students’ content knowledge and to plan appropriate learning activities and instructional strategies. This paper focuses on designing, implementing and evaluating the introduction of STEM projects in the first year of the engineering degree of a Dual Engineering School. The focus of this study is the use of learning activities such as “predict and argument” and “design and do” to integrate STEM content into lessons and help students develop core competencies through engineering design processes. The evaluation results indicate that students moderately improve their scientific skills and conceptual understanding in the disciplines included in the project.


Introduction
Science, engineering, and technology are important in almost all facets of modern life and key to address many of humanity's most pressing current and future challenges.This requires people who are competent in science, technology, engineering, and mathematics (STEM) and so higher education in STEM plays an important role in providing this human capital which is competent in the skills inherent to the practices [1].However, the scarce number of students that choose STEM degrees at higher education is a cause for growing concern.The Relevance of Science Education (ROSE) study revealed that science, as it is taught in schools, does not help students to realise how important it is for society [2].The often fragmented, non-real-world way of teaching STEM disciplines such as physics, chemistry and mathematics in science and engineering degrees does not attract students or nurture their interest in science and technology [3].The lack or loss of interest in science and technology among young Europeans contrasts strongly with the world's growing problems, that require urgent technological innovation based on scientific research (matters such as energy, sustainability, ageing and climate change).Therefore, STEM higher education requires a new focus, where STEM disciplines regain their relevance for students as they explore connections between the different STEM disciplines.
On the other hand, STEM higher education faces the challenge of educating its students in top level skills such as abstraction, generalisation and transfer required to solve society's scientific technological problems.Creative solutions to environmental problems require not only in-depth understanding of basic STEM concepts and ideas, but also the capacity to make interdisciplinary connections [4].In this respect, not only the quantity of knowledge is important but also the depth of understanding and the ability to generate and apply ideas to solve problems.On the contrary, the bibliography [5] shows that educational practices which reflect real science and technology applications can have a positive impact on the interest and learning of the STEM students.Therefore, a STEM approach to higher education is necessary, where the subjects regain their relevance for students as they see connections between the different STEM disciplines.
An engineering degree curriculum should incorporate the relevant scientific knowledge, mathematical analysis, and technological applications to implement processes from product design conception to production.Integrated interdisciplinary curricula revolve around the engineering design process and connect the different types of knowledge acquired by students using problems [6].Many studies related to STEM curricula have considered that integrated designs with active teaching methodologies favour student learning [7].However, the bibliography [8] demonstrates both significant improvements and results that are not as good as might be expected.In our research, several STEM projects were designed around an engineering degree to be implemented in the classroom.These project designs focus their capacity to integrate content from different disciplines (physics, mathematics, IT) and use the design process in engineering to offer students the chance to integrate different STEM concepts in the project analysis and development.We ask the following research questions: -To what extent does developing and solving STEM projects embedded in the curriculum help firstyear engineering students to improve their disciplinary content knowledge?-To what extent does carrying out the activities help students to increase their scientific-technical skills?
In this paper, we are going to show one of the projects which was designed and its influence on improving conceptual learning of the different disciplines integrated in the project.

Framework to implement a STEM project in an engineering degree
The STEM training bibliography indicates that discipline integration is one of its main characteristics.This type of integration in STEM projects can be understood as the contextualised work to be developed in a complex problem where it is necessary to use the knowledge and skills from different disciplines to be able to understand it and resolve it [9].[10] consider this integration as an ability to integrate knowledge and ways of thinking from two or more disciplines to produce cognitive development.It is therefore necessary to focus explicitly on the disciplines in the STEM project that guarantee integrating at least two disciplines to be able to consider it a STEM project.
Prior studies indicate that the disciplines can be integrated at different levels and although there is no consensus on the most appropriate, three levels can be differentiated [11], [12], [13] and [14].A pre-STEM level is defined where one dominant discipline works aspects of other disciplines.The first level of STEM integration is characterised as defining a common theme among the integrating disciplines.Depending on the importance of the integration, two sub-levels can be determined: multidisciplinary and interdisciplinary.At the multidisciplinary level, a common theme is defined but the disciplines are worked on separately, determining their own learning objectives.In the interdisciplinary level, as well as defining a common theme, the learning objectives are also common, considering the overlap of the STEM content areas.
The second level of STEM integration considers project development or problem solving where it is necessary to use content and skills from different disciplines: in this case interdisciplinary, disciplinary and cross-disciplinary skills are important.At this level, two sub-levels are differentiated, depending on discipline integration.If the disciplines are maintained, this is called transdisciplinary integration and if, on the contrary, the disciplines disappear, the integration is called metadisciplinary.This study includes an engineering project that is at the second level of integration, so transdisciplinary.As we explain below, the project that is suggested to the students integrates content and skills from four disciplines in the first year of an Engineering degree.
Most of the STEM study plans are based on perspectives of social-constructivist learning [15], that emphasise student-centred learning and the process of "learn by doing".Our study has considered that an approach centred on the engineering design is the best way of teaching students how to solve a project included in the study plan for our engineering degree [16].Through the engineering design, this focus connects different disciplines and provides students with opportunities to apply multidisciplinary knowledge through different teaching approaches such as Problem Based Learning, projects, or investigation [17].Education proposals may vary slightly with the focus.In our study, we chose an educational Project Based Learning approach (PBL) [18].PBL is a teaching strategy that sets complex tasks based on questions or problems that encourage students to design possible solutions, solve problems, take decisions, or research activities that allow them to work autonomously, working on real products to increase their motivation.Project Based Learning is generally defined in six steps: (1) firstly, a context should be set for the phenomenon or the model that you wish to analyse during the project.This context or problem must be interesting and familiar for students, although it should not have a sole or obvious solution.(2) After the presentation, the students must be allowed to express their ideas and initial hypotheses on the project that they have just presented using graphs, drawings, and written arguments, etc.Consequently, the knowledge acquisition process starts on this basis.(3) During the process, the evidence must be prioritised by means of planning, evaluation or development of a design that can be closed, structured, guided, or open.The decision depends on the students' PBL experience and skills.(4) To make the project more realistic, it is advisable to analyse real or hypothetical data that allow the results to be represented, assessed, and connected.(5) In this way, the students can confirm or reject their initial ideas using the data analysed and ( 6) solve the project according to their learning.These conclusions must be reported while the students become aware of what they have learned, how they did it and what application it had [19].Actually, working on a real problem/project in the classroom involves identifying the professional skills targeted as necessary in a project-solving process that is represented in our study by figure 1.

Study context
To answer the research questions, a STEM project was implemented with a group of 50 students (46 male and 4 female) in the 1st year of Engineering at the IMH Dual Engineering School.The chosen type of STEM integration is the second level where the disciplines involved are maintained in the project, therefore this is transdisciplinary integration and the disciplines involved are Physics, Electrical Physics, Information Technology and Mathematics.
The STEM project is imbibed in the curriculum throughout the whole semester (see table 1).The project is presented in two 2-hour sessions at the beginning of the course.During the classes of each discipline, constant references are made to the contents related to the project, which the students must take into account in their development when filling in the project worksheets.At the end of the semester, two more two-hour sessions are held to discuss the students' doubts about the development of the report and tutoring is offered by the four lecturers from each discipline involved in the project.A real problem is set, asking the students to carry out a study on the efficiency of the machining process for three machines in a company.The formulation of the project is as follows: In the company MAFIN, S.A. the piece with reference AFX29913 is mechanized in three different machines: i) KONDIA-SEASKA; ii) DANOBAT TM-750; iii) IBARMIA ZVH-38 L1600).Each machining process follows a different process of work for made the same piece AFX29913.The data of the machining process of each machine, necessary to develop the project, will be presented in a table.The main objective of the company is to analyse which is the most competitive machine based on the following criteria: 1) Lower energy costs; 2) Machine efficiency; 3) Process speed.The starting billet for the three processes is the same, a low-alloy steel F-1280 measuring 200x100x80.For this study, an analysis of anomalies detected in the electrical system of each machine is also required.
One of the objectives of the project is that first year engineering students, on the one hand, appreciate the interest of projects that improve the production system and optimise the use of energy.On the other hand, to familiarise them with engineering practices.The project-solving teaching with the students was developed in three phases.
The first contextualises the project (interest), working to identify the necessary knowledge from the various disciplines with scaffolding questions (interdisciplinary knowledge) and determining steps to carry out the project (practice engineering skills).Students are guided through the project by worksheets that propose activities that order the resolution process.Below is the worksheet that helps groups of students to contextualise the project.

Figure 2: Contextualisation phase worksheet
The second phase groups together the results obtained in each of the disciplines to give traceability to the report which describes the study carried out on the machines to identify the most efficient machining process of the three.Finally, in the third phase, the project was developed using the work carried out in the two previous phases.One strategy to structure how the project set for the students would be solved is to give them worksheets (a worksheet contains tasks related to the questions or problems from the project that the students must solve).Worksheets are highly versatile and can be adapted to meet specific goals [20].The worksheets were written by the research team which made it possible to pinpoint very specific goals set in the project.The worksheet design criteria followed the results obtained by the research in teaching strategies with worksheets [21].The students worked in Name of the students' group: - ----------------------------------------------------------------Once you have read the project, we propose you to answer some questions in order to clarify the issues of the project.1.-Explain the machining process we are going to analyse.What is the starting workpiece and what the final workpiece?What process is followed to obtain the final workpiece?
2.-What variables are considered to assess the efficiency of the machining process of the three machines?Explain the meaning of each variable.groups of six or eight, and each group had to hand in their worksheet.Three worksheets were designed, one for the initial phase and two for the final phase.

Results
This section shows the results obtained after implementing the STEM project.Regarding students' understanding in the disciplines, the results are obtained using a pre and posttest design relating to the conceptual learning achieved in each of the disciplines included in the project.In this paper, we are referring to the results obtained in the discipline of physics.For the pre and posttest design, objectives were defined for learning to be achieved on the concepts and theories of physics included in the project (see table 2).The objectives are specified by defining learning indicators for the questions in the questionnaire.Table 3 defines the assessment protocol based on the indicators for objective O1 (indicators I1, I2, I3) and objective O2 (indicators I4, I5).The data observed in table 3 show an improvement in the conceptual learning regarding the physics contents worked on in the project.The outcomes from the pretest indicate values in most of the indicators below the mean value of 2.5 except for indicators I.1 and I.5.However, the outcomes from the post-test and measured after implementing the project show values above the mean.
The analysis was carried out by two of the authors, separately, contrasting the protocol criteria with the students' answers.Once the first results were obtained, a joint meeting was held to see the evaluation differences.The scores were very similar and when there were differences, they were resolved by means of agreement based on what the students wrote.At the end of the meeting, there was strong consensus between the researchers and the indicator evaluation.Cohen's kappa reliability coefficient was 0.91 for the average values of the indicators, which is satisfactory for a confidence interval of 95%.
After this analysis, each indicator in isolation shows improvement, and this is significant in the case of indicator I.3.Therefore, the students' ability to "represent the value of the forces and positions correctly in a graph" has improved after implementation.
As for I.1, despite being above the mean value, after implementation it does not show a significant improvement, its value remains in level 2 of the section, which indicates that the student has problems measuring the magnitudes in accordance with their units.This might indicate problems with the mathematical calculation.
Another indicator that has significantly improved, exceeding the mean value, is I.2.In this case, the result indicates that the students have improved their comprehension of the concept of variable force and are capable of analytically interpreting the functions represented by the variable forces in each of the graphs included in the activity.
If we look at I.4 that measures the students' ability to calculate the work of a variable force analytically and numerically, the results have improved since implementation, it is seen that the indicator value after implementation is also over the mean value, considering a less significant improvement given the importance of this concept in developing the project.
In the case of indicator I.5, the improvement is not significant which shows us that the students present difficulties when calculating power from the point of view of Newtonian mechanics.The strategy to teach this concept should be improved in future versions of the project.
To round off, it should be highlighted that the students developed learning objective 1 better than learning objective 2. This result indicates that running the activity led to conceptual improvement of the concepts worked on in the activity.However, no improvement was seen in how to solve the problems.
Although this study has focused on the quantitative and qualitative analysis of the evolution of the learning objectives of the discipline of Physics within the STEM project that has been proposed, data have also been obtained from the other disciplines (Mathematics and Computer Science).The data from these two disciplines have not been analysed in such detail for the moment because they are going to be part of another study.
Below is a representation of the data obtained from the pre-post-test for both disciplines in the form of a bar chart.
In the case of Mathematics, most students show progress in learning the concepts embedded in the project.However, almost half of the students are at level 1 of learning for most of the learning indicators.In the case of Computer Science, an advance in the learning of the concepts integrated in the project work can be observed, being more notable in the case of learning indicators:1 (Import data into a spreadsheet from an external file in CSV format), 4 (Use functions to carry out each of the intermediate calculations identified) and 5 (Identify the libraries necessary to carry out the different calculations).In the rest of the indicators, the level of student learning is lower, but with better results than in the case of Mathematics.The results obtained in learning objective 1 should be highlighted.
For assessing the students' ability in use engineering skills, one of the instruments used was the analysis of the students' groups' final report on project resolution.We used a rubric of scientifictechnical skills based on the epistemological skills in engineering work [1,9].The rubric has four levels in increasing order from poor/low use of the skill (level 1) to excellent use (level 4).The reports of the 16 groups of students who carried out the project were analysed.The results are showed in table 5 In the reports (N=16) most of the groups use skills 3,4,5 and 6 well or very well, to explain how they construct the steps to reach a solution, argue based on evidence and consequently construct explanations.In most of the reports the definition of the problem is not explicitly stated (S1) and only a minority explain the planning of the project development (S2).In informal conversations with the students, when asked about the lack of these explanations in the reports, they argue that the definition of the project is already in the statement itself and that the steps for its development are implicit in the different sections of the report.They do not consider further explanations necessary.However, both aspects are necessary to detect possible erroneous approaches when planning the project (Skill.1)and when communicating the development of the project to external participants (Skill.2).It will be necessary to insist on these skills in later implementations of the project

Conclusions
In this study we have designed and evaluated a project for students starting the Engineering Degree to familiarise them with scientific-technical practices and improve their understanding of concepts in an interdisciplinary context.The study attempts to answer two questions about the effectiveness of projectbased teaching in relation to improving the understanding of disciplinary concepts and the use of engineering work skills.In relation to the first research question, an overall improvement in student learning is observed in the pre-post-test for all disciplines.However, the post-test results are not as good as expected for most of the mathematics items and for some of the physics items.For example, in physics, there is a clear need to provide students with more activities focused on the calculation of magnitudes and their units in the different metric systems when results are obtained in the development of a problem or project.Likewise, students have difficulties in calculating the work of a variable force.This result coincides with the results of other studies that mention the persistent difficulty in learning about the concept of work with variable forces.This result is consistent with the results of other studies that mention the persistent difficulty in learning the concept of working with variable forces.Another aspect to improve is the design of the sequence of conceptual questions in the development of the project, corresponding to the discipline of Mathematics.In this discipline, the improvement in learning is significant, but the results of the post-test indicate a low level of learning.In the case of Computer Science, it would be worthwhile to focus in the classroom on the concepts that the data show the least learning by the students in the development of the project.
In relation to the students' use of scientific and technical skills in the resolution of the project (second research question), the results are encouraging, although they show that it is necessary for students to have opportunities to use scientific and technical skills over a longer period of time.It is not enough to solve a single project for students to use engineering skills with ease.This study has been carried out in a specific centre, the Dual Engineering School-IMH, and its results will have to be seen in other engineering centres in order to be able to draw generalisable conclusions.However, the results obtained are consistent with other experiences at international level [23].The design of more integrated projects within the Engineering degree curriculum and cooperation with other Engineering Schools will be tasks that we intend to develop in the future.

Figure 3 :
Figure 3: Results of pre and post-test in Mathematics

Figure 4 :
Figure 4: Results of pre and post-test in Computer Science

Table 1 :
The project in the curriculum in the first year of Engineering Degree

Table 2 :
Physics learning objectives included in the project

Table 3 :
Assessment protocol for learning indicators in Physics.

Table 4
shows the outcomes from the pretest and posttest in relation to the students' conceptual learning in the physics discipline, measured from the indicators defined in table3.

Table 4 :
Results of pre and post-test in physics learning objectives

Table 5 :
Results of analysis of the reports of the 16 groups of students who carried out the project