Engaging theoretically primed students in second year lab courses

During their second year, about 50 undergraduate physics students engage in an open inquiry lab course of 56 hours that prepares them for their bachelor research project. Two combined theory-practical courses (of which 84 hours are spent in the lab) precede this open inquiry course. These preceding courses focus on specific research skills or the use of new theory in experiments (among others the Fourier transform and signal processing). These courses are based on Holmes’ SQI-model. The final open inquiry course that focuses on general research skills is based on Etkina’s ISLE-model. From literature we have identified 5 educational design criteria to engage students in such lab courses and implemented them. One of these was effective in our implementation. We have found that quantitative physics research as opposed to design research or focusing on certain skills is needed to make our theoretically-primed students appreciate our lab courses more. It helped our students to see the meaning of our experiments better and see the need for what we want them to learn in our lab courses. For the other 4 design criteria anecdotal evidence was found shedding some light on the way in which they may work.


Introduction and course structure
In the country our university is better known to students for its contributions to theoretical physics and less known for its contributions to experimental physics.In general, students enjoy the theoretical courses more than the lab courses.And, engaging them into experimental physics, is a challenge.
Each year about 50 undergraduate physics students start their second year in which there are two combined theory-practical courses (Physics Experiments, PE, 1&2) followed by one open inquiry course (Physics Experiments, PE3) (see Table 1).
PE1 focuses on the Fourier transform (FT), PE2 on signal processing, noise reduction, and feedback systems.These two courses culminate in a final open inquiry lab course (PE3) in which the students are free to conceive their own research project as long as it contains several aspects of the content of PE1 & PE2.
The main goal of our physics lab courses is to develop students' general research skills.Besides that, we also want them to develop more specific research (apparatus & analytical) skills and develop their understanding of the role of, and the need for, theory in physics research.This can be summarized with our main learning objective for the lab courses as a whole: Students are able to perform a 56 hour research project independently, from first conception to presenting and reporting.This way, we prepare our students for their 560 hour undergraduate research project in the final semester of their third year.

Rationale behind the course outline
Our physics lab courses have been continuously renewed from 2017 onward.
We distinguish lab courses in which the main research question is given (PE1 & PE2), from open inquiry lab courses in which students come up with their own research question (PE3).
In experiments within the first labs (PE1 & PE2), the students are focused on learning how to use certain theory, apparatus, and analytical skills necessary to answer the given research questions.We expect the students to develop those skills and to begin to understand the role of theory within experimental physics.During PE1 and PE2 the specific skills are taught within the frame of a full physics research cycle.To that end, besides the research question, some underlying theory (both physics and analysis theory) and some information on the available apparatus is given.Students have to build their experimental setup themselves.We structure the experiments according to Holmes' Structured Quantitative Inquiry Labs (SQI) [1].An important part in SQI labs is that the students go through the research cycle twice.In the first research cycle they simply try to find an answer to the research question.
In the second research cycle much of the research is reiterated but now the students either need to answer the same research question in a different context, or they need to answer a more elaborate research question, or they need to improve the precision of their results from the first research cycle.
In the open inquiry lab course (PE3) we focus on teaching general research skills.The students need to come up with their own research question within any field of physics.These research questions are 'only' limited by our students' imagination and creativity.The only requirement for their research is that it should contain aspects from PE1 and PE2.These experiments are structured according to Etkina's Investigative Science Learning Environment (ISLE) [2].In these ISLE labs, there is a natural need for iteration while students go through multiple research cycles.
Learning objectives have always been our focal point, and they are closely connected to the Dublin descriptors [3,4].Besides these learning objectives we have always tried to engage our students in the lab work as much as possible.
Engaging students in learning is the subject of ongoing research.An extensive overview is given by Leach & Zepke [5].Some of the most important findings of Physics Education research on engaging students that we have selected from literature and that we were able to implement are the following: • Assignments need to be meaningful to the students We implemented these from 2017 onward and in the academic year 2019-2020 added the changes described in the following paragraphs.
We tried to make students' assignments more meaningful by giving the students authentic challenges [6].Last year, based on students' evaluations and suggestions by our teaching assistants, we included physics research questions to each assignment.For example, students are challenged to seek the limitations of using the NI MyDAQ data-acquisition board rather than only teaching them the apparatus skill of reading and writing data using this data-acquisition board.Similar solutions were implemented for all options within PE1 and PE2.We created challenging, authentic, problem-based, physics research questions in which the need for the various learning objectives should be apparent to the students [6,7].In the same manner and because of our theoretically primed population, we also moved away from design research in PE3.The content of PE1 and PE2 (feedback systems, signal processing, and noise reduction) seemed to naturally call for design research: it was the choice in research for about half our student population.
To guide students away from design research, from 2019-2020 onward we required the students to investigate some quantitative relationship and come up with a physics research question instead of a design research question.Students still use their newly developed experimental design skills but now it is almost always intended to optimize a setup that is used to do quantitative physics research with.This changed the mood of the lab courses strongly.For example, instead of building a kart that would maintain distance to the kart in front of it automatically, students would now use such a setup (after building it themselves) to investigate how large the magnitude of changes in speed for the first kart could be for the karts behind not to crash into the first one (or each other).
To make the students feel competent we give the students assignments at their level so they can experience successes while learning [8,9,10].The relevant theory will be discussed just before the lab sessions in the connected theoretical part of the courses.Students will also be given the chance to work on exercises concerning the theory in exercise classes before the practical sessions take place.Of course this level may still be challenging for some students.Furthermore, to give the students time to get acquainted with and master their newly developed apparatus and analytical skills we do not grade the first parts of the lab courses.
We try to give the students some autonomy by giving them choices from a list of various options during PE1 and PE2.For example, there are 3 options for the students in the PE2-session on noise characterization: avalanche noise in diodes, pink noise in a thermistor, or optical shot noise in CCD cameras.During PE3 we give the students full autonomy in choosing their physics research [8,9,10].By creating pairs in which students need to work together with each other, their teaching assistants and their teacher provide the students with a supportive and collaborative learning environment [8,9,10].
An overview of our adaptations is given in Table 1.Working in pairs together with teaching assistants and teacher All of the above resulted in the following description of our lab courses.In our research we want to find out whether these criteria from literature can be implemented to increase students' engagement in our lab course.Therefore, our research question becomes: In which way and to which extent do the above educational design criteria have a positive effect on engaging students in our second-year undergraduate lab courses?

Research
We followed our students in the year before the implementations (2018-2019), the year of implementation (2019-2020), and the year after as well (2020-2021).For every course, students are asked to fill in an anonymous evaluation form.Unfortunately, not every student does so.
Questions from the evaluation form that are applicable to our research were the following:

Educational design criterion
We use educational design research [11] to evaluate our implementation of educational design criteria.This means that we will measure the effects of the various design criteria of our implementation by the results on the various questions of the student evaluation and triangulate those with student remarks during the course and our experiences while teaching the lab courses.This way we can assess which design criteria had the biggest effect.The students' remarks are intended to shed light on how the design criteria work.
An overview of how we will analyze the various responses of students to the evaluation form is given in Table 3.
The number of students and the number of responses to the evaluation through the years are given per course in Table 4.The number of responses to the evaluation form were especially low during the courses PE2 and PE3 in the academic year of 2019-2020 because at that time, due to the COVID-19 pandemic, the evaluation forms were given online.In addition, students were working solo, instead of in pairs, and at home, with equipment bought through the university.
During the academic year of 2020-2021 teaching slowly returned to normal.During the PE1 course students were still working solo but no longer at home so we were able to hand them the evaluation form on paper again.From then on, the number of responses returned back to normal.

Results
We will discuss each design criterion and connected implementation separately.During all of the years reported on, the percentage of students that passed each of the courses remained constant at about 95%, varying only between an incidental 90% and an incidental 100%.This comes down to maximally 6 students failing one of the courses.

Meaningfulness & Need to learn: focusing on physics research questions
Let us start with an example description of how the courses were changed.Our example concerns the first lab course: PE1.
Before the changes, every assignment within PE1 was meant to contribute to developing an automated spectrum analyzer: 'The final goal for Physics Experiments as a whole is being able to perform and analyze any experiment that incorporates feedback and noise reduction.To be able to do so you will first need to create a spectrum analyzer that can analyze various experimental systems automatically.Today you will learn how to read and write signals onto the MyDAQ using python.'This task was subdivided into small and concise assignments without research questions from which students were meant to learn how to read and write using the MyDAQ data acquisition card: 'Write some sample voltages to the MyDAQ and connect an oscilloscope to the output of the MyDAQ to prove that the signal that you are measuring with the oscilloscope is indeed your intended signal.' In 2020-2021 the need to learn how to read and write using a MyDAQ was clarified by the following: 'Many physics experiments nowadays are controlling the setup and measuring some quantity automatically.'The connected assignment for the students was changed into answering the following physics research question: 'What is, using the MyDAQ, quantitatively the highest frequency you can observe with a given sample rate?'.The assignment was also no longer subdivided into smaller pieces.
Both before and after the implemented changes, students remarked that they found the practicals fun and useful.Before, several of them stated for example 'the practicals are fun', and 'practicals are interesting and useful'.Afterwards, they stated for example 'fun and useful practicals', and 'the acquired knowledge is useful'.So in student remarks we could find no effects of the implemented changes.
However, students' appreciation for the preparing combined theory-practical SQI-courses grew after implementation in 2019-2020 (both PE1 & PE2 showed a growth from about 6 out of 10 to about 7 out of 10: see Table 5).This growth may be explained by one or both of the following changes.The newly implemented physics research questions may be more engaging and meaningful to students than design research questions and learning plain apparatus and analytical skills.Or the need to learn the specific skills had become more clear to the students by connecting the assignments to physics research.
Where students' appreciation for the open inquiry course (PE3) was already high before the changes, it remained high after as well.This makes PE3 the highest appreciated lab course in the undergraduate physics curriculum.By being so highly appreciated, with a score of about 7.5 out of 10, PE3 challenges the highly rated theoretical courses.Only a few of those theoretical courses score higher (around 8.0).Before 2019-2020 students made remarks on, for example, why they needed to build a spectrum analyzer or use a GUI to control their data acquisition ('learning how to create a GUI is a nasty module').The need to learn this skill apparently was not clear to the students and our teaching assistants regularly had to explain why the students needed to build the automated spectrum analyzer.
After changing to physics research questions for each experiment we no longer encountered such remarks.Our teaching assistants confirmed that students seemed to understand better why they needed to learn what we wanted them to learn.
Both, the need for the given experiments and the meaningfulness of those experiments, seem to have become clearer to our students.

Competence: Assignments at the level of the theoretical part of the course
Before the changes, theory would have been taught well before using it in the lab.After the changes we had synchronized theory and labs as much as we could.Students would be taught new theory through lectures and exercise classes and would then be able to apply that knowledge directly in their next lab session instead of having to wait for a longer period.
Before this synchronization, a few students remarked that the 'Practicals and theory are disjunct' or that the assignments were unclear.These students did not feel competent.After the synchronization, students' remarks changed to 'Good connection between practical knowledge and mathematics.''Good connection between theory and labwork.' 'The fact that by following the lecture you understand what you are doing in the lab.'The number of explicit positive remarks about the connection between theory and practicals grew a little for both PE1 and PE2 (from 5% to over 10%: see Table 6).Generally, there were fewer negative remarks.For PE3 there was no connected theoretical part.
The perceived difficulty of all 3 courses remained about the same (also in Table 6).These scores that are mostly just above 3.0 mean that the students rate the course average in difficulty: neither easy nor difficult.Students that returned to the normal course of PE2 after the COVID-19 pandemic reported a higher difficulty level.We do not immediately have an explanation for this.Before implementation we were worried that some students would still find the theoretical level of the course challenging but apparently this was not so in most cases.We have managed to create assignments that are at a level that students can handle competently.The perceived difficulty of the course is only just above average.

Mastery: No grading for first parts of the lab courses
The synchronization described in 4.2 may also contribute to students' feeling of mastery of the subject.To further enhance their feeling of mastery we decided not to grade all assignments of PE1 and PE2 anymore.Instead we only graded their choice of 3 experiments out of 6 options at the end of PE1 and their two final sessions on feedback and OpAmps at the end of PE2.We did not change the grading system for PE3.
Up to almost half of the students remarked that the study load for the courses was high, both before and after the changes had been implemented (see Table 7): 'Practicals contribute too little towards the final grade.[The course is] worth more EC.'.Even though there were only very few positive remarks on their feeling of mastery, one can be highlighted: 'The combination of practicals and lectures/theory makes you really understand and master the course.It forces you [to] understand the new theory directly rather than a week before the exam'.
The general opinion of the students' remarks agrees with the students' responses to the question about perceived study load (also in Table 7).We had hoped these numbers would have gone down somewhat by not grading the first parts of the lab courses.However, students were still complaining about having too little time, mostly in relationship with the time needed to write a preparation for their chosen experiments ('too much preparation work').
For PE3 the study load went up during the COVID-19 pandemic when they had to work solo and at home but then again, in that year only 4 students filled in the evaluation form.However, even after that year the perceived study load for PE3 remained higher than before.In all three reported years there were no remarks or only a couple of remarks, on the study load during PE3.Even though many students were still acquiring a lot of knowledge in applying their skills from PE1 and PE2 during PE3.Table 7. Students' perceived study load of the second year physics lab courses 'Compared to the prescribed study load, the actual study load of this course is:' (Likert scale of 1-5; in 2019-2020 only 4 students filled in their evaluation form for PE3, negative remarks on study load have also been indicated in percentages of the number of respondents) Finding ways to reduce students' study load is still a pending item that needs to be given attention in future iterations of the courses.We need to come up with other means to give students enough time to master their new skills.

Autonomy: Choices in PE1 & PE2, open research in PE3
Before the changes all students were given the same assignments in PE1 and PE2.After changing the lab courses students got the opportunity to choose from various options.For example, before the changes, all students had to use an OpAmp to build an IV-convertor.Afterwards, students could choose from 7 options in applying the OpAmp: building an IV-convertor, amplifier, buffer, adder, subtractor, differentiator, or integrator.Or even come up with some electronic scheme using an OpAmp themselves.For PE3 nothing changed because students already had full autonomy in choosing their research subject.
Giving students autonomy within PE1 and PE2 by giving them options to choose from, did not provoke many remarks from students (see Table 1).The few remarks were never negative ('Freedom in programming: you are really creating something that is yours').During PE3 these remarks were more common, probably because of the open nature of PE3 ('There is a lot of freedom in choosing an experiment').It seems that giving options to choose from is appreciated but not as explicitly as being given a completely open assignment.The number of positive remarks however did not change much for any of the courses.Students appreciate autonomy in the lab courses but not many of them explicitly state so.The open assignments give more rise to such explicit statements.Giving students options to choose from, does not seem to influence the number of such remarks.Of course, our students do not realize the implemented changes.However, teaching assistants did notice students to be more engaged and work harder on the experiments after they were given the possibility to choose.

Relatedness & Collaboration: Working in pairs together with teaching assistants and teacher
Even before implementing extra efforts on collaboration with teacher and teaching assistants, students were working in pairs.Due to the COVID-19 pandemic, students were forced to work solo.This was the case during PE2 & PE3 in 2019-2020 and during PE1 in 2020-2021.This gave us the opportunity to see whether working solo or in pairs influenced students' relatedness and feeling of being supported.
There were not many remarks from the students on this topic.There were a few negative remarks ('Teaching assistants give different information', 'I wish there was more support before the practical started especially with ordering equipment and choosing a good research topic').But in general, most remarks were very positive ('There was a lot of support from the TA's [teaching assistants] for the preparation and during the sessions', 'Practicals with two students together is a lot nicer: more thinking capacity', 'Well attention and willingness to bring support towards students with special circumstances and needs').
In Tables 9 and 10 one finds students' appreciation for the teacher and the teaching assistants.These numbers have been quite high and did not change much over the years except for PE3 in the year 2019-2020 when only 4 students filled in the evaluation.Perhaps, we have reached a ceiling for appreciation and always managed to maintain a good relationship with the students.For PE1 teacher appreciation did jump from 7 to about 8 in the year of our new implementation.That year there was a change of lecturer for the theoretical part of the course, so we assume this change has got nothing to do with the lab part of the course.
In the times in which students were not able to work in pairs, we did not observe significant changes in students' appreciation of the teacher and teaching assistants.Likely, working in pairs does not influence students' appreciation of the teacher and teaching assistants.But, as stated above in the quote by one of our students, working in pairs is appreciated.Just like the way we have shaped the collaborative relationship between the teaching team and the students.

Conclusion
By changing our focus from design research to physics research questions and connecting those to current research being done at our institute, our students find the lab courses more engaging and meaningful.They also see the need to know what we want them to learn better.
By timing the assignments to take place just after the relevant theory has been discussed in lectures and exercise classes, students feel competent to take on the given assignments.Unfortunately, this did not coincide with a hoped-for decrease in perceived difficulty of the courses.
The perceived study load of the courses remained high as well.This mainly seems to focus on the time students need to prepare their experiments.We need to find other ways to reduce students' study load in order to create time for students to master their newly developed research skills.
Students appreciate being given autonomy in our lab courses, but not many of them explicitly state so.The open assignments give more rise to explicit positive remarks on this.Giving students options to choose from, does not seem to influence the number of such remarks.
Not seeing big changes in teacher and teaching assistants' appreciation, we think that even before the new implementation of the lab courses we already maintained good and collaborative relationships with our students.Working in pairs or not did not seem to influence this.For the future, we intend to create even more authentic research experiences for the students by placing them in fictitious research groups, each headed by one of our teaching assistants.

Ensuing challenges
This research has focused on only a small and quite specific population of students.Further research in other populations needs to be undertaken to confirm or disprove our findings.
Course appreciation (Likert scale of 1-10) ('My overall rating of this course on a scale from 1 to 10 is:') • Perceived difficulty of the course (Likert scale of 1-5) ('In terms of difficulty, the course is:') • Study load compared to ECTS (Likert scale of 1-5) ('Compared to the prescribed study load (1 EC = 28 hours including contact hours), the actual study load of this course is:') • Teacher appreciation (Likert scale of 1-10) ('What is your overall opinion about this lecturer?')• Teaching assistants appreciation (Likert scale of 1-10) ('What is your overall opinion about assistant …?') • Open positive remarks on the course by students ('Strong points are:') • Open negative remarks on the course by students ('Weak points are (please suggest improvements):') If the reader is interested in the full list of questions of the evaluation form they may contact the author.

Table 6 .
Students' perceived difficulty of the second-year physics lab courses 'In terms of difficulty, the course is:' (Likert scale of 1-5; in 2019-2020 only 4 students filled in their evaluation form for PE3)

Table 1 .
Implementations in 2019-2020 of how to engage students in our lab courses Educational design criterion Implementation Meaningfulness & Need to learn Authentic challenges with physics research questions Competence Assignments at the level of the theoretical part of the course Mastery No grading for first parts of the lab courses Autonomy Choices in PE1 & PE2; open research in PE3 Relatedness & Collaboration

Table 2 .
The outline of the second-year physics lab courses

Table 3 .
Analysis matrix for our educational design criteria

Table 4 .
Student participation and evaluation response in the second-year physics lab courses ( * in 2019-2020 students were working at home and solo during PE2 & PE3; ** in 2020-2021 during PE1 students were still working solo but on campus) Normally students would work in pairs on campus and fill in the evaluation form individually on paper.

Table 5 .
Student appreciation of the second-year physics lab courses: 'My overall rating of this course on a scale from 1 to 10 is:' (Likert scale of 1-10; in 2019-2020 only 4 students filled in their evaluation form for PE3)

Table 8 .
Positive remarks of students about being given autonomy in the second-year physics lab courses (in 2019-2020 only 4 students filled in their evaluation form for PE3)

Table 9 .
Student appreciation of teachers for the second-year physics lab courses 'What is your overall opinion about this lecturer?'(Likert scale of 1-10; in 2019-2020 only 4 students filled in their evaluation form for PE3)

Table 10 .
Student appreciation of teaching assistants for the second-year physics lab courses Average of 'What is your overall opinion about assistant …?' (Likert scale of 1-10; in 2019-2020 only 4 students filled in their evaluation form for PE3)