Impact of working with Arduino on mathematics and science teacher students’ self-assessment of TPACK and self-efficacy

To prepare mathematics and science teacher students for the implementation of digitally transformed teaching, we are developing and researching a course using the design-based research paradigm at the University of Graz. Based on the findings of curriculum analyses and surveys, we selected two main content areas for the course: digital data acquisition with Arduino and dealing with misinformation. Learning arrangements were developed based on empirical findings and theoretical foundations. The initial implementation of the course took place in the summer semester of 2022 with 17 teacher students. We examined the teacher students’ learning processes and the effectiveness of the learning arrangements in terms of the teacher students’ learning using a mixed-methods design (pretest, posttest, reflection journal, and field observations of course instructors). This article presents the triangulation of the research findings related to the work with Arduino and the derived criteria for redesigning the learning arrangements. The results show that self-assessment of technological competencies and self-efficacy expectations related to working with Arduino differ between teacher students with and without prior programming knowledge. Our findings suggest that there is a need to implement scaffolds that support teacher students as they undertake practical work with Arduino.


Introduction
Digital technologies are becoming increasingly important in general and society is undergoing change via digital transformation [1].As a result, there is a growing demand for digital competencies and the ability to creatively solve problems through the use of digital tools in the job market [2].Thus, appropriately professionalized teachers are needed to ensure that students are equipped with these skills [3].However, empirical findings have indicated that (prospective) mathematics and science teachers often have low prior experience in using subject-specific digital media, such as digital measurement systems or augmented reality applications [4,5], and partly express low self-efficacy expectations regarding the use of such tools [4].
Frankfurt Triangle [14] to form a competency model that maps digitality not only from a technological but also from a socio-cultural perspective [3].Thus, it can serve as a guideline for preparing teacher students for digitally transformed teaching.
Exemplary design criteria: a. Teacher students are offered problems that are significant to them.(assuming DC A) b.The content of the course links to the prior knowledge and attitudes of the teacher students on the subject matter and the use of digital media.(assuming DC A) c. Teacher students are provided opportunities to interact with fellow students and instructors.
(assuming DCs A & B) d.To promote understanding of algorithmicity, teacher students are provided with learning arrangements to work with Arduino and to critically discuss algorithms.(assuming DC C)

Design of the course "Facts, Fakes and Algorithms"
The first design of the course was developed along the initially formulated design criteria.Based on the results of previous surveys (curriculum analyses and survey on teacher students' attitude, experience, self-efficacy expectation and competence assessment in the context of digital media) [5], we divided the course into two parts, each covering a main content area: digital data acquisition with Arduino and dealing with misinformation [5].Working with Arduino addresses the teacher students' lack of prior knowledge in the areas of digital data acquisition and programming identified in the pre-surveys [5] (dc b).Likewise, programming an Arduino and discussing opportunities and challenges of algorithms also addresses the aspect of algorithmicity (dc d).The second part of the course focuses on the aspect of referentiality, promoting a critical approach to (mis)information (DC C).This article focuses on the first part of the course, on digital data acquisition with Arduino.This part lasted for approximately 12 hours and was implemented in two-hour units.As an introduction to working with Arduino, a digital learning arrangement was developed in Moodle.There, the basics of working with Arduino are worked through by teacher students in a flipped-classroom setting in preparation for the course [5].As the preliminary surveys indicated that teacher students exhibit heterogeneity in terms of prior knowledge of Arduino, we concluded that there needs to be a learning arrangement where the teacher students can acquire or consolidate basic knowledge of Arduino at their own pace.[5].To deepen and expand their competency acquired during the Moodle learning arrangement, teacher students worked collaboratively on physical computing projects with Arduino in the remaining first part of the course (dc c).Using 3D-printed model heads, CO2 and fine dust sensors, as well as Arduino boards, they worked to inquire questions about the effectiveness of FFP2 protective masks (dc a).The teacher students presented and discussed their collected data in the form of poster presentations at the end of the first part of the course [5].
The course provided learning arrangements that promote technological knowledge (TK) [5].In the introductory Moodle learning arrangement, teacher students could acquire basic knowledge of the structure and operation of the Arduino board, sensors, and the Arduino integrated development environment (IDE).In addition, they were taught fundamentals of the Arduino programming language.Within the framework of the subsequent collaborative physical computing project, teacher students could apply and expand their theoretical knowledge.The course also provided learning arrangements to promote technological content knowledge (TCK).As part of the physical computing project, teacher students worked to inquire a mathematical-scientific question about the effectiveness of FFP2 protective masks using an Arduino board, CO2 and fine dust sensors, the IDE, and Microsoft Excel [5].
In the course, there were few learning arrangements designed for teacher students to develop technological pedagogical knowledge (TPK) and technological pedagogical content knowledge (TPACK).Teacher students largely took on the learner role when working with Arduino.However, they created checklists for using Arduino in school, which have methodological and educational aspects, as well as for the equipment that is required to use Arduino in school.Furthermore, reflections and discussions regarding the use of Arduino in school were stimulated on a metal level [5].Despite the few learning arrangements for specifically supporting TPK and TPACK acquisition, we assume based on other research that changes in self-assessment of TPK and TPACK can occur [14].

Research aims and research questions
To enable teacher students to implement digitally transformed subject teaching, it is first necessary to ensure that they feel confident using digital media in the classroom.Hence, one aim of the "Facts, Fakes and Algorithms" course is, to support teacher students in using digital tools-Arduino-in their subject lessons.TPACK self-assessment has been shown to impact the intention to use digital media in the classroom [15].Additionally, the positive influence of a high self-efficacy expectation in the area of digital media on the use of digital media in the classroom has been outlined by [16].Together, a high TPACK self-assessment and a high self-efficacy expectation increase the probability that digital media, in this case Arduino, will be used in school [15,16].
Therefore, we investigated the implementation of the developed course on digital data acquisition using Arduino with the following main research questions (RQ), among others: • How does the designed course format affect teacher students' self-assessment of their TK (RQ1), TCK (RQ2), TPK (RQ3), and TPACK (RQ4)?• How does the designed course format affect teacher students' self-efficacy expectations for using Arduino to solve mathematical and scientific problems (RQ5)?

Sample
The first implementation and research of the course took place in the summer semester of 2022 with 17 teacher students specializing in mathematics and science subjects.

Methods
To provide the best possible insights into teacher students' learning processes, a mixed-methods approach was chosen to address the research questions.The qualitative data used in this study were derived from reflection journals.Each teacher student kept a digital reflection journal, to reflected their personal learning processes.While working with Arduino, each teacher student made three entries: one entry after working with the Moodle learning arrangement (entry 1), one entry after deepening and expanding their knowledge by using different sensors (entry 2), and one entry after implementing the group project on physical computing (entry 3).Quantitative data were collected by conducting a pretest (before working with Arduino) and a posttest (after implementing digital data acquisition).Pre-and Posttests were administered online and assessed TK, TCK, TPK, and TPACK.Self-report items from [17] were used for this purpose.The constructs of TK, TCK, and TPK were assessed using four items each, scored on a 5-point Likert scale, and TPACK was assessed using five items that were scored on the same 5-point Likert scale [17].
The teacher students also provided information on their self-efficacy expectations in the context of digital data acquisition with Arduino.Items from [4] were used and adapted for this purpose.Selfefficacy expectancy was assessed using four items that were scored using a 4-point Likert scale and a "don't know" option [4].The internal consistency of the scales used was assessed by calculating the respective Cronbach's alpha (at least .7).The scales were therefore considered suitable for this context.In the posttest, the teacher students also evaluated the extent to which their TPACK and self-efficacy expectations had changed compared to the pretest stage.Furthermore, they reported the cause(s) of this change (if any).
We used the Wilcoxon signed-rank test to determine whether the teacher students' self-assessment of TK, TCK, TPK, and TPACK and their self-efficacy expectations differed in the pretest and posttest.A post hoc power analysis showed that for a sample size of eight, with logistic distribution of the data, an effect size of r = .478is required for a statistical power of 1 -β = .9and a level of significance of α = .05.We used the Mann-Whitney U test to compare the groups of teacher students with and without PPK.A post hoc power analysis showed that for sample sizes of eight and nine, with logistic distribution of the data, an effect size of r = .578is required for a statistical power of 1 -β = .9and a level of significance of α = .05.Given that our sample was too small to reliably detect effect sizes up to these values with statistical significance, we do not report non-significant results because the risk of statistical type II error is too high to interpret the results quantitatively.The obtained quantitative results were combined with the qualitative results derived from the teacher students' statements in their reflection journals to formulate statements about teacher students' learning processes and the further development of the course design.These statements only refer to the group of teacher students investigated and are not generalized.

Results
Results are reported first descriptively and then according to the research questions posed in Section 4.

Descriptive statistics
Table 2 shows the descriptive statistics associated with the teacher students' assessments of their TK, TCK, TPK, TPACK, and self-efficacy expectancy in the context of using Arduino.The data were obtained from the pretest and posttest surveys.

Influence of the course on the self-assessment of technological knowledge (RQ1)
Depending on their PPK, the teacher students rated their TK differently before and after participating in the course "Facts, Fakes and Algorithms" [18].In the pretest, all the teacher students with PPK (N = 8) rated their TK higher than most of the teacher students without PPK (N = 8 out of 9) (U = 11, Z = -2.431,p = .015,r = .589).This finding was entirely expected.Teacher students with PPK indicated that they had either the same or greater TK after working with Arduino.Due to the statistical power of the sample, we have not made any statements about statistical effects here.Six out of nine teacher students without PPK reported that their TK decreased after completing the learning arrangements (z = -2.157,p = .031,n = 9, r = .719,Mdnpre = 3.00, Mdnpost = 3.00).
The differences between the two groups of teacher students were particularly clear when the following two items for assessing TK were examined more closely [18]: • "I find it easy to learn how to use digital media." • "I know how to solve technical problems independently when using digital media." In order to make statements about possible reasons for these quantitative results, we triangulated these data with observations by course instructors and information provided by the teacher students in their reflection journals.The results show that acquiring theoretical knowledge about the set-up and functioning of the Arduino board, sensors, and IDE in the digital Moodle learning arrangement was not a major problem for either group.However, working independently with Arduino to digitally acquire data, implementing the physical computing project, and visualizing data in Excel were often described as challenging by teacher students without PPK.This was reflected in the fact that the teacher students were able to work well on their own on the first parts of the Moodle learning arrangement, which focus on the basic structure and functioning of Arduino, and had difficulties when working on the more advanced parts, which focus on the use of Arduino for data acquisition.The following text passages were extracted from the reflection journals of teacher students without PPK and have been translated from German: "The basic theory in the first three Moodle lessons was completely okay, logical, comprehensible and understandable.However, it was very difficult for me to get into practical work with Arduino in lessons 4 and 5, because I simply do not have experience with these technical gadgets, programming etc. (...)." (student 13, entry 1) "I think the basics of the Arduino board were quickly internalized, but as soon as I started working on the laptop, I came up against certain difficulties."(student 5, entry 1) In comparison, the teacher students with PPK indicated in their first reflection journal entries that they could work well with Arduino and that the system is very intuitive and user-friendly.The following are some examples of such comments, and the text passages have been translated from German: "The handout was a bit much to read but even without it, all tasks could be mastered.With some prior experience of programming, most of it works intuitively."(student 17, entry 1) "For me it was something completely new to work with an Arduino, but I think that it is very userfriendly.Although I already have mini-experience with programming, it was still challenging again briefly, but in the meantime, I find my way around."(student 16, entry 1) PPK made it easier for the teacher students to learn theoretical as well as practical knowledge and skills in the context of using Arduino.All in all, the teacher students with PPK gave quantitatively higher self-assessments regarding their ability to solve technical problems than the teacher students without PPK.At the qualitative level, both groups of teacher students reported difficulties in dealing with problems that arose within the work with Arduino.The teacher students with PPK were more likely to manage these problems within their group, while the teacher students without PPK tended to ask the lecturers for help.

Influence of the course on the self-assessment of technological content knowledge (RQ2)
After completing the learning arrangements on physical computing with Arduino, both groups of teacher students reported that their TCK had increased (teacher students with PPK: z = -2.533,p = .011,n = 8, r = .896,Mdnpre: 3.50, Mdnpost: 4.50; teacher students without PPK: z = -2.401,p = .017,n = 9, r = .800,Mdnpre: 3.25, Mdnpost: 4.00).Despite both groups reporting increases in their TCK, and there was no statistical difference between them in the pretest, the posttest results showed that the teacher students without PPK assessed their TCK to be lower than those with PPK (U = 11, Z = -2.337,p = .019,r = .567).
Triangulating these quantitative data with teacher students' statements in their reflection journals, this difference in confidence in using Arduino was also evident.Only three of the eight teacher students with PPK reported difficulties in using Arduino or completing the physical computing project in the third reflection journal entry, after completing the physical computing project.In contrast, in their third entry, eight of the teacher students without PPK reported minor difficulties and challenges encountered during the physical computing project.Nevertheless, overall, both groups of teacher students commented in their reflection journals on their learning progress and knowledge gains in the context of using Arduino.

Influence of the course on the self-assessment of technological pedagogical knowledge (RQ3)
Despite the few learning arrangements in this area, the pretest and posttest results showed that 12 of the 17 teacher students felt that their TPK improved after completing the learning arrangements on working with Arduino.Among the remaining teacher students, three reported lower TPK, and two reported no change in their TPK.According to the self-assessment data, the TPK of the teacher students with PPK increased significantly (z = -2.395,p = .017,n = 8, r = .847,Mdnpre: 3.50, Mdnpost: 4.38).Due to the statistical power of the sample, we are not in a position to make any statements about statistical effects in terms of the teacher students without PPK.
However, our analysis of the qualitative data obtained from the reflection journals and surveys showed that teacher students in both groups would have liked the course to have provided a stronger connection between the use of Arduino and educational practices.They expressed the wish for specific experience reports and examples of teaching with Arduino, best practice from practice, lesson planning, and independent implementation of projects in schools.

Influence of the course on the self-assessment of technological pedagogical content knowledge (RQ4)
The pretest and posttest results showed 11 teacher students felt that their TPACK had increased after working through the learning arrangements on working with Arduino; although few learning arrangements were provided for this and the teacher students remained in the learner role.Five teacher students reported a decline in their TPACK, and one reported no change in the posttest.The Wilcoxon signed-rank test results showed that TPACK increased only in teacher students with PPK (z = -2.047,p = .041,n = 8, r = .724,Mdnpre: 3.40, Mdnpost: 4.10).Due to the statistical power of the sample, we have not made any statements here about statistical effects in terms of the teacher students without PPK.However, the mixed-methods approach we chose allows us to make assumptions related to TPACK selfassessment as well: Regarding the question of the influence of the learning arrangements offered in the "Facts, Fakes and Algorithms" course on the teacher students' self-assessment of their TK, TCK, TPK, and TPACK, it can be added at this point that the teacher students were asked in the posttest to report the extent to which they believe that their self-assessment had changed compared to that in the pretest.Ten teacher students stated that after completing the learning arrangements on physical computing with Arduino, they were able to perform a more realistic self-assessment in relation to the aspects of TPACK.The reasons given for this were the practical work within the course, the broadening of the view of possible uses of digital media, and the methods used in the course.Twelve teacher students stated that they were able to assign a higher value in the self-assessment after completing the learning arrangements.The presentation of specific possibilities for implementing Arduino, the use of checklists for using Arduino with students at school, the exchange with fellow teacher students, and the practical work with Arduino were cited as reasons for higher self-assessment scores.
Two teacher students stated that they might have given a lower self-assessment score in the posttest compared to the pretest.The reason for this was that they did not yet know about the many possibilities of using digital media in the pretest.Four teacher students reported that they did not perceive a noticeable difference in their TPACK, as they only took on the learner role in the course.

Influence of the course on self-efficacy expectations in the context of Arduino (RQ5)
Among the 17 teacher students, 13 indicated that they had a low (< 10 out of 20 points) self-efficacy expectation for working with Arduino in the pretest.None of these teacher students had prior knowledge of digital data acquisition with Arduino.After working with Arduino in the course, almost all the teacher students reported an increase in their self-efficacy expectation regarding digital data acquisition.For teacher students with PPK, this was reflected in the Wilcoxon test results: z = -2.379,p = .017,n = 7, r = .899,Mdnpre: 2.25, and Mdnpost: 3.75.Due to the statistical power of the sample, we have made no statements about statistical effects in terms of the teacher students without PPK.
These results are also shown and partly explained by the qualitative data.Seven of the teacher students with PPK consistently indicated in the posttest that practical experience and newly acquired knowledge had led to them providing a more realistic self-assessment.However, seven of the nine teacher students without PPK stated in the posttest that they had only acquired fundamental knowledge of digital data acquisition with Arduino.They indicated that they still needed to practice and gain more practical experience to gain expert knowledge.

Discussion: Design conjectures, design criteria, and local teaching-learning theories
Looking at quantitative and qualitative data side by side, we deduced descriptively that the learning arrangements that focus on physical computing with Arduino in the "Facts, Fakes and Algorithms" course had an impact on the TK, TCK, TPK, and TPACK self-assessments and self-efficacy expectations of the teacher students.It is important to note that the results of this study can be used to make statements that apply to the researched group of teacher students within this specific context and not to formulate generalizations.Future studies should investigate whether our findings are replicable on a larger scale.
In this study, the teacher students without PPK rated their TK lower after working with Arduino compared to before.This could have been due to difficulties in learning how to use Arduino, especially the Arduino programming language.Scaffolds must be implemented that support teacher students without PPK.
Regardless of the teacher students' PPK, the learning offered within the course increased the teacher students' self-assessed TCK.In the context of using Arduino, the teacher students with PPK reported that their self-efficacy expectations increased; however, the statistical power of the sample limits further statements being made about this result.Nevertheless, the results indicate that an intervention in the areas of TK and TCK may also increase teacher students' self-assessed TPK and TPACK.Similar results were described by Link and Nepper in their 2021 paper [14].
Examining the teacher students' statements in their reflection journals in combination with their selfassessment results revealed that it may have been easier for the teacher students to acquire theoretical knowledge about Arduino and sensors than to use them for digital data acquisition to answer research questions and thus to apply the theoretical knowledge in a practical way.According to the data obtained from their reflection journals, the teacher students with PPK might have found it easier to implement digital data acquisition and problem solving within the context of Arduino than those without PPK.
From the results, we have derived the following prospective contribution to local teaching-learning theories: For teacher students without specific prior knowledge in the field of digital data acquisition with Arduino, it is easier to learn about the Arduino system than to use the Arduino system for digital data acquisition and working on mathematical and scientific problems.
Teacher students need help and support to undertake practical work with Arduino.Therefore, based on our results and the work described in [19] and [20], we have derived the following design conjecture for the redesign of the learning arrangements that focus on physical computing with Arduino and have extended our existing design conjectures (partially presented in [18]): Scaffolding can reduce the cognitive load of teacher students during investigative work with Arduino on a specific investigation question and prevent them from being overwhelmed.
To enable the translation of this design conjecture into learning arrangements and to address information provided by the teacher students in their reflection journals, we have defined two design criteria.The first design criterion is: When working on physical computing projects with Arduino, teacher students are supported by scaffolds, specifically via peer tutoring.In the re-designed course, peer tutoring is used as a macro scaffold.The formation of groups for project work already takes into account that heterogeneous groups of learners are formed.In addition, peer tutoring is specifically promoted by the lecturers in the course through the setting of tasks.It is also implemented in the course as a micro scaffold.The intentional heterogeneity of each group formed for the joint project work encourages teacher students with specific prior knowledge to provide situation-specific and needoriented word-based or process explanations or other assistance to their colleagues.
The second design criterion is: Frequently asked questions (FAQ) are integrated into the course as a macro scaffold and placed in a glossary within the Moodle learning arrangement.For the redesign of the learning arrangements, a list of FAQ is created by the course lecturers to provide teacher students with information that they can use to solve problems that arise while working with Arduino.Teacher students are encouraged to expand the FAQ independently.
The teacher students indicated in their survey responses and reflection journals that they would have liked to have had the opportunity to take on the teacher role during the course and gain insight into teaching with Arduino.To ensure that this request is fulfilled in the redesigned course, we established the following new design conjecture, based on findings from [21] and [22]: Teaching vignettes, excerpts from real teaching situations, provide a simulation of teaching reality and stimulate reflection on teaching-learning actions in teaching situations.
To implement this design conjecture in specific learning arrangements within the "Facts, Fakes and Algorithms" course, we have defined the following design criterion: To reflect on teaching-learning processes in the context of digital data acquisition with Arduino, teaching vignettes will be developed and used in the course according to [23].
An Arduino workshop will be implemented in schools to develop teaching vignettes.Text, image, and video vignettes will be developed using student artifacts, videos and audio recordings.

Outlook
Based on the results presented in Section 7, the design conjectures and design criteria presented in Section 8 led to the redesign of the individual learning arrangements and the implementation of the adaptations listed in Section 8.The next iteration of the course, with the new design, was implemented in the winter semester of 2022/23.

Table 1 .
Description of the sample of teacher students participating in our study.

Table 2 .
Descriptive statistics for the teacher students' assessments of their TK, TCK, TPK, TPACK, and self-efficacy expectation in the pretest and posttest.