Eyetracking identification of cognitive decision-making modes in STEM education - a case study

We present a description and oculographic characterisation of task solving methods on the basis of eyetracking research carried out on a group of 60 university students, of various specialisations, within the age bracket of 20-21. We also provide preliminary answers to two questions: Does eyetracking identify cognitive decision-making modes? If so, which oculographic parameters are the most useful for this purpose? In this paper, we present a selected fragment of the results obtained from a series of tasks with a difficulty level not exceeding that of secondary school, based on a task concerning the interpretation of changes in the distance from the ground at a selected point of a carousel during its movement. In addition, the study concerns university students’ difficulties in interpreting graphs.


Introduction
According to studies [1][2][3], the main communication channel of Generation Z seems to be graphics, multimedia, and social media.This is where Generation Z tends to react, share opinions, and comment.The need to be online also relates to their preferred learning styles.Generation Z generally works better in simple structures -the quicker and more intuitive something is, the better.Young people also expect facilitation and simple mechanisms in the classroom.In addition, Generation Z can sometimes avoid situations that would require searching for complex and long-term solutions, which does not contribute to the effectiveness of learning and understanding STEM.It is worth noting that Generation Z often struggle to maintain concentration and attention, which is often subject to distraction and susceptible to external stimuli.The challenge in teaching STEM is therefore to hold their attention for a longer period of time and to interest them enough to be willing to perceive the content being taught to them.
The ability to mathematically model physical phenomena in the form of graphs and to interpret them is essential to the teaching of both physics and mathematics.These issues are the subject of research and analysis in both physics and mathematics education.They are also considered to be key skills in primary and secondary school curricula.Therefore, the difficulties of high school students and university students in understanding and interpreting graphs and using them to model physical phenomena have been widely studied (e.g.[4][5][6][7][8][9][10]).The research, a fragment of which is presented in this paper, is part of a strand of empirical research that diagnoses and describes difficulties and their causes in understanding and solving problems in this area.).The application of the cognitive continuum theory in education may involve identifying the characteristics of the tasks performed by the research participants which require the use of analysis or intuition, as well as situations in which the use of pure forms of basic cognition is impossible.In these situations, it is necessary to apply a quasi-rational model, i.e. an appropriate combination of rational analysis and intuition with common knowledge.The effectiveness of using quasi-rationality has been confirmed by empirical research, for example in the field of management theory (e.g.[16]).
In mathematics and science education literature, intuitive knowledge is described as immediate knowledge, i.e. a form of knowledge that seems obvious to a person (e.g.[17]).Fischbein also writes that it is surprising h under certain circumstances a subject is ready to completely ignore important information because of the need for immediate action.Because of the automaticity of this decision, the subject not only does not use this information, but also ignores its importance to the validity of the judgement.
In our study, we aim to initially answer two questions: Does eyetracking identify cognitive decision-making modes?If so, which oculographic parameters are most useful for this purpose?

3.1Participants
The participants of this study comprised of 60 undergraduate university students of varying fields, mainly ages 20-21, 33 females, and 27 males.This group included people who described themselves either as humanist or science-minded, as well as those who identified with both.

Materials
The questions asked during the study focused mainly on the ability to model and analyse motion at a level not beyond basic secondary school physics and mathematics.
The whole study consisted of 16 tasks.Out of these tasks, 13 had a practical context related to movements observed in everyday life, while the last three tasks had a purely mathematical context, also related to movement analysis.The study is part of a larger ongoing study, the results of which are currently being analysed.There was no time limit for responses, although the participants -university students, in keeping with the characteristics of Generation Z, of which they were representatives -tended to respond quickly.The participants' extrinsic motivation to complete the tasks correctly was low, as they were not assessed on this basis.Therefore, the overall results were unsatisfactory.
In this paper, we focus only on one question, inspired by the analysis of everyday life.The question was: Imagine you are sitting in a chair on a Ferris wheel.Which of the graphs best describes the changes in distance of the chair from the ground over time?A photograph of the Ferris wheel and 8 proposed graphs representing the distance of the chair from the ground in time are shown in figure 2.

Measurement procedure
Eye movements were recorded using the Tobii Pro X3-120 at a sample rate of 120 Hz.The eye-tracking system was calibrated for each participant beforehand using a 9-point calibration algorithm.The questions were presented on a 24" monitor at a distance of 70 cm from the participant's eyes.After choosing the answer, the participant was presented with next question.
After the eyetracking part of the study, the students answered an open questionnaire in which they assessed their level of knowledge and skills in high school mathematics and physics on a scale from 1 to 10 (1-I have great difficulties; 10 -I am doing very well), declared themselves to be humanists and/or science-minded, and were able to provide feedback on each task.

Results and discussion
Tobii Studio software was used for eye movement data analysis.We first examined the duration of the participants' work on the task.The average time spent on the task by the whole group of participants was 38.51 [s].The working time of all 60 participants are presented in figure 3.Then, we identified the persons with the longest and shortest time of solving the task as well as those who selected the correct and incorrect answers respectively.Among the incorrect answers, we distinguished two types -impossible answers (no. 1 and 2) and those with mistakes of differing nature: (No 4-8).This group required further qualitative analysis.

Figure 3. Distribution of dwell times for all study participants
We selected participants P02, P05, P08, P43, P44, and P48, and analysed their oculographic parameters: total dwell time of the gaze and the number of fixations for the areas of the whole slide, the wording of the task, and the photograph.Figure 4 shows the number of fixations for the selected subjects for the aforementioned areas.We then analysed the scan paths recorded during their work on this task.

Study participants P08 and P43
The selected students achieved an almost identical number of fixations while solving the task (59 and 58), and their total time spent on the task was the shortest among all study participants (17.4[s] and 12.9[s] respectively).It is worth noting the way in which P08 analysed the text, which revealed 3 fixations on the wording of the task, including a fixation on the location of the text describing the relationship -the phrase 'chair from the ground' (figure 5, left).The acquisition of this information and the analysis of the selected graphs, combined with the analysis of the photo, enabled the respondent to provide the correct answer in a very efficient and expert manner.The same cannot be said for respondent P43 (figure 5, right), who completed the task just as quickly but spent considerably more time reading the task text.The large number of fixations on answers 1 and 2, ending on answer 1, show that we are dealing with an intuitive indication of the answer, which reveals the student's misconception of confusing the graph of the chair's distance from the ground over time with the trajectory of the Ferris wheel's movement.

Study participants P02 and P05
The working time on the task for P05 was 60.1 [s]; and for P02, 73 [s], almost double the average time for the whole group, while the number of fixations for P05 was 207, and for P02 was 252.The two scan paths represent significantly different ways of analysing the task.P05, who gave the correct answer, concentrated on analysing both the content of the task and all the graphs, without analysing the photographs (figure 6, left).This demonstrates the lack of need to use the illustration to imagine changes in the chair's distance from the ground.All this demonstrates the activation of analytical thinking.Student P02 (figure 6, right), on the other hand, despite having worked on the task for a relatively long time and achieving a large number of fixations, which indicates a thorough analysis of the text of the task, unfortunately concentrates her attention mainly on the photograph of the Ferris wheel and the search for a representation of this movement in diagrams 1 and 2. The final answer being 1 confirms that it was decided on the basis of intuition, with the same misconception as in the case of participant P43's solution described above.

Study participants P44 and P48
For students P44 and P48, with the longest analysis time, 44 and 54 fixations respectively were recorded on the photo alone.Despite giving incorrect answers (P44 -answer no. 5, P48 -answer no.6), the students did not, as in previous cases, relate the trajectory to the shape of the graph.This shows that they overcame the intuitive approach and integrated an analysis based on their knowledge.It is important to note the very intensive analysis of the whole task, including the illustrations, by both students (figure 7).

Conclusions
After analysing the examples presented, we can conclude that the time spent working on the task does not determine the cognitive mode adopted by the respondent to solve the task.
For example, the work of P02 manifests the "Pure Intuition Mode" or "Mostly Intuition & Analysis", despite a relatively long working time (73 [s]), almost twice as long as the average (38.51 [s]).For a student who does not have scientific secondary school level knowledge of the task at hand, the only way resulting from his/her abilities will be the application of colloquial knowledge, which is often incompatible with scientific knowledge.Therefore, despite the attempt to overcome the intuitive approach to the task, the solution is still limited to the application of colloquial knowledge, making it a mainly intuitive approach.In contrast, the example of P08's work shows that his experience in solving STEM tasks meant that he found the problem to be of low complexity, even trivial, and the answer was quick and expert.This student solved all the problems in the set included in our study in a similar way, demonstrating the ability to read quickly and efficiently, and showing his expertise, which was also confirmed by the questionnaires.On the other hand, it should be emphasised that we could not conclude from the oculographic analysis of the work on one task alone whether this student had expert knowledge or whether he answered correctly only by chance.A quick reading of the content of the task in the oculographic analysis could be misinterpreted, so it is important to use a wider range of analyses or complementary methods in this context.
The complexity of the analysis and the wide range of durations and amounts of fixations makes it possible to say with certainty which cognitive mode the respondent has used in only a few cases.For example, student P43 shows the "Pure Intuition Mode", which is evidenced by the short amount of time spent on the task, a low number of fixations, and the selection of the incorrect answer suggesting a path of movement.
In many situations, the scan path record provides relevant information, e.g.regarding the use of colloquial knowledge by the respondents, which helps to indicate the cognitive mode they adopt.For example, P05 manifests a mixed approach from the quasi-rationality category, as one can see the influence of intuition on the reasoning (analysis of answers 1 and 2) where the correct answer was finally given.Similarly, thanks to the analysis of the scan path, it can be concluded that it is either the "Pure Intuition Mode" or "Mostly Intuition & Analysis" -despite the relatively long working time mentioned above -due to the attention paid to the photo (63 fixations) combined with the analysis of answers 1 and 2 (visible multiple, alternating fixations in these areas).
On the other hand, it is not possible to say with certainty which mode P08 used in this task.His high level of experience and previous training favour an analytical approach, while his familiarity with the task, which would favour an intuitive system, is not excluded.To resolve his cognitive mode in this and many other cases, eye-tracking alone is not sufficient, as it only allows to indicate the cognitive decision-making mode with a certain, often low probability.An additional method, such as an interview, is then required.
Eyetracking is therefore not a fully sufficient method of diagnosis, even though it sometimes seems enough.It certainly provides relevant data from the point of view of the cognitive decisionmaking mode and STEM didactics.Furthermore, it should be emphasised that the essential information provided by eyetracking regarding task solving cannot be obtained by other methods.

Figure 1 .
Figure 1.Modes of cognition along the cognitive continuum according to Dhami & Thomson([15], p. 320).The application of the cognitive continuum theory in education may involve identifying the characteristics of the tasks performed by the research participants which require the use of analysis or intuition, as well as situations in which the use of pure forms of basic cognition is impossible.In these situations, it is necessary to apply a quasi-rational model, i.e. an appropriate combination of rational analysis and intuition with common knowledge.The effectiveness of using quasi-rationality has been confirmed by empirical research, for example in the field of management theory (e.g.[16]).In mathematics and science education literature, intuitive knowledge is described as immediate knowledge, i.e. a form of knowledge that seems obvious to a person (e.g.[17]).Fischbein also writes that it is surprising h under certain circumstances a subject is ready to completely ignore important information because of the need for immediate action.Because of the automaticity of this decision, the subject not only does not use this information, but also ignores its importance to the validity of the judgement.In our study, we aim to initially answer two questions: Does eyetracking identify cognitive decision-making modes?If so, which oculographic parameters are most useful for this purpose?

Figure 4 .
Figure 4. Number of fixations for six selected participants in areas of entire task, wording of task, and Ferris wheel photo.