Research on the results of secondary school physics education of Generation Z in the Czech Republic

There are no uniform exit tests in physics at the end of secondary school studies in the Czech Republic to give a picture of what students take away from their physics studies. Therefore, we decided to take advantage of the fact that in 1995 the Czech Republic participated in the TIMSS international survey of learning outcomes in mathematics and science, in which for the first and also the last time secondary school students were tested, including those who had taken more demanding mathematics and physics courses. The specialised physics 90-minute test was then taken by students in the final year of four-year gymnasiums in the Czech Republic. This particular physics test was administered to current gymnasium students. Gymnasium students represent the population of secondary school students who have physics instruction listed as a compulsory part of their education in the Framework Educational Programmes. Therefore, these respondents can provide very valuable information about what students take away from studying physics in primary and secondary school. The paper describes ongoing research, its methodology and how the data were collected.


Introduction
In 1995, the international study of comparative educational achievement known as TIMSS (Trends in International Mathematics and Science Study) was conducted for the first time.TIMSS is a flagship of IEA (International Association for the Evaluation of Educational Achievement).The aim of the TIMSS study is to assess student achievement in mathematics and science subjects.
More than 40 countries participated in the main part of the TIMSS 1995 study and three student populations were tested: nine-year-old students (3rd and 4th graders in most countries), thirteen-yearold students (7th and 8th grade) and final-year upper secondary school students.Students from the Czech Republic took part in all three parts of the study mentioned above.Population 2 consisting of 7th and 8th graders was mandatorily tested in each participating country.For description of target grades tested in each participating country defined for Population 1 and Population 2, please see Appendix B in [1], and for Population 3, please see Appendix B in [2].
Three types of tests were created for students in their final year of secondary school: mathematics and science test for students of all types of schools, a specialized mathematics test for students who are enrolled in special advanced mathematics programs, and a specialized physics test for students undertaking physics courses.
In the Czech Republic, the last two specialized tests mentioned above were completed by finalyear students of general upper secondary schools (also known as gymnasiums).This part of the study is called TIMSS Advanced.

Motivation to conduct research in 2023
Although since 1995 there has been a transition from a uniform curriculum to a strengthening of school autonomy in relation to the content of education in secondary schools in the Czech Republic, compulsory teaching of physics in gymnasiums has been preserved, albeit to a reduced extent.
Our literature searches [3] showed that although physics education appears to some extent in the Framework Educational Programmes for secondary vocational education, physics is explicitly mentioned as a compulsory educational field (we use the terminology of Framework Educational Programmes, hereinafter referred to as "FEP") in the FEP for gymnasiums.The FEPs for secondary vocational education define the minimum time allotment for physics only for a few study programs.
The final exam that studies at the gymnasium and in the study program technical lyceum are completed by is the Maturita exam.Its requirements are defined by the Section 77 of the Education Act [4].Students can choose physics as one of the optional subjects of the Maturita exam.
Data from a questionnaire survey [5] conducted at gymnasiums and secondary vocational schools with the study field technical lyceum showed that in practice approximately 10% of students choose the Maturita examination in physics.Moreover, its form is not unified in any way and varies to some extent between schools.
It follows that there are no uniform exit exams in physics that would provide us with information about what students take away from the study of physics in primary and secondary school.Therefore, we decided to take advantage of the fact that in 1995 a representative sample of Czech secondary students participated in the TIMSS Advanced physics test and to repeat this research in 2023.

Methodology of the study in 1995
In this section, we will summarize the main aspects of the TIMSS 1995 methodology.We will focus on the sample of students and the research tool -tests.

Sample construction
As it was already mentioned in the introduction, the Population 3 was defined at the international level as "all students in the final year of secondary school, with those having taken advanced mathematics courses and those having taken physics courses as two overlapping sub-populations."[2, p. 15] Education systems differ to some extent from country to country.In an effort to avoid testing and comparing an elite group of students from one country with a more general sample of students from another country, the so-called TIMSS Coverage Index (TCI) was developed.TCI estimates what percentage of the school-leaving age cohort is covered by the TIMSS final-year student sample.Participating countries could exclude from the sample schools that were from geographically very distant regions, schools with disadvantaged students who would be difficult to test, or pupils who did not understand the language of the testing.
In 1995, the Czech Republic was one of the countries with a high TCI of 77.6%.This means that more than ¾ of the students of the respective age cohort were represented by the sample.At the same time, the Czech Republic was included among the countries that met the TIMSS sampling requirements specified by TIMSS guidelines and satisfied sample participation rates.
The proportion of the entire school-leaving age cohort that participated in the specialized TIMSS physics test was indicated by the Physics TIMSS Coverage Index (PTCI).In the sample for the Czech Republic, 14.1% of students in the Population 3 (final year of secondary school) had physics lessons included in their timetable, which represented a PTCI of approximately 11.0% [6].
The basic sampling design was a two-stage process, with schools sampled with probability proportional to size (PPS) in the first stage, and a fixed number of students sampled in the second stage (usually the class of students per school).This guaranteed the same probability of selection for each respondent.If any of the schools refused to participate in the study, this originally sampled school was replaced by a replacement school.Chapter 3 of the publication [7] describes in more detail the sampling design of TIMSS 1995.

Categories of students tested in Population 3
Each student within the sampled school was assigned to one of four groups of students: OO: Students who studied neither advanced mathematics nor physics.OP: Students who studied physics but not advanced mathematics.MO: Students who studied demanding mathematics but not physics.MP: Students who studied both -advanced mathematics and physics.
Based on the category, the type of test booklet was assigned to the respondent as shown in table 1

A sample of students in the Czech Republic who participated in the specialized physics test TIMSS Advanced 1995
In physics, students in the final year of four-year gymnasiums were tested, which represented about 11% of students in their final year of secondary school studies.90 gymnasiums and 819 students participated in this part of study.

The tests -who created them and how
The tests were created based on curricular frameworks.The tests were developed by experts in natural sciences from around the world, taking into account the curriculum that students in the participating countries were studying.All test items were developed in English and then translated into the language of the given country so that the content or difficulty of the task did not change.

Types and number of tasks in physics tests
The aim of TIMSS was to test knowledge and applications of a wide range of physics concepts.Since the test time was limited to 90 minutes after compromises, TIMSS used the matrix sampling.A total of 65 physics test items were developed.The tasks were divided into three test booklets.
Basic physics constants and physics formulas were given at the beginning of each test.Students were allowed to use calculators during the test.
The test items included: 1) multiple-choice items -a student chooses one correct answer out of 4 -5 options, 2) free-response items with a short answer -a student writes his own brief answer, 3) free-response items with an extended answer -a student provides a more extensive written answer, often requiring reasoning.Free-response items were analysed by the so-called coders, who on the basis of uniform coding manuals (codebooks), assigned two-digit codes to the answers, e.g.20.The first digit indicated the correctness of the answer 2 -completely correct answer, 1 -partially correct answer, 7 -wrong answer, 9 -missing answer.The second digit indicated the type of response.
The matrix sampling made it possible to test a wide range of physics knowledge and skills in five content areas: mechanics, electricity and magnetism, heat, wave phenomena, and modern physics -particle physics, quantum physics, astrophysics and relativity.The specific distribution of tasks according to physics content area and type of answer is shown in table 2

Results from TIMSS 1995
The average physics achievement of Czech students was below the international average.Out of 16 participating countries, the Czech Republic ranked 14th.If we considered only 11 countries that satisfied the TIMSS guidelines for sample participation rates and student sampling, the average physics achievement of Czech students would be the lowest.
The males had significantly higher physics achievement than females in all but one of the participating countries.The difference in physics achievement between males and females in the Czech Republic was one of the highest among the countries.
The achievement in physics content areas for Czech students was statistically significantly below the international average for tasks in the areas of mechanics, electricity and magnetism, wave phenomena and modern physics.The average achievement for heat tasks was lower than the international average, but the difference was not statistically significant.[6]

Methodology of repeated research in 2023 and its course
Since our aim was to repeat the TIMSS Advanced physics test from 1995 among Czech gymnasium students, we preserved the same testing conditions in the 2023 testing.

Tests
We used the physics test booklets 2A, 2B and 2C that were used in TIMSS 1995.We preserved the paper form of testing.

The sample
We sent the request of participation in the testing to the same schools as in the original study.Of the 90 original gymnasiums, 72 participated in the repeated research, but one of them did not meet the requirements for testing and had to be excluded.Another 10 gymnasiums participated in the testing as replacement schools for those schools that refused to participate.Replacement schools were selected after consideration of geographical proximity and size comparability of the cities in which the original schools were located.The participation rate for both the original and replacement schools is 90%.
In order to preserve the original sample, we primarily wanted to test students in their final year of the four-year gymnasium study program.We accomplished this in 68 schools.
Five schools from the original sample from 1995 no longer provide the four-year gymnasium study program.In order to keep the original sample of schools, we tested final-year students of six-and eight-year gymnasium study program.In addition to these five schools, instead of final-year students of four-year gymnasium study program, due to organizational reasons, students of the final year of a six-year gymnasium study program were tested at three schools, and at another school, final-year students of an eight-year gymnasium study program.At two schools, a mixture of students in their final year of the four-year and eight-year gymnasium study program was tested.
Another three schools agreed to test only those students who attended an elective physics seminar that year.In these cases, there was a mixture of final-year students of the four-year and eight-year gymnasium study programs.We explain this attitude of the schools by the concern about the test results of the students, since physics is no longer a compulsory subject in the final year at most gymnasiums [5].A summary of the changes in composition of our sample described above is shown in table 3.

Testing period
The testing was carried out during February and March 2023.Only one school requested testing in January 2023.Since the form of testing was paper, we presented the tests at schools in person.There were 3 main test administrators (one tested 32 schools, the other two colleagues tested nine and eight schools).Units of schools were tested by another five of our colleagues and 16 schools were tested by external administrators (employees, friends and former graduates of our department).All test administrators were instructed and filled out the Test Progress Record.It contained information about the tested sample (number of students, specification of the study program), the beginning and end time of testing, or information about unexpected events during testing.
Altogether, we tested more than 1 800 final-year students of gymnasium study programs.

Coding free-response items
The evaluation of free-response items was conducted from April to June 2023 using the same coding manual as in 1995.Four coders participated.One of them has been involved in coding free-response items in TIMSS on a regular basis since 1995 and she also conducted the training of the remaining coders.Multiple coding by two coders was also carried out for 46 % of the test booklets.

Changes compared to the survey in 1995
The test booklets used in 2023 were, except for the details described below, identical to those used in 1995.As in TIMSS 1995, in the 2023 repeated research students were provided with physics constants and formulas at the beginning of test.
In 1995, information on the gender of respondents was collected by questionnaires.We did not distribute them in 2023.For this reason, we added questions to the beginning of the tests to determine the student's gender.Furthermore, we added questions to find out information about whether the student attends a physics seminar in the given year and whether he or she will undertake the Maturita exam in physics.
At the end of the test, the student was asked to answer two questions about the use of the calculator during the test.The tests were anonymous.
The start of the research was accompanied by the analysis of the FEP for gymnasium and the analysis of physics textbooks for gymnasiums.We investigated whether the educational content in the FEP and physics textbooks covers the curriculum necessary to successfully solve all tasks in the tests.The result of the analysis was that the students had the opportunity to acquire the knowledge necessary to solve the tasks in the tests based on the RVP and textbooks.The exception was one task (H18) from the area of particle physics.
The only content change in the tests (compared to TIMSS 1995) was made in test booklet 2C, from which we removed task H18, because the concept of television as a particle accelerator would be unfamiliar to students today.
In the paragraph of the previous section, we mentioned the inevitable changes in the sample caused by facts beyond our control -the reluctance of schools to participate in the survey, the transition of study programs provided by schools (change from four-year gymnasium study program to multi-year programs), replacement of initially agreed to-be tested students of four-year gymnasium study programs by students of multi-year study program based on the school's will.

Future plans
We expect several results from the research.We plan to compare the results of gymnasium students from 1995 with the results of the current testing.We will compare the average achievement of males and females, those students who take the Maturita exam in physics and those who do not, we will compare the average achievement in tasks from individual content areas, and last but not least, we will provide participating schools with feedback on the achievement of their students in testing.

Discussion and conclusion
The TIMSS physics tests in 1995 were based on what students in secondary school had the opportunity to learn.In order to make the results from 1995 and 2023 comparable, we administered the tests in 2023 to an identically constructed sample of students.The test booklets in terms of the content of the tested curriculum remained unchanged, except for one task, which we removed from the tests in 2023, as it would put current gymnasium students at a disadvantage.
The research will be the source of a large amount of data.Above all, it will provide us with information about what gymnasium students take away from the study of physics in primary and secondary school.

Table 1 .
. Student categories and respective types of test booklets.Table1shows that physics tasks were included in booklets 2A, 2B, 2C and 4. Purely physics booklets were 2A, 2B and 2C.Test booklet 4 consisted of literacy items, mathematics items and physics items.

Table 3 .
Differences in the length of the study program of the tested students in the sample.