Creation of an evaluation rubric for the assessment of written preparation for teaching physics

In the article, we present a proposal for an evaluation rubric for evaluating preparations for physics lessons. The evaluation rubric is intended for the evaluation of written preparations created by physics preservice teachers. We describe the method of creating an evaluation rubric and verifying its reliability. A well-conducted lesson by the teacher presupposes high-quality preparation for the lesson. Therefore, we consider it important that pre-service teachers would be able to develop a well-thought-out preparation in written form. We plan to improve its quality among students (pre-service teachers) by providing feedback based on the evaluation tool designed by us. We verified the reliability of the final evaluation rubric, therefore it can be considered a suitable tool for evaluating student preparations for physics lessons and providing feedback.


Introduction
Lesson planning is a purposeful process that is an important part of a teacher's preparation for a lesson.Every good teacher entering the classroom should have an idea and a plan for the upcoming lesson.The essence of its preparation is to help the implementation of the lesson as best as possible and most beneficial for the student and to fulfill the teaching goals.Jensen [1] states that preparation in written form is an extremely useful tool that can serve as a document reflecting the philosophy of teaching, the goals of working with students, or can serve as a brief guide.
Haynes [2] divides the teacher's activity into three parts: before the lesson, during the lesson and after the lesson.Before the lesson, the teacher deals with the preparation and planning of the teaching process, during the lesson it is about the actual implementation and execution of the educational activities, and after the lesson there should be an evaluation and assessment of the completed preparation.In our contribution, we focus on the preparation of the teacher before the lesson.According to Fink [3], we need to identify the following points to draw up a curriculum: 1. Identify important situational factors.

Determine:
• Educational goals.(What do I want the students to learn?) • Educational activities.(What will the teacher and students have to do for the students to achieve the learning goals?) • Feedback and evaluation.(How will the students and the teacher know if the set educational goals have been met?) 3. Make sure that these key components support and reinforce each other.

GIREP-2022
Journal of Physics: Conference Series 2750 (2024) 012040 IOP Publishing doi:10.1088/1742-6596/2750/1/012040 2 According to Gallego [4], the following components must generally be considered when planning lessons: • regulatory (control) framework, • school educational program and curriculum, • the context and conditions under which the lesson is to be conducted, • expertise, skills and special characteristics of the teacher and students, • the nature of the taught content of the lesson.
Furthermore, for specific preparation for a lesson, Gallego [4] states that it should be included: • context analysis (school education plan, curriculum, analysis of students, conditions, environment, materials, aids), • student characteristics, • subject analysis (intersubject relationships), • teaching goals, • lesson content (relevant and useful for students, organized, follow-up), • methods, • rating, • focus on diversity (variety of content, consideration of the needs of pupils with special needs)

Student preparations for the lesson
In this post, we will focus on physics lesson preparations created by physics preservice teachers.Preparation for physics lessons is part of the university training of physics preservice teachers.We consider it necessary learning how to make the best possible preparations and being able to formulate them appropriately, also in written form.By student preparation for a physics lesson, we mean written preparation that is brief but contains all the essential parts -categories that we have identified and listed below in the evaluation rubric (table 1).The created preparation should be clearly written so that according to it, another physics teacher can also teach the same lesson.
We require the created preparations to be sufficiently detailed in written form.If we want to get a holistic view of the student's planned lesson and then be able to provide feedback, then it is necessary that they also state the parts that an experienced teacher does not need in written form.We want students to learn to create quality preparations.We realize that a teacher with several years of experience does not need to create written preparations for lessons in such a detailed form as we expect from physics preservice teachers.For this reason, we are presenting an evaluation rubric aimed at student preparations for physics lessons.

Evaluation rubric
Based on the analysis of professional literature, we concluded that the most appropriate tool for evaluating student preparations for a physics class is an evaluation rubric.
According to Davidson [5], the evaluation rubric is a table that describes individual levels of performance based on a certain criterion of interest.Furthermore, Jane Davidson [5] states th at created rubrics should encourage the use of sound evaluative judgment and should use common language and commonly used concepts, which can increase agreement between evaluators.

The history of the creation of evaluation rubrics
A brief history of the creation of evaluation rubrics will introduce us to the meaning and reason for their creation.
Turley and Gallagher [6] present a brief cross-section of the history of the development of evaluation rubrics.According to these authors, the first extended rating scale was a scale for evaluating student written compositions (styles/articles), known as the Hillegas scale, after a professor working at Columbia University (Columbia University's Teachers College).Millo B. Hillegas was a student of Edward Lee Thorndike, who outlined the issue of measuring, unifying, and evaluating qualitative results.
We could identify the origins of the evaluation rubric in the text of Thornidike [7], where he expresses the need to establish a linear scale of points, which refers to only one thing, but differs only in quantity.The differences between the individual points of the scale must be precisely and unambiguously defined.As in physical scales and scales, it is necessary to define zero.
Hillegas [8] expresses a great need to define educational requirements and set standards.Student performance assessed by conventional assessment (grade or percentage) may or may not represent the same level in different education systems.In 1912, Noyes [9] published the need for uniform assessment of students' works, whose evaluation.Is an extremely complex process [9], as the teacher must consider several facts.Differences in the evaluation are a result of insufficiently defined evaluation standards, not a disagreement on the most important things.Although the extreme complexity of the facts to be observed and estimated in forming any judgment, together with the varying emphasis placed by individuals on vocabulary, sentences, paragraphs, ideas, etc., makes uniformity of judgment difficult, it is not impossible.According to Noyes [9], it is necessary to create standards according to which teachers would be able to equally evaluate students works.For a better idea, he gives an example: If we say about the student's work that it is good, we are just as inaccurate as if we say that it was hot yesterday.If we were to indicate the temperature in °Celsius or °Fahrenheit, this is a standard that is understandable in different parts of the world.However, comparing the two teacher ratings is very different.
Although Noyes [9] focuses on the evaluation of students' compositions (student styles), he expresses ideas that are applicable to almost all areas related to the evaluation of students works.Current assessment methods are too dependent on personal opinion, which varies from individual to individual.What is required is a clear, concrete standard of measurement that will be unambiguous to all people in all places and independent of individual opinion.

Methodology
In our work the process of creating an evaluation rubric for evaluating student preparations for physics lessons consisted of several steps.In the first phase, we concentrated on the study of literature focused on lesson planning and the creation of evaluation rubrics.We were inspired by several authors dealing with evaluation rubrics (e.g.[10,11,12]) implemented in the educational process.When creating the content and key elements of the evaluation rubric, i.e., the parts that will be evaluated in the presented preparation, we were based on the works of various authors (e.g.[1,2,3]).Subsequently, we created the first draft of the evaluation rubric, which was modified several times.When editing, we took into account the comments of experts in the field and the study of literature.In the final evaluation rubric, we established the following categories: • compliance with the state educational program, • time schedule, • year/class, • goals, • previous knowledge, • teaching forms, • teaching methods, • equipment, • formulated questions, tasks, activities for pupils • empirical methods of cognition -choosing an empirical method, -procedure of the empirical method, -conclusion of the empirical method, • conclusion of the lesson, • physical correctness, • general impression.
It was necessary to verify the reliability of the evaluation rubric.Reliability represents the accuracy and reliability of a research instrument.We chose the method of assessment of the evaluation rubric by several external evaluators (inter-rater reliability) [13].The evaluators, experts in the field, assessed the preparations for the lesson compiled by physics preservice teachers with the help of a rubric compiled by us.If the evaluators achieve a high degree of agreement during the evaluation, we can declare the preparation evaluation rubrics created by us as reliable.We modified the evaluat ion rubric several times, until the evaluators achieved a high degree of agreement during the evaluation.

Results
The main result of the work is the creation of an evaluation rubric for evaluating student preparations for a physics lesson.We chose the criteria that we assessed in the preparations, namely: compliance with the state educational program, timetable, year, teaching objectives, previous knowledge of pupils, teaching forms and methods, equipment, questions and tasks, experimental activity, conclusion, physical correctness, general impression.For each criterion, we developed a scoring scale defining individual points.A maximum of 3 points could be obtained in each category, except for the year/class category, where a maximum of 1 point can be obtained.
Below we present the final evaluation rubric (table 1).The rated categories are listed in the first column.In the rubric, all evaluation levels are defined for each category.For any activity, it is not clear in which teaching form it should be implemented.

Teaching forms and methods
Teaching methods are appropriately chosen, they can effectively achieve appropriately set teaching goals.
In most activities, the teaching methods are appropriately chosen, and it is possible to achieve the set teaching goals with them.
In most activities, the teaching methods are not appropriately chosen in order to achieve the set teaching goals.
In all activities, the teaching methods are inappropriately chosen due to the effectiveness of achieving the set goals.Or it is not clear from the preparation what teaching methods are to be used.

Equipment
At the beginning of the preparation, all the equipment needed for the lesson is listed, which must be prepared in advance.The tools are defined precisely enough.
The necessary equipment is listed in the preparation, but it is not listed at the beginning of the preparation or is not sufficiently specified.
Not all equipment needed for the lesson is listed.
The necessary equipment for the lesson is not listed.

Questions and tasks
In

The procedure of the empirical method
In the preparation, a brief but clear implementation procedure is given.
The preparation contains a clear implementation procedure, but it is written unnecessarily extensively.
The implementation procedure is indicated in the preparation, but it is not completely clear.It is written with insufficient precision.
The preparation contains an ambiguous implementation procedure.Or the procedure is not indicated at all.

Conclusion of the empirical method
Additional tasks or questions related to the empirical method are listed, which lead to the determination of the acquired knowledge.
There are several tasks or questions that partially follow the empirical method, which lead to a partial confirmation of the acquired knowledge.
A minimum of additional tasks or questions are listed.The tasks and questions are only marginally related to the empirical method and do not lead to almost any confirmation of the acquired knowledge.
There are no additional tasks or questions that would follow the empirical method and lead to the determination of the acquired knowledge.

Conclusion of the lesson
At the end of the preparation, there is a summary pointing out the most important concepts, activities, or findings from the given lesson.It can be a speech by the teacher, questions asked by At the end of the preparation, there is a summary pointing out almost all the most important concepts, activities, or findings from the given lesson.

Determination of interrater agreement
After creating the evaluation rubric, we verified its reliability.Three experts in the given topic and the author of the article evaluated three randomly selected student preparations for the physics lesson with the evaluation rubric.
For the quantitative determination of the agreement between the evaluators, we used the calculation of the standard deviation.We have determined that a standard deviation within 5% of the total number of points is considered a high degree of agreement.
We used the following relationship to calculate the standard deviation s = √(∑(( −   ) 2 /), where individual designations mean: sstandard deviation, the total number of points awarded by the evaluator to a specific student preparation,  the total number of points awarded by the author to a specific student preparation, number of evaluated preparations.
In the following table 2 we present the number of points awarded by individual evaluators in specific categories of the evaluation rubric.In the first column, we list the maximum number of points that could be awarded in individual categories.Next, in table 3, we present the calculated standard deviation for the evaluators.We have determined that a standard deviation within 5% of the total number of points is considered a high degree of agreement.5% of the total number of 43 points represents 2.15 points.For two evaluators, the standard deviation is less than 2.15.At evaluator no. 1 is a standard deviation of 2.16, it means, that is 0.01 greater than the limit set by us.Since this is a very small deviation, we can consider it negligible.Based on the results of the reliability verification of the evaluation rubric designed by us, we created a suitable tool for evaluating students' preparation for physics lessons.

Use in practice
Based on these findings, we can provide feedback to students.We expect that, thanks to the feedback, the students will improve their preparations, which will lead to a better preparation for their future professionphysics teacher.

Conclusion
In the article, we presented the process of creating an evaluation rubric intended for the evaluation of student preparations for physics lessons.When creating it, we based it on the study of professional literature and the recommendations of experts in the given field.We edited the draft of the evaluation rubric until all the comments and recommendations of experts in the field were incorporated.We provided the final evaluation rubric to 3 evaluators.Each of the evaluators plus the author of the article evaluated three randomly selected student preparations for physics lessons.Based on the point score given by the evaluators -experts, we verified the reliability of the evaluation rubric, using the inter-rater reliability method.Based on the obtained results, we declared the final evaluation rubric reliable and therefore a suitable tool for evaluating student preparation for teaching physics.
Assessment and evaluation of student preparation informs us about the ability of physics preservice teachers to plan physics lessons.Based on these findings, we can provide them with feedback.Thanks to the feedback, the students will improve their preparations, which will lead to better preparation for their future profession -physics teacher.

Table 1 .
Evaluation rubric for evaluating preparations for a physics lesson .

Table 2 .
Scores awarded by evaluators to 3 randomly selected preparations.

Table 3 .
Standard deviation of evaluators.