Profile of student misconceptions: limited scale trial of diagnostic assessment development based on science literacy

The purpose of this study was to analyze the profile of student misconceptions using a scientific literacy diagnostic assessment. This research is a limited-scale trial stage in developing scientific literacy diagnostic assessments. The diagnostic is a four-level diagnostic test that includes 40 questions consisting of system materials referring to physics topics and biology topics that contain science literacy indicators. This diagnostic question was given to students of the elementary school teacher education at class 3B for the 2019/2020 academic year, consisting of 34 students. After that, the results of the students’ answers were analyzed and interpreted according to the categories of conceptual understanding, misconceptions, and not understanding the concept. Based on data analysis, 9.49% of students understood the concept, 52.73% of students experienced misconceptions, and 37.73% of students did not understand the concept.


Introduction
Problem-solving is an absolute prerequisite for this era. In order to have the skill to solve problems properly, we must have 21 st -century skills and good scientific literacy because scientific literacy and 21 st -century skills are basic concepts that must be mastered to solve problems [1]. 21 st -century skills are skills that involve critical, metacognitive thinking skills, being able to integrate and understand real situations, and authentic teaching skills by collaborating technology in learning [2]. At the same time, scientific literacy is the ability to understand nature, use scientific knowledge and thought processes to solve problems [3]. Scientific literacy is needed to complement teacher teaching methods so that learning is of higher quality.
However, the scientific literacy skills of Indonesian students are not good based on the PISA test. Only 40% of students can reach level 2, namely explaining scientific phenomena and using this knowledge to identify problems and conclude based on data [4]. Likewise, students from the primary educational teacher department, Universitas Muria Kudus, obtained results from their scientific literacy skills that were in the nominal and functional categories [5]. Students could identify and provide problem-solving solutions, but they could not explain in more detail. Likewise, the results showed that the students' computational thinkings were at the algorithm level [6]. They could solve problems, but the solution was not systematic [7]. Computational thinking skill is a modernized literacy skill; if students have good literacy skills, the students' computational thinking skills will also be good, and vice versa [8], [6]. These results indicate that students could provide solutions but had not been able to explain each step of the solution with partial thinking. If they were faced with a case, for example, making a 2 science media project for students' concept discovery, they could do it well and understood the concept [9], but the computational thinking skill results did not show a more systematic solution pattern. These results indicated students experienced misconceptions because they could identify problems and provide solutions. Unfortunately, the solutions were not systematic. These results were obtained with the initial indication of student misconceptions through open questionnaires based on the feedback about the computational thinking given. Based on the results of the cluster analysis, it showed that 33.38% of students experience misconceptions [10].
The misconception is a student misunderstanding in explaining the concept of science [11]. Students might understand the basic concept, but sometimes they interpreted it incorrectly. Even students might deviate from the mastered theory. If this misconception is not resolved, it will result in low student learning outcomes [12]. Therefore, a diagnostic test is needed to reveal student misconceptions. Diagnostic tests aim to measure students' understanding of a concept, diagnose student difficulties, and identify sources of error [13]. Because it can identify students, analyze difficulties, sources of error, and student strengths. This diagnostic test can improve teacher learning. There are various kinds of diagnostic tests that have been developed, including a four-level diagnostic test. The four-level diagnostic test consists of questions, choice of answers, level of confidence in choosing answers, choice of reasons, and level of confidence in choosing reasons [14]. The four-level diagnostic test has the advantage of being able to measure misconceptions in detail because each answer choice or reason has a level of confidence, so that it prevents students from speculating. Validity and reskill are required as part of the flow of test item development to create valid and feasible test results. Therefore, this research is important.

Methods
This research is a limited-scale trial phase that is part of research and development [15] in developing scientific, literary diagnostic assessments. The developed diagnostic questions consist of a four-level diagnostic test based on scientific literacy. The assessment consisted of questions, answer choices, answer level beliefs, choice of reasons, and confidence levels in choosing reasons. The choices of answers and reasons were not determined by the lecturer but the answers of students that often appeared. So, these choices consisted of one correct choice and three distractors. The questions consisted of 40 questions about the topic of Motion Systems since it had low learning outcomes [16]. After obtaining a valid four-level diagnostic form, the question was tried out limitedly for thirty-four 3B students who were taken by random sampling. The expert judgment was used to validate the results of the limited scale trial, the items, distinguishing power, and difficulty level of the questions, understand the concept, misconception, and do not understand.
Reliskill analysis was applied by using Cronbach's alpha [17]. If students answered correctly with the correct reason, they would have a score of 3. If the answer was correct, but the reason was wrong, the score would be 2. When the students answered incorrectly but the reason was correct, the score would be 1. The distinguishing power analysis and the difficulty level of the questions were done by describing the types of questions. At the same time, the misconception categories were carried out qualitatively by interpreting the data based on the truth in answering and selecting reasons and the choice of the answer level in answering and choosing reasons. The four-level of confidence consists of six choices, namely (1) mere guessing, (2) average, (3) uncertain, (4) confident, (5) super confident, and (6) extremely confident. The scale (1) to (3) shows the low level of confidence was coded 0, and (4) -(6), showing a high level of confidence, was coded 1 [18]. The results of the answers, reasons, and the level of confidence in answering and reasoning were then qualitatively categorized to get the categories of understanding the concept, misconceptions, and not understand the concept. If the answer were correct, the level of confidence in answering would be high. The reason was right, and the level of confidence in choosing the reason would be high. Then the student would be in the category of understanding the concept. If the answer were correct, the level of confidence in answering would be right or high. When the incorrect reasons were chosen and the level of confidence in choosing the reason was high, then it would be the category of misconception [14], [19]. The four-tier misconception categories can be seen in Table 1.

Result of tests item analysis
This study aims to develop a four-tier diagnostic test that consists of a preliminary study, development, and validation steps. This research is a limited-scale trial phase at the development stage [15]. After obtaining the form of a four-tier diagnostic test, this instrument is given to the experts for analysis and input. The experts consisted of experts on physics concepts, experts on biological concepts, assessment experts, and science learning experts. The validation results of 7 validators obtained an average score of 94.83 or are in a very valid category so that the questions could be used for research. Furthermore, the questions were tested on a limited scale trial to obtain the characteristics of the tests based on the reliskill analysis, the distinguishing power, and the difficulty level of the questions. The results of Cronbach's alpha analysis showed that the item reliskill was 0.899, with t table 0.193. These results indicate that t count > t table so it could be concluded that the questions were reliable. Reliskill results were between 0.70-0.90, indicating that the questions had high reliskill [17].
Furthermore, the tests were tested for discriminatory power to find out whether these questions could distinguish students who had misconceptions or not. Based on the data analysis, it was found that there were three questions to be removed, 0 questions to be revised, five questions to be accepted, and 32 questions with excellent acceptance criteria. These results indicated that the questions were suitable for research. The unfit item test with the criteria was not immediately removed, but they were reviewed again for improvement. After measuring the distinguishing power of the questions, then the level of difficulty of the questions was measured by comparing the number of answers with the number of students and the maximum score for each item of the question. The results of data analysis showed that 20 questions were categorized as difficult, 17 questions were categorized as a medium, and three questions were categorized as easy. These results indicated that the criteria were per the thinking level of students who have high-level thinking. Thus, the questions were feasible and could be used for research.

Students Misconception Profile
Based on the analysis of the items and the results of the validation, the tests could be used for research so that these tests were feasible to measure student misconceptions. The developed diagnostic test was about the motion system on the concept of physics and the concept of biology. The concept of physics consisted of uniform rectilinear motion, uniformly accelerated rectilinear motion, projectile motion, circular motion, force, and Newton's law. Meanwhile, biological material consisted of lower living things, plants, animals, and humans. The 40 four-tier questions were then categorized according to Table  1. The results showed that 9.49% of students understood concepts, 52.78% of students experienced misconceptions, and 37.73% of students did not understand concepts. The high category of misconceptions and not understand the concept were because of various scientific backgrounds of the student. Most of them came from social science and vocational school backgrounds so that their natural science skills were still limited [5]. Students memorized more formulas than understood problemsolving concepts and strategies [20]. Besides, in understanding a concept, students relied on teacher explanations, books, and other learning resources [21]. Therefore, even though students were included in the adult learning category but instruction and explanation from the lecturers were still needed. The lecturer used the diagnostic results to analyze the strengths and weaknesses of each student's sub-topic so that the lecturer can improve their learning.
Furthermore, the diagnostic questions were analyzed based on the physical and biological concepts. In physics material, the topic of the motion system being tested was uniform rectilinear motion (RM), uniformly accelerated rectilinear motion (ARM), circular motion (CM), projectile motion (PM), force (F), and Newton's law (HN). Based on the analysis of Table 1 on physics material, the students' misconception profiles were obtained, as shown in Figure 1.   Figure 1 shows that the highest misconception is experienced in Newton's Law material, while the lowest misconception is in ARM and RM material. For the understanding concept category, Newton's law is the material with the lowest percentage of understanding concept, namely 2.94%, while ARM is the material that has the highest percentage level of understanding concept, namely 8.82%. In the RM material, the student's misconception was 45.59%, and in the ARM material, the student's misconception was 41.18%. The results tend to be good compared to the results for other materials. In this material, students must be able to distinguish position, distance, velocity, and acceleration. However, when it comes to the quantity of a vector, students tend to ignore the direction of the vector and the cause of the object to move because of the applied force. This agrees that the motion of an object is not determined by position, velocity, and acceleration but is determined by force [22].
Furthermore, in the circular motion material, the misconception experienced was 55.8%. When studying rotational motion, they did not take into account the angular velocity of the motion. They thought that the cause of an object to rotate was due to the linear motion it exerted on it. In the matter of rotational motion, students assumed that circular motion was caused by translational motion on the object so that the force exerted on an object only affects its translational motion [23].
In the material of projectile motion, 5.88% of students understood the concept, 60.29% of students had misconceptions, and 33.82% of students had not understand. The misconception experienced by students in this material was that they could not analyze the motion that worked when the parabolic moving object. There was a component of motion in the direction of the X-axis and a component of motion in the direction of the Y-axis where the motion of this parabola was a combination of RM and ARM. Students assumed that when an object reached its highest point, the object lost energy, no force affected it, and the object did not have velocity, so that the object moved slowly. Whereas when the object reaches its highest point, the object had a velocity in the direction of the X-axis, and there was gravitational acceleration that caused the object to move downward. One way for the students to explain this was -they had to explain through the motion diagram components. Explanation of the material with another force diagram representation could reduce student misconceptions [24].
In the force material, students experienced a misconception of 55.15%. In this concept, students were already able to explain the physical meaning of force. However, if it involved various forces that had angles and motion related to acceleration, students experienced confusion. Students were confused about the effect of constant force, which resulted in constant acceleration through increasing velocity [20]. Also, students had difficulty working on force problems in a graph. Some students had difficulty understanding graphs on kinematic concepts [25].
Newton's law material is complex in the motion system of the physics concept because it already involves vectors, the concept of velocity, acceleration, force, angle in a complex system of objects. Students had been able to identify the sound of the first Newton's law, second Newton's law, and third Newton's law but had not been able to apply them to object systems with complex vector and force analysis. Thus, the results of student misconceptions on this material were very large than other materials. The difficulty of students with this material was because the concept of Newton's law required students to think about partial concepts of force, motion, and acceleration, constant motion, tension, and the gravitational force on the same object [20]. Besides, students found it difficult to explain the effect of force on objects. When an object is at rest, there is no force acting on the object, and the greater the mass of the object will provide a greater force [26].
In addition to the material concept of physics, it was the concept of biology. Furthermore, based on data analysis in Table 1, the misconceptions profile in biological material are shown in Figure 2.  Figure 2 shows that students experienced the highest misconception on the topic of animal movement systems, and the lowest profile misconception was experienced on the topic of human movement systems. The sub-topic of the human movement system could be seen and observed directly, while the material in lower living things was abstract and difficult to observe. Therefore, the material for animal movement systems had the lowest misconception and had the highest percentage of understanding the concept. On the contrary, when compared to the material for motion systems in lower living things, many students experienced misconceptions related to the way prokaryotic cells move, for example, the bacteria and the location of the flagella used for movement [27]. Some students also mentioned that bacteria were animals, so that their ways of moving and their properties were also like animals [28]. It is in line with Anwar's research which found that the concept of cell organelles was one of the materials causing the highest misconceptions because it was difficult to observe directly [29].
On the topic of animal movement systems, students could explain the types of bones, types of vertebrates, and how they moved. However, they were confused about some invertebrates, such as snakes and worms. They considered that worms were invertebrates while snakes were vertebrates. Some students answered like that because they stated that when they were still in elementary school, the concept was like that. Some of the misconceptions experienced by students in animal movement systems are the classification of vertebrates and invertebrates, the structure and function of animal body parts [30].
Understanding the concepts on the sub-topic of plant movement systems was at least compared to other topics because students found it difficult to observe them, especially microscopic movements such as nasti and taxis related to food transport, movement of water, and mineral salts in plants. Vitharana [31] explains that some students experience misconceptions about the process of transporting food, water, and mineral salts. In teaching the material on the topic of the biology movement system, students do not understand if they only read books. Conversely, if they are invited to practicum, they will be more enthusiastic. But lecturers' explanations are still given as signs for student learning. Teacher's understanding of human organ system material affects the teaching skill of teachers [32].

Conclusion
Misconceptions often occurred in students when they were less than optimal in accepting concepts so that it hampered their learning outcomes. Therefore, a diagnostic test was needed to measure student misconceptions, identify student difficulties, analyze the weaknesses and strengths of the learning carried out. The questions were suitable for research based on limited-scale trials in the development of  scientific literacy diagnostic assessments. The four-tier diagnostic question with scientific literacy was able to measure the misconceptions of grade 3B students on the movement system point with the result of 52.78% of students had misconceptions, 37.73% of students did not understand concepts, and 9.49% of students understood concepts. After giving the questions, students should be given a non-test instrument to analyze the factors causing the misconception