The development of student worksheets: questions of PISA model to analyze the ability of mathematical literacy in junior high school

Mathematical literacy was urgent to promote the industrial revolution 4.0. There are two purposes of this study. 1) Describing how to develop questions of PISA model in content of uncertainty and data that were valid and feasible. 2) Examining the potential effect of students in complete the questions of PISA model. It was research and development with ADDIE type, which consisted of analysis, design, development, implementation, and evaluation. The research was conducted in SMP Negeri 1 Karanganyar on 2018/2019 academic year. Technique of data collection used: walkthrough, assessment, and task-based interview. Based on findings, this research had produced the questions of PISA model that were valid and feasible in content of uncertainty and data. Valid drawn from the result of validators judgement which states that the questions had developed were good in content, construct, and language. It demonstrated that 26 of 32 students were able to solve the questions of PISA model with the details i.e.: 26 students had achieved level 1 (81.25%), 20 students had achieved level 2 (62.50%), 16 students had achieved level 3 (50.00%), 12 students had achieved level 4 (37.50%), 8 students had achieved level 5 (25.00%), and 4 students had achieved level 6 (12.50%).


Introduction
Currently, Indonesia is entering the era of the industrial revolution of 4.0. The industrial revolution 4.0 needed human resources with the ability of productive and reliable, critical, systematic and logical thinking [9]. Indonesia as a country that is endowed with rich resources, including human resources, has to challenge in the education world. Education in Indonesia is demanded to be able to produce the nation's successors with a complete competence (overall), which includes competencies of knowledge, attitudes, and skills. This can be achieved by directing student-centered learning activities, meaning students are required to find independently related to the resolution of a problem in question [4]. Literacy ability is one of the factors that can encourage the implementation of student-centered learning. Literacy includes the ability to analyze, give reasons, convey ideas effectively, formulate, solve, and interpret problems in various forms and situations [10].
Mathematics is one of the compulsory subjects in primary and secondary education. In the process of learning mathematics, literacy is one of the abilities students must have. Currently, there are International organization that assess students' mathematical literacy abilities, namely Programme for International Student Assessment (PISA). Indonesia has participated in PISA studies six times during 2000-2015. However, since this participation, Indonesia students' mathamatical achievements have IOP Publishing doi: 10.1088/1742-6596/1538/1/012065 2 not shown satisfactory results. During the period 2003-2009, there were almost 80% of Indonesian students who were only able to reach the level limit below level two of the six levels of questions tested [5]. Stacey [15] explained that in PISA mathematics 2009, almost all Indonesia students only reached level three, while only 0.1% of Indonesian students were able to reach levels five and six. The deterioration in these results is further strengthened by the results of the latest PISA survey in 2015, which placed the mathematical literacy ability of Indonesian students ranked 63 out of 70 countries with relatively low levels of achievement were almost all Indonesian students in this survey were only able to reach level three [11].
The low mathematical achievement in Indonesia on PISA is due to the appreciation of mathematical literacy in Indonesia still on minimum. According to Bloom's taxonomy [6], the division of cognitive domains is classified into six levels i.e.: remembering (C-1), understanding (C-2), applying (C-3), analyzing (C-4), evaluating (C-5), and creating (C-6). In Indonesian schools, students are only accustomed to being given questions at the C-1, C-2, and part of C-3 levels, while the PISA International standard test questions are not only questions that measure the ability to solve ordinary problems, but it also measure the students' ability to solve problems started by analying, formulating, and communicating ideas to others in written form. Mathematical literacy problems tested in PISA require reasoning and problem-solving abilities [1]. A student is said to be able to solve problems if he can apply the knowledge he had previously obtained into new situations that are not yet known. This ability is commonly referred to as higher-order thinking skills.
Shukla and Dungsungnoen [14] states that higher order thinking occurs when a person takes new information and interrelates information stored in memory to achieve a purpose or find possible answers in perplexing situations. This explanation means that the ability to think at a higher level occurs when someone associates new information with information already stored in his memory and relates it and/or rearranges and develops the information to achieve a goal or find a solution to a difficult situation solved. There are four categories of content related to mathematics in PISA 2015, namely change and relationship, space and shape, quantity, uncertainty and data [10]. Uncertainty and data content are one of the contents related to probability and statistics. The problems presented in this content are familiar and simple, but the solution is often difficult for students.
Problems related to mathematical literacy at level 4-6 (analyzing, evaluating, and creating) presented in PISA are problems that demand high-level thinking. Higher-order thinking skills (HOTS) are complex and cannot be easily defined, but their characteristics are quite easily observed in practice. Yen and Halili [20] explained that the characteristics of HOTS included, resulting in several solutions, involving uncertainty, and the process of making meaning.
The researcher did document analysis of learning implementation plan on mathematics in SMP Negeri 1 Karanganyar. The analysis showed that the teacher in implementing learning had not implemented HOTS-based student worksheets. The results of the document analysis were strengthened by the interview results of a mathematics teacher who explained that the teacher had difficulty in developing HOTS-based student worksheets. Thus, he preferred to test the available questions in the textbook. Thus, school had not yet facilitated student worksheets based on higher order thinking skill that could be improve the ability of analyzing, evaluating, and creating.
In principle, higher-order thinking skills are skills that can be trained [7]. This should be a concern for educators to train students to have these skills. Higher-order thinking skills cannot be formed instantly, but they need to be practiced continuously. However, educators can improve students' mathematical literacy skills by designing, developing, and implementing PISA student worksheets on content of uncertainty and data in learning.

Research method
This research is research and development (R&D). Research and development is a process or steps to develop new products or improve existing and accountable products [16]. The study was conducted in SMP Negeri 1 Karanganyar on 2018/2019 academic year. The development model was conducted by applying ADDIE model which consisted of 5 stages, namely analysis, design, development, implementation, and evaluation. There were many data collection techniques in this study, namely walkthrough, assessment, and task-based interview. 1) Walkthrough was used to examine the validity of products on content, construct, and language. 2) Assessment by students were used to examine the feasibility (including clarity, readability, and usability) of product. 3) Task-based interview was used to examine the potential effects of student's literacy abilities. The measurement of feasibility for this research was used a Likert scale with five scales, i.e.: very feasible (score 5), feasible (score 4), less feasible (score 3), not feasible (score 2), and very improper (score 1).
Researchers developed a product in the form of a PISA student worksheet based on the clues. Furthermore, expert validation was conducted to examine its validity. Students' worksheet declared valid then were tested to students on a small scale (limited test). This is aimed to examine the feasibility of the products. Products that had been declared worthy were then tested on students on a large scale (field test). This aims to examine the potential effects of students in solving PISA model problems. The implementation of the research activity is illustrated in Figure 1.

Result of research
The results of this development research were the student worksheet: the questions of the PISA model on content uncertainty and data. This development research was conducted to examine the potential effects of students' mathematical literacy abilities. The explanation related to the results of the activities at each stage of the research development is as follows.

Analysis
The analyses stage conducted in this study cover needs analysis, curriculum analysis, and literature study. Based on the results of a needs analysis study in pre-survey activities, schools need to develop a product in the form of a PISA student worksheet. This is because the problem solving presented in PISA requires students to use higher-order thinking skills. Researchers need to adjust the student worksheets that will be developed with the applicable curriculum in the school. Because of this, I Implementation E Evaluation D Design A Analysis Conduct need-analysis, curriculum analysis, and literature study.
Designing the initial draft, designing a validation questionnaire instrument, and designing a question's clue.
Develop student worksheets based on the initial draft for expert validation. researchers need to do curriculum analysis. In this case, the researcher examines the data relating to core competencies and basic competencies which are the main objectives of learning achievement in schools.
The use of literature documents served as the guideline for needs analysis and curriculum analysis. The researcher examined the documents needed to develop the product. These documents included "Minister of Education and Culture Regulation No. 37 of 2018 concerning core competencies and basic competencies" and a guidebook on "Assessment and Analytical Framework" issued by PISA. Thus, the researcher always refers to kinds of literature documents related to the product.

Design
The second stage of this research and development is designing a product (students' worksheets based on the PISA model). Activities done in this stage were: designing the first draft based on the blueprint and designing the instrument validity of the product. The researcher designed the first draft based on the study that has been conducted in the analysis stage. The assessment instrument of the product was developed to examine the validity and the advisability of the product, also to examine the potential effect of the product toward students' mathematical literacy ability.

Development
The researcher developed the first draft in a product. They developed questions based on the PISA model started from level 1 (the ability of remembering) to level 6 (the ability of creating). The product had been developed then validated by expert. The result gave a solution that the researcher was asked to add the figure in developed questions. The aimed to make students understand the problems easily. Then, the researchers revise the product based on the experts' suggestion. The result of the developed products in the HOTS criteria (the ability of analying, evaluating, and creating) as shown in Figure 2 Adit estimates that he can climb mountains at an average speed of 1.5km/hours, and he goes down the mountain at twice the speed of the climb. The estimated of speed includes the rest time during the trip. By using Adit's estimate, when he must start the climb at the latest so he can return at 8 pm? Sari cycled to school from her house of 4 km (pass through Semeru street) and takes 9 minutes. Sari takes the short route of 3 km (pass through Merbabu street) and takes 6 minutes to back home. What is the average speed of Sari (in km/hours) for trips to school and back to her home?

Implementation
In this stage, the researcher examine the developed products in two kinds of test, namely limited test and field test. Limited test aimed to test the effectiveness of the developed product by involve 6 subjects. This aimed to get a direct suggestion from the students in the form of responses and comments. The summary of the feasibility test results in limited test as shown in Table 1. The result of the feasibility test in Table 1 shows that the developed products are proper to be used and it can be continued to the next test (field test). This field test involved 32 students. The result of the field test of the developed products in the criteria of HOTS (ability of analyzing, evaluating, and creating) can be seen in Figure 5, Figure 6, and Figure 7.  Figure 5 shows that the subject could finish the worksheet in the analysis level well. Subjects understand the problem that should be solved. Thus, they were able to solve the questions correctly. The result of an interview with the subject when he solved the problems can be seen as follows (R= the researcher; A= the subject who could solve the problem until the analyzing level). R = Have you ever answer questions like this before? A = Not yet, Mom. R = What did you think about this worksheet? A = The questions are easily understood, but it is difficult to answer. R = What did you feel after accomplishing this worksheet? A = I was very happy because the questions were easily understood, although I could not answer those questions yet. The maximum of time that we need to climb and down the mountain is 9 hours. They must start at 11 am. Some students could accomplish the worksheet until the evaluation level, as it is shown in Figure  6. The subject understands the given problem. They finds the formula that could be used to solve the problem. Results of an interview with the subject when he solved the problems can be seen as follows (R= the researcher; E = the subject who could solve the problem until the evaluating level). R = Have you ever known questions like this before? E = I think, not yet Mom. R = What did you think about this worksheet? E = The question is unique and challenging enough to be solved. R = What did you feel after accomplishing this worksheet? E = I was happy Mom, although I hesitated the questions could be answered.  Figure 7 shows that subject understands the given problem. Subjects used an effective and an efficient strategy to accomplish the questions. Thus, they were able to complete the questions appropriately. When conducting field tests, some students could finish student worksheets starting from level 1 (remembering ability) until level 6 (creating ability). The result of the subject in the creating level can be seen in Figure 7. The result of an interview with the subject when he solved the problems can be seen as follows (R= the researcher; A= the subject who could solve the problem until the creating level). R = Have you ever known questions like this before? C = Not yet, Mom. R = What did you think about this worksheet? C = The questions were easily understood thus those were challenging enough to be solved, Mom. R = Did you accomplish all questions and how did you feel when you answered these questions? C = Yes Mom. I answered all of the questions. The questions were interesting and very easy to understand.

Evaluation
The evaluation result of the developed product was 26 of 32 students could finish developed questions based on the PISA model. The percentage result of the students in every level of developed PISA model questions can be shown in Table 2. Based on the implementation result of the product and the result in Table 2, the developed PISA model questions had potentially positive effects on the students' mathematical literacy skills. So, the product can be used as an alternative material in the learning.

Discussion
The first stage of this research and development was analysis. In this stage, the researcher conducted a need-analysis, curriculum analysis, and literature review. The use of need-analysis aimed to identify the priority of needs that must be fulfilled soon [2]. Based on the result of need analysis, the school needs to developed students' worksheets (questions based on the PISA model). This is because the questions of PISA model could be stimulated students to use higher order thinking skill. The higher order thinking skill could not be created instantly, but it needs continual training [17]. The researchers developed the product based on the curriculum in the school. This is because the curriculum was an educational program that contents various materials and learning experiences programmed, it is planned and designed systematically, and it is become guidance in the learning to reach educational goals [13]. The researcher also developed a product based on the study of literature that can be used as references.
The second stage of this research and development is called design. In this stage, the researcher designed and adjusted the first draft based on the obtained result of the study in the analysis stage. This aims to make the product efficient (it can be used optimally in the learning). Developing the product efficiently can optimize the quality of the existing resources [8].Thus, through the appropriate design of developing a products, it can produce an efficient product resulting in synergy in improving the quality of human resources.
In the third stage, the researcher developed the design of the first draft in a product. Then, the researcher validated the product that has been realized to experts. The validation result by experts conclude that the developed product was valid in case of the content, construct, and language. The validation result from the experts seems important. This is because the quality of the product can be said well if it meets some aspects including valid, practical (it is easily used), and effective (having benefits) [19].
Implementation is an activity of implementing a plan that has been made in detail and has been programmed before [3]. In this stage, the researcher conducted tests twice including a limited test and field test. From the assessment result of the pilot test, it can be concluded that the developed product is proper to be used. This proves that the product can be used on a larger scale. Based on the students' responses in an interview tasks-based in the field test, the developed product got positive appreciations. On the other words, the developed product is proper to be used in learning. This is because the proper product always gets positive appreciation from the product user [12].
Evaluation is the last stage in the research and development. From the result of the evaluation product, it can be concluded that 26 of 32 students can solve the developed questions based on the PISA model. In other words, the developed questions based on the PISA model have a positive potential effect on students' mathematical literacy skills. It is very important for the students having this mathematical literacy. In consideration, the benefits of mathematical literacy skills are able to help an individual to implement mathematical in the real world as a form of participation of constructive and reflective community [18].

Conclusion
This research had produced the questions of PISA model that were valid and feasible in content of uncertainty and data. Valid drawn from the result of validators judgement which states that the questions had developed were good in content, construct, and language. It demonstrated that 26 of 32 students were able to solve the questions of PISA model with the details i.e.: 26 students had achieved level 1 (81.25%), 20 students had achieved level 2 (62.50%), 16 students had achieved level 3 (50.00%), 12 students had achieved level 4 (37.50%), 8 students had achieved level 5 (25.00%), and 4 students had achieved level 6 (12.50%).