Development of self-diagnostic assessment media by scores based on desktop

This article was written based on research which aims to develop self-diagnostic assessment media by scores based on desktop on impulse and momentum material. In addition, it aims to determine product quality based on expert assessment and student responses. Product was equipped with the results of the diagnosis and learning suggestions for students based on the scores obtained on each item. The type of research used was the 4-D Model Research and Development. The subjects of this study consisted of 2 experts, 2 practitioners, 3 colleagues, and 44 grade 10 students at Muhammadiyah 2 Yogyakarta High School. Data collection was carried out using product assessment sheets and student response questionnaires. Assessment sheet was used as a guide to the feasibility of the product. The results of expert assessment showed that the products that had been developed were in the “Very Good” category with an average score of 0.90 on a scale of 0-1. Student responses were used as an assessment of product legibility. Student responses showed the “Good” category, with an average score of 3.28 on a scale of 1-4. Therefore, self-diagnostic assessment media by scores based on desktop on impulse and momentum material was appropriate to be used to diagnose students’ initial abilities. In addition, the results of the diagnosis can be used as a basis for evaluation to carry out learning by teachers, so that effective learning can be achieved.


Introduction
Now, the use of computers, especially in the field of education, cannot be avoided.This era has been marked by the significant disruption of technology to the education system [1,2].The speed and accuracy of data processing in the computing era can help teachers process all student data as evaluation material [3,4].This process could only happen if you used an application that matched the program, device, and purpose of the user.One way to use it was to use an application as an assessment media that worked on a computer or desktop device.Desktop-based application as assessment media had advantages, the results of the assessment could be obtained instantly, thereby reducing the burden on teachers to correct students' answers [5,6].The pandemic has played an important role in the transition from online learning to online learning.
Technology has helped teachers implement effective learning.To achieve effective learning, first, the difficulties or problems experienced by students need to be known.Furthermore, the solution was analyzed and formulated by teachers [7,8].Diagnostic assessment is needed by teachers to recognize their students.Diagnostic assessment is used to measure and detect understanding and difficulties experienced by students so that appropriate treatment can be continued as a follow-up step [9,10].However, based on the results of interviews with teachers, this was not done at the beginning of learning.This is because the data processing process takes a long time.So that computer-based assessment is needed to help teachers.
Impulse and momentum were materials that were often encountered in everyday life.There are many applications that can be observed in human activities.However, obstacles were found in the learning process of impulse and momentum.These findings were reinforced based on previous research [11][12][13].Therefore, it is necessary to diagnose students early before being given treatment during the learning process.
Self-diagnostic assessment media by scores based on desktop is a practical tool for students to use anywhere and anytime.Self-diagnostic assessment media based on desktop became a solution in an era of rapid technological development.The results of the suggestions to students that were displayed, can be used as a basis for evaluation for treatment when learning takes place.Based on this description, the purpose of research was to develop a self-diagnostic assessment media by scores based on desktop as a tool to help teachers achieve effective learning.In addition, knowing the quality of products developed based on experts judgment and student responses.

Research method
The research used in developing an independent diagnostic assessment based on desktop applicationbased score was research and development.Development procedure was adapted from a 4D model.The model consists of 4 stages of development, namely: define, design, develop, and disseminate [14].
The activities carried out at the define stage, namely: preliminary analysis, student analysis, and material concept analysis.The results obtained from the define stage were interviews with teachers, learning observations, literature reviews, student characteristics, and the material studied.
Activities carried out at the design stage were selecting media and diagnostic assessment formats, designing initial software designs, compiling diagnostic assessment instruments, compiling product feasibility instruments.The results obtained from this stage are the initial product design, diagnostic assessment instrument, and product feasibility instrument.
The develop stage had two main activities, namely: product evaluation and product testing.The products that have been developed based on the initial design were assessed by media experts, physicists, practitioners, and colleagues.The results of the assessment and suggestions were used as the basis for product improvement.Media experts and physicists were lecturers from Universitas Negeri Yogyakarta.Practitioners were educators working at Muhammadiyah 2 Yogyakarta High School and TQ Sultan Fattah High School Cilacap.Colleagues were 3 graduate students at Yogyakarta State University.After being revised based on the assessment results, the application in the form of a self-diagnostic assessment media based on desktop-based scores on impulse and momentum material, was tested.The trial was conducted on 44 grade 10 students majoring in Mathematics and Natural Sciences at Muhammadiyah 2 Yogyakarta High School.The results obtained from the trial were in the form of student responses in using the product.Students' responses were used as a reference for revising the product and producing the final product.
The last stage or disseminate was used as a publication to the public.The data from research that has been done is processed and written in a single article.Completed articles are submitted to accredited international publishers.
The instruments used in data collection process consisted of 2, namely the product assessment sheet and student response questionnaires.Assessment sheet was used to obtain product feasibility data.Student response questionnaire functioned as a tool to obtain student responses to the product.
The data obtained, analyzed by descriptive analysis.Analysis was carried out by calculating the average score of the results of expert assessment and student responses.On the expert assessment sheet, each aspect consisted of 3 statements with each statement being worth 1 as the maximum score and 0 as the minimum score.On the student response sheet, each aspect consisted of 6 statements with each statement being worth 4 for the maximum score and 1 for the minimum score.The average score of the assessment results and responses to each aspect was interpreted in five assessment categories [15].The 5 categories of assessment in question, starting from the lowest to the highest, were very bad, not good, fair, good, and very good.

Define stage
Observations were made during online learning.The results of observation showed that teachers could master the class well even though the learning was done in a network.The learning resource used was a summary of the material on the power point sheet made the previous day by the teacher.In addition to displaying the presentation sheet that was made, the teacher also used the help of a graphic table as an aid in directing the subject.Even though they have used the latest learning methods, the teachers still treated students with the same treatment without adjusting the abilities that the students had brought.Some of the students, seemed enthusiastic in answering the teacher's questions because they already had more abilities than others.Several other students also looked passive because they did not understand the material even though it had been explained.This is in accordance with previous studies which showed variations in the initial abilities of students [16][17][18][19].In general, students showed various kinds of behavior seen from the aspect of abilities brought by students in the classroom.
The results of observations and interviews were the reason for developing an application in the form of self-diagnostic assessment media by scores based on desktop.Other results of the analysis, one of the materials that became an obstacle and was appropriate for diagnostic assessment was impulse and momentum.While the results of the analysis of learning objectives were adjusted to the revised 2013 curriculum that applies in schools.

Design stage
The results of design stage were in the form of storyboards, flowcharts, and initial product designs.Storyboard and flowchart were used as guidelines for designing desktop-based applications in the form of diagnostic assessment media.The software is run using a computer or the like running on a Windows program.
The initial design of application-based diagnostic assessment media on the desktop had components in the form of 1) cover, 2) identity form, 3) instructions, 4) diagnostic test of impulse and momentum material aspects of understanding concepts and mathematical representations, 5) item navigation, 6) summary of results.consisting of scores, answer keys, and suggestions to students for each item, and 7) developer profile.Student suggestions were based on the scores obtained.The classification of the calculation of the score for each number can be seen in Table 1 [20].

Develop stage
Develop stage produced the final product in the form of an application in the form of desktop-based self-diagnostic assessment media.Figure 1 shows the initial appearance of the product after filling in the identity.Diagnostic test assessment consisted of 40 questions.Each item consisted of one question and one reason.Questions and reasons were in the form of multiple choice with 1 correct answer key and 4 distracting choices.This is needed to reduce the possibility of students making guesses and reduce the rate of diagnostic assessment errors [21,22].One of the questions developed can be seen in Figure 2. Based on Figure 2, it also shows the existence of navigation to make it easier for students.The navigation of questions is useful to make it easier for students to choose questions that are easier to work on and reminders of questions that have been completed, some have been done, are in progress, or have not been done [23].Navigation is blue for questions that have been completed, pink indicates that the questions have been partially completed, yellow indicates the position of the items being worked on, and white for items that have not been completed.
Students who have successfully completed all items will move to the results page.An example of a display of the results page is shown in Figure 3. Figure 3 shows the level of ability of students in each aspect and the level of ability of students on impulse and momentum material.Score was obtained by calculating the average of the scores for each item.So that it is easy for students to understand their abilities, at the bottom of the results, details of students' answers, answer keys, and suggestions that students need to do in subsequent learning activities have been made.Shown scores and suggestions on one of the numbers that can be seen in Figure 4. Shown in Figure 4, students got a score of 3 (right reason but wrong answer).These results indicate that students have understood the definition of momentum conservation.However, students needed to re-learn how to compare a result of conservation of momentum.Scores and suggestions have been provided across all items, so students can see a summary of the results of their work.
The summary of student work can be downloaded and then sent to the teacher.The teacher analyzed students' abilities based on the summary.The results of the analysis were used as a basis during learning to determine the best treatment for students, so that the achievement of effective learning objectives was maximized.

Product rating.
Assessment was carried out by experts, practitioners, and colleagues.Every assessor assessed the product covering several aspects of the product, including appearance, language, and usage.Each of aspects has several sub-aspects which are then further divided into three statements.Assessment results can be seen in Table 2.The results of the assessment showed that every aspect of the assessment was in the "very good" category.In addition, several suggestions were given, including: 1) the display of moving box lines as a background was changed so that students would not get dizzy, 2) writing vectors was written according to applicable rules, 3) the use of adapted words was simpler for understood by students.Based on the results of the assessment and the results of previous studies [24], desktop-based application in the form of self-diagnostic assessment media by scores, stated to be used during learning.

Student response.
Application-based self-diagnostic assessment media for desktops that were assessed by experts, were tested on students to obtain their responses.It aims to obtain student responses about the use of the product that was developed.Student responses consisted of aspects of appearance, language, usage, and novelty.The results of student responses that were successfully obtained are presented in Table 3. Students' responses showed that all aspects were in good category.Usage aspect got the lowest score when compared to other aspects.This was because there were still several problems that occurred due to students' devices not being appropriate.In addition, the display on some devices became a bit cropped.Therefore, it was necessary to replace the temporary device.Nevertheless, the categories of students' responses and the results of previous research [22], in the application of self-diagnostic assessment media by desktop-based scores, received positive responses so that they could be used as a tool to support independent learning by students.

Disseminate stage
The activity carried out at this stage was in the form of publishing the results of the development at the 5th International Conference on Research and Learning of Physics (ICRLP 2022).

Conclusion
The quality of the application developed as a self-diagnostic assessment media based on a desktop-based score was "very good" based on the assessment of experts, practitioners, and colleagues.In addition, the product quality category based on student responses was "good".Therefore, the product developed was feasible to be used as a learning support tool to achieve effective learning objectives, especially in impulse and momentum material.

Figure 1 .
Figure 1.Initial Appearance of the Product After Filling in the Identity.

Figure 2 .
Figure 2. Display on Question Item number 7.

Figure 3 .
Figure 3. Summary of Student Work Results.

Figure 4 .
Figure 4. Example of Scores and Suggestions on One of the Items.

Table 1 .
Score Classification

Table 2 .
Product Assessment Results

Table 3 .
Student Responses to Products