The Effect of Psychometric Analysis Series Training on the Item Analysis Ability of High School Teachers in Bandung

The Psychometric Analysis Series (PAS) is software developed for item analysis, which consists of item difficulty, item discrimination, distractor power, reliability and descriptive statistics of a test. This study was conducted to look at the effect of the Psychometric Analysis Series training on the Item Analysis Ability of high school teachers in Bandung. The study was conducted using a one-group pretest-posttest design. The ability of Item Analysis is measured using an instrument that is scored between 0 to 10. The training participants consisted of 35 high school teachers in Bandung. The data obtained were analyzed using the t-test. The results showed a significant increase in the ability of item analysis in the posttest compared to pretest (t = 8,777; p = 0,000). Based on these results it can be concluded that the Psychometric Analysis Series training can improve the ability of item analysis of high school teachers in Bandung.


Introduction
Psychological tests are widely used in various fields, both for selection, evaluation etc. Therefore a good understanding of tests is needed by Psychologist.Students in the Faculty of Psychology are taught the concepts of Psychometric and TestConstruction. In these courses students are taught how to construct a test and evaluate test empirically. According to Kaplan (2001) the psychometric properties of a good test are:  A good test consists of good items  A good test must be reliable  A good test must be valid To evaluatea test empirically, students are also taught a variety of software for analyzing items, reliability and validity. One of the most widely used software for analyzing test is ITEMAN. ITEMAN is one of the analysis programs that comprise Assessment Systems Corporation's Item and Test Analysis Package since 1981. ITEMAN analyzes test and survey item response data and provides conventional item analysis statistics (e.g., proportion/percentage endorsing and item-total correlations) for each item, in order to assist in determining the extent to which items are contributing to the reliability of a test and which response alternatives are functioning well for each item. In addition to item-level statistics the ITEMAN program also provides statistical indicators on the performance of the test as a whole (e.g., mean, standard deviation, reliability, median p-value) (Assessment Systems Corporation, 2016).
Beside ITEMAN, Lertap has been around since 1973, running on mainframe computers, then desktop computers. jMetrik and SAS University were two possibilities that came immediately. jMetrik has benefited from substantial enhancements at the end of 2016, when its support for IRT was extended to cover more IRT models. jMetrik's graphics are excellent, and they include an option to overlay IRT plots with empirical points. The University edition of SAS was released in 2014. A special macro in Lertap 5 makes it possible so that users could take advantage of the strengths found in SAS's IRT procedure (Nelson, 2017).
The high price of this software causes rampant piracy of existing software. Therefore this condition has encouraged the interest of researchers to develop test data analysis software that is cheap, appropriate and comprehensive for researchers in Psychology and other sciences.
Jatnika, Purwono, and Haffas (2019)  The making of PAS is done by using C++ programming language and SDLC method. SDLC (software development life cycle) is a step or phase sequence that presentsa model for development management and the life cycle of an application or software.This method consists of stages: Planning, Analysis, Design, Implementation, Testing, Maintenance (Blanchard & Fabrycky, 2006). PAS will be able to overcome the weaknesses found in the existing item and test analysis software and will also be the solution for the high-priced item analysis software. PAS needs to be evaluated so it can be the right and "user-friendly" software that can be used to evaluate tests.
In this study PAS was tried on a teacher at a high school in Bandung. The selection of teachers as research samples was done because teachers must also understand how to evaluate a test as part of evaluating learning outcomes. The research will be conducted using an experimental design. Using this design, the researcher wanted to find out how the effect of PAS training on the item analysis ability of high school teachers in Bandung.

Method
This research was conducted using one group pre-test post-test design to look at the effect of the PAS training on the Item Analysis Ability. PAS is software developed for item analysis of a test that contains menus like:  Item difficulty  Item discriminality  Distractor Power  Inter item correlation  Reliability  Descriptive Statistics An evaluation of the training program will be carried out using the concept of program evaluation from Kirkpatrick (2006). The program evaluation stages will be carried out as follows:

Figure 1: Training Evaluation
Evaluation of the training is carried out at the learning stage, to find out the extent of the increase in knowledge and understanding of participants after attending the training. For this reason, pretest and posttest measurements were carried out for learning outcomes. The pretest and posttest measurements are performed using the Item Analysis measuring tool in the form of multiple choices with 5 answer choices.The ability of Item Analysis is measured using an instrument which is scored between 0 to 10 In addition, the participants' reaction to PAS training was also measured using a questionnaire consisting of 4 statements with a rating scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Agree and 4 = Strongly Agree.
The training participants consisted of 35 high school teachers in Bandung. The data obtained were analyzed using a paired-sample t-test.

Result and Discussion
The data processing gives the following result:  Based on Table 1, it can be showed that item analysis ability at the posttest is higher than the pretest. T-test results show that this increase is significant (p<0.001). It can be concluded that training can improve the ability of participants in item analysis.
In addition to the item analysis ability, training reaction measurements were also carried out. The results of the training reaction can be seen in the following table: Based on Table 3, it appears that PAS training is felt to be useful and easily understood by participants. Besides that, the training participants felt that the facilitator mastered the material well so that the material was well delivered. This condition caused item analysis ability at the posttest is higher than the pretest.