Usability Testing for Student Academic Information System in State Polytechnic of Creative Media

SIAM (Student Academic Information System) is an academic services information system that has been implemented at the State Polytechnic of Creative Media (Polimedia). It’s been two months SIAM has been integrated using Single Sign On (SSO). After we implemented SIAM new version we received a lot of complaints every day. In this research, the authors conducted a usability test on the SIAM new version in order to determine the effectiveness and performance of the application. Usability test components include effectiveness, efficiency, and satisfaction. The average usability 70% indicates that the SIAM new version has a good usability performance.


Introduction
Student academic information system (SIAM) is a system developed by State Polytechnic of Creative Media (Polimedia). The student academic information system (SIAM) can be accessed at https://siakad.polimedia.ac.id. Accessibility is one of the criteria for website. Accessibility is the level of user reach and the ability to achieve certain goals and contains all the attributes that allow the user to run the system [1]. Using SIAM students can find out academic information such as study result, study plan, lecture info such as class schedule and exam schedule, student information, bills and student account settings.  Currently SIAM has been integrated using the web Single Sign On (SSO) using Google. It's been two months after we implemented SIAM new version, we receive a lot of complaints every day, regarding changes to the new version of the SIAM user interface. Usability testing by ensuring the best method is to perform a test in which the representative participant interacts with the representative scenario. Examiners collect data on the success of the performance and satisfaction of the participants [2]. According to [2], the study results showed that the improvements made between the original website and the redesigned website resulted in sixty-seven percent greater user satisfaction. A second study reported that eight of ten tasks were performed faster on the web site that had been iteratively designed. Finally, a third study found that forty-six percent of the original set of issues were resolved by making design changes to the interface.
Therefore we need to usability testing the SIAM new version for increase the profitability of a new system and ensure that the SIAM application can be used effectively and efficiently. Effectively related to the user's success in achieving the goal of using the software and efficiently with regard to the smoothness of the user to achieve it [3]. Rubin J [4] purposed of testing is to collect data as a basis for identifying and correcting deficiencies in a product. The goal is to ensure the manufacture of a product that:  Useful and valuable according to target users  Easy to use  Become more effective and efficient  Satisfying and fun to use Usability as a measurement of how easy it is to find, understand and use the information displayed on a web site' [5]. Usability testing was conducted to ensure the improvement made and quantify the differences [6]. The benefits of testing a system for the organization include documenting the test result so that the organization can ensure that the product is better in the future or at least can maintain current usability standards [4]. Useful products can be more profitable and require less help desk assistance. Products that can be used properly can create customer satisfaction and tend to keep using these products in the future compared to other similar products.
Therefore, we conducted a usability test on new version of SIAM in order to determine the effectiveness and performance of the application. We use a questionnaire with Polimedia student as respondent.

Design usability measurements
Usability testing of SIAM is done by assign tasks to 138 respondents 73% male and 27% are female students respectively. The initial step of usability testing is provide the user with a number of prepared tasks when interacting with the system being tested. These tasks were given to respondents who used SIAM new version so they are no longer experience difficulties when doing these tasks. These tasks are used as "means of interaction" in measuring usability. No Task  1 Login into the systems using google email in https://siam.polimedia.ac.id as user, and then logout and login again. 2 Add and edit data on the profile menu 3 Display registration payment bill information 4 Display the summary study scores 5 Search for lecture schedule information.
Each of the above tasks can be explained as follows: Task 1. Login into the system as a user, then log out and log in again. The user is asked to log in as a user, starting from finding and filling in the login form to enter the user page, then being asked to look for the logout button then log in again. Task 2. Add and change data in the profile menu. Students are able to add data such as NIK, cellphone number, addresses, religions and other The task is considered completed when user success to edit data in bio. Task 3. Display billing information and virtual account number to carry out user registration in each semester. The task is considered completed when the user makes data edits in the bio Task 4. Displays the value of the semester study result. The task is considered when the user can see the information on the student semester learning outcomes. Task 5. Find lecture schedule information. The task is considered to complete when the user can see the student lecture schedule information.

Result
The level of Effectiveness and efficiency is measured using user's success rate. effectiveness and efficiency are calculated by the following [7] [8]. , Where is user's success rate, Xi = {0,1}. Satisfaction is the percentage comparison between the satisfaction score of the respondent with the multiplication of the maximum weight of the Likert scale with the number of respondents (n), Where is user's success rate, Xi = {0,1,2,3,4,5}.

Usability of applications is the mean of Effectiveness, Efficiency and Satisfaction by the following equations :
(%) = (Effectivenes+ Efficiency+ Satisfaction) 3 100% (3) After the user completes all existing tasks, the next step is to distribute questionnaires to 138 student respondents. The questionnaire is design using language that is easily understood by the respondents.and adopted from [9]. After conducting a trial based on the assignment that has been given, the result of the calculation of the usability value in detail can be seen in Table 2, Table 3 and Table 4. The text on SIAM is easy to read 76 2 The color display on SIAM is comfortable to see 72 3 Symbols, icon and instructions (navigation) are easy to understand 68 4 The menu display on SIAM interface is easy to recognize 68 5 Information details in SIAM is easy to find 54 6 You find the link you clicked in error 66 7 There is clear message about the error link 62 8 You find when you click the menu it doesn't give any response 62 Average 66 Table 3. Result of Efficiency.

No Question Efficiency (%) Efficiency
1 The menu that you click display quickly? 68 2 The SIAM application is easy to operate 72 3 Easily accessing information on SIAM 70 4 The functions offered in accordance with the purpose of application 74 5 The SIAM application provides appropriate information feedback 70 6 Easy to edit data on SIAM's profile menu 68 Average 70 Table 4. Result of Satisfaction.
Guritno defined interpretation of usability score described as follows [10]: Based on Usability testing, we gather score of SIAM is calculate of average of Effectiveness + Efficiency +Satisfaction, we found that score is 70 %. It means that the system indicate a good performance.

1
You want to visit SIAM page again next time 74 2 The Information I got was useful and informative 78 3 SIAM menu easily to display and remember 74 4 The way to operate SIAM is easy to remember 74 Average 75

Conclusions
The result of usability testing can be summarized as follows the relationship is adjusted with each of the usability aspects in table 3, 4 and 5 , it can be said that the SIAM application software that has been used so far has good usability values, including good effectiveness, efficiency, and satisfaction. This is indicated by the value of the usability results for 3 criteria, as follows: a. The average attribute value of 66 % indicates that the SIAM applications has a good effectiveness aspect value b. The average attribute value of 70 % indicates that the SIAM application has a good efficiency aspect value c. The average attribute value of 75 % indicates that the SIAM application has a good satisfaction aspect value d. And overall, The average usability 70 % indicates that the SIAM applications has a good usability performance.