Heuristic evaluation and analytical network process for analyzing the role of websites in higher education

It is currently thought that university websites play important roles in maintaining communication with stakeholders. The evidence shows that the usability aspect of the website positively correlates with governance support. This paper presents a new approach to identify, assess and evaluate usability aspect base on ten Heuristic Evaluation principles by implementing the ANP method. This method works by determining the weight of the best site criteria according to Webometric version ranking. The initial questionnaire was tested for validity and reliability using total item correlation analysis, the r-value was obtained in the range 0.706-0.919 (valid) and the reliability value in the range 0.861-0.939 (reliable). It was also found that by implementing expert judgment tests, some questionnaires are required more editorial adjustment. Interestingly, the layout of the websites hit the usability criteria. Another ten HE aspects run well while some light improvements were required for error prevention, recognition rather than recall, help users recognize, diagnose, and recover from errors aspects. The most remarkable result to emerge from the data was that visibility of system status, match between system and the real world, and consistency standards shown good result.


Introduction
The Internet has changed the traditional way an organization interacts with the public and provides access to users to improve operational efficiency. The website is a key component of organizational survival in global competition. The website represents the organization to convey information about its products and also functions as a platform where organizations can interact with stakeholders. The success of a website is to support the organization's interaction and communication with users [1]. Evaluation of university websites is important to do to determine the level of customer satisfaction and pay attention to the ease of usability factor. Usability is a set of methods used to evaluate human interactions with products, aiming to identify problems and make improvements to increase usability [2].
Heuristic Evaluation (HE) is a usability evaluation method that is used to identify usability problems in interface design on a website [3]. HE involves a group of evaluators who are in charge of examining and assessing functionality in the interface design of a system based on predetermined usability principles. HE is performed by people experienced in interface design and examines what factors represent a potential problem for less experienced users. Through the HE evaluation, the developer can IOP Publishing doi: 10.1088/1742-6596/1943/1/012112 2 pay attention to things that need to be improved based on the evaluation process that has been carried out. HE was developed by Jacob Nielson and Rolf Molich to assess whether a website is easy to use or not [4].
Besides the evaluation results with HE, the action is needed for decision making related to the quality of the website in order to produce a better quality website. Therefore, it is necessary to have an interrelated qualified assessment between the problems found in the results of the HE analysis. The qualifying application using the ANP method is a method that takes into account the relationship between the criteria and sub-criteria for assessing usability with one another [5]. Previous research has conducted an evaluation of the library website to improve library services to users by using the HE method to identify usability problems and the Analytic Hierarchy Process (AHP) to determine the qualified criteria from subjective assessments from decision makers. AHP is used to help determine further needs and set priorities for a problem that had to be fixed first. This research can be used as decision-making material for stakeholders; it can be developed better and in accordance with the usability aspect. This research method used ten HE principles to identify usability on the website, and the Analytical Network Process (ANP) method was used to assess priority criteria. This research was research that focused on evaluating the usability of the website object and evaluating the user interface of the university website.

Method
This research was a mixed method research, namely research that combined two forms of research approaches, namely qualitative research and quantitative research. Mixed research is a research approach that combines qualitative research and quantitative research [6]. Qualitative methods with a descriptive approach were used to find out what interface factor designs describe potential problems for users through the HE evaluation; the developer could pay attention to things that need to be fixed based on the evaluation process that has been carried out. The quantitative approach was used for the evaluation analysis process using the HE method. An inferential analysis would be needed to examine the correlation between these usability factors whether there was a causal correlation between the variables of the usability factor model, namely between the usability aspect factors and the usability aspect performance on the website condition associated with the 10 HE principles. Furthermore, for analysis with the ANP method, the writer used a qualitative approach. ANP analysis was to determine the amount of contribution between clusters and elements from one another that were paired with each other. Table 1 shows the five best universities according to Webometrics. Table 1 shows the best five universities according to the webometrics ranking of world universities. This study classified the user subjects into 3 categories, namely lectures, students, and prospective students. The subject/sample in this study was determined by using the purposive sampling method. Purposive sampling method was done by determining the criteria that would be used as resource persons. The user criteria were as follows: a. Inclusion criteria were criteria that were appropriate as research subjects to be used as samples. The inclusion criteria in this research were: a) Lecturers, students, and prospective students. b) The user had visited the website of each university b. Exclusion criteria were subject criteria that could not be taken as a sample. The exclusion criteria in the study were: a) Not a Lecturer b) Not as students and prospective students c) Never access the website at all Based on the user criteria that had been set, the subjects in this study could be selected according to their needs. The number of users in this study was chosen as many as 50 lecturers, 50 students and 50 prospective students. Figure 1 shows the Research sample proportion. The first data analysis design was testing the website usability value using the Heuristic Evaluation method. There were 2 stages in this analysis, namely the pre-user testing and prost-user testing stages. In the pre-user testing stage, the authors conducted a questionnaire mapping from the usability aspect associated with the 10 Heuristic Evaluation principles. After the questionnaire was tested, it was tested on the user to obtain the level of validity and reliability of the instrument. Besides, the validation test involved experts for expert judgment on the results of the validity and reliability trials to get more relevant data collection tools.
Furthermore, the second stage, post user testing, in which was the stage of the website testing results would be processed with a descriptive statistical process to determine the tendency of respondent behavior in assessing the usability aspects of the website. At this stage, a descriptive statistical test was carried out. Furthermore, the results of data collection looked for the mean, median, and mode values of the questionnaire filling results. In the process of calculating descriptive statistics, the output obtained was the mean value of each Heuristic Evaluation principle. After getting the results at the Heuristic Evaluation analysis stage, then the conclusion from the results of the Heuristic Evaluation aspect was used as an alternative selection of usability with the Analytic Network Process (ANP) method to determine the magnitude of the influence between clusters and elements with one another [3].

Construct validity test results for every aspect of HE
After the validity test by an expert judgment, the instrument was tested. Constructive validity was intended to show the extent to which the instrument was able to reveal the measured data. The technique of testing the validity test was to use the Pearson bivariate correlation (Pearson moment product) [7]. This analysis was done by correlating each item's score with the total score. The total score was the sum of all items were able to provide support in revealing what Valid wants to reveal. The basic formula as follows: If the results of the calculation were r count ≥ r table or the value of Sig <0.05 (5%), then the instrument or question item had a significant correlation to the total score so that it was declared valid. It was better if r count < r table or Sig value > 0.05 (5%) then the item was declared invalid and must be removed or dropped in further research. Based on the construct validity for each aspect of HE, the result could be seen in Table 2.

Usability evaluation with the heuristic evaluation method
In order for the system to achieve certain goals effectively, efficiently, and to achieve user satisfaction, usability evaluation was needed. One of them is Heuristic Evaluation (HE), which is a usability evaluation method to improve a design effectively by using a simple set of related heuristic. The HE processing allows the evaluator (user) to evaluate and rate the system from any heuristic that indicates     Based on the result of the all table by the user, it was found that the usability evaluation contained five aspects of HE, which were categorized as Don't Agree (No problem), namely visibility of system status, a match between system and the real world, consistency and standards, flexibility and efficiency of use, and aesthetic and minimalist design. These five aspects had been evaluated very out to build user satisfaction levels when operating the website.

HE aspect inferential analysis
CFA (confirmatory factor analysis) was an analysis of the correspondence measurement model with survey data and was carried out by using confirmatory factor analysis, which was used to determine items from the HE aspect in running the system without experiencing problems during operation [8]. From the results of the analysis, a model was obtained. Figure 2 shows HE Aspects CFA Test Results. <---Evaluation Heuristic 1,000 Table 13 above showed the analysis of HE aspects inferential. It could be seen that the 10 aspects of HE had causal correlations among variables from the usability factor model on the college website. It could be seen from the magnitude of the P value in each HE was smaller than alpha level of 95%, so it meant that it had an influence on the level of usability.

Analytical network process (ANP) Method in usability evaluation
The ANP method was able to interpret the level of importance from various sources while still paying attention to the interaction between the criteria and the alternatives specified. While in the Analytic Network Process (ANP), it was used to derive the composite priority ratio from the individual ratio scale, which reflected the relative measurement of the influence of the interacting elements with respect to control criteria [9]. The ANP network had a level of objectives, criteria, and sub-criteria, each of which had elements. Figure 3 was the ANP method in determining the HE aspects to find out which aspects were already running well and "no problem" were considered the most important. ANP was used to select the results of usability evaluation on five aspects of HE, which were categorized as Don't Agree Visibility of system status, match between system and the real world, Consistency and standards, flexibility and efficiency of use, and Aesthetic and minimalist design were considered the most important ones. It could be seen that each alternative could depend on each node as well as the hierarchy but also depend on each alternative. In addition, feedback increased the priority that was derived form judgment and made the reduction much more accurate. In the ANP analysis, the prediction of the important level from the HE aspect was considered appropriate in the evaluation of usability assessments. Interactive Design 0,064 3 10 The correct color, font, and information settings 0,008 9 The limit matrix as the selection of the best alternative to the value of these elements was in accordance with the ANP model created. The alternative with the highest global priority was the best alternative. The criterion with the highest score was "latest information" as the top priority. Renewed information was an important factor in a website because basically the website was a medium for disseminating information. The second priority of the system was "easy to run website operations". Easy in this case was easy to absorb all the information obtained, the menus, the content provided was easy for the user to run. The third priority was "interactive design", which was the design form of the website to attract users. The design itself was very broad both from the concept, form, presentation of content, presentation of menus, presentation of writing, etc. Figure 5 shows all cluster alternative priority results:  Furthermore, here were the results of the usability priority test in the HE cluster (limit matrix), table 15. shows the results of the limit matrix on each usability evaluation criterion could be seen from the five selected HE aspects. Flexibility and Efficiency of use 0.033 5 5 Aesthetic and minimalist design 0.155 4 These five aspects were basically aspects that were considered to be not problematic by the user, then the Analytical Network Process (ANP) method helped further needs in determining the best criteria weight. The results of the analysis above were that the priority results that were considered the best of the HE1: Visibility of system status with a weight of 0.336 and HE2: Match between system and the real world with a weight of 0.140 and H4: Consistency and standards with a weight of 0.131 and HE7: Flexibility and efficiency of use with a weight 0.131, and HE8: Aesthetic and minimalist design with a weight of 0.143. Figure 5 Show HE clusters alternative priority results HE. Usability was a perspective of non-functional needs in its use which had the most important benchmarks and the most fundamental factor on the website. This was because this perspective could determine the success or failure of the website when the website had been released and used by its users [10].

Conclusion
Based on the results of research the evaluation of the usability of HE by the user, it was found that the usability evaluation had five aspects of HE which were categorized as Don't Agree (no problem), namely Visibility of system status, Match between system and the real world, Consistency and standards, Flexibility and Efficiency of use, and Aesthetic and minimalist design. The weight of the best criteria produced by ANP is expected to be a guideline for universities to compete. In the future, the research evaluation should be carried out using a Focus Group Discussion (FGD) system so that evaluators can interact and discuss with each other so that the evaluation can be more comprehensive and effective.