Ranking Consistency Analysis and Evaluation of Multiple Attribute Decision Making Method

In the actual decision-making process, different multiple attribute decision making methods used for the same decision-making problem may get different ranking results. This inconsistent ranking can easily lead to decision-making risks. This paper simulates the ranking consistency of six typical multiple attribute decision making methods, including the ranking consistency indexes, consistency probability, the overall consistency of the decision-making methods, and the influence of attribute number and attribute weight on ranking consistency. The results show that various methods have different ranking consistency indexes; the overall consistency of decision-making methods increases with the decrease of the number of attributes; under the condition of equal attribute weights, the decision-making methods have generally better ranking consistency. The research results have reference value for the selection of multiple attribute decision making methods and attribute optimization.


Introduction
Multiple Attribute Decision Making (MADM) is a decision-making process that uses multiple attributes to rank alternatives. MADM technology is widely used in many fields such as economy, management and engineering design. However, for the same decision-making problem, different MADM methods may get different ranking results [1]. This phenomenon is called "ranking inconsistency". Inconsistent ranking of MADM methods directly affects the rationality and scientificity of decision-making.
In order to reduce the risk of ranking inconsistency, literature [2] deals with the inconsistency of results by nonlinear programming model with constraints. Reference [3] uses the preference information of decision makers to modify the ranking results and improve the ranking consistency. In reference [4], hybrid MADM method is used to avoid the problem of inconsistent ranking. Most of the studies comprehensively use a variety of MADM methods to reduce the risk of decision-making caused by inconsistent ranking.
In addition, the MADM method also has the phenomenon of rank reversal [5]. The two kinds of problems use the same measurement method when judging the similarity of ranking.
Firstly, this paper determines the measurement method of ranking consistency, gives the consistency index and defines the consistency probability; then, six typical MADM methods are selected to simulate the ranking consistency. Finally, the correctness of the method is verified by an example. The results show that different MADM methods have different performance in ranking inconsistency.

Ranking Consistency
In practice, different MADM methods for the same MADM problem may lead to different ranking results, which will lead to inconsistent ranking and increase the risk of decision-making errors. For example, for a MADM problem, M and N decision methods are used respectively, M R and N R two ranking vectors are obtained. In order to measure the ranking consistency of MADM method, we need to consider the similarity of ranking vectors and the occurrence probability of ranking inconsistency.

Consistency Indexes
The similarity of ranking vectors can be used to measure the consistency of ranking, which is a problem of calculating rank correlation coefficient. There are ten main measures of ranking consistency. Spearman, Kendall and Rank-Biased Overlap (RBO) are selected as ranking consistency indexes in this paper. In order to ensure the same range of variation of the three indexes, the scale of RBO index is adjusted to make its range of variation be [- If L different MADM methods are used to solve the same problem, a ranking consistency matrix can be established for the corresponding problem. The ranking consistency matrix is expressed as: Matrix elements are equivalent to the pairwise ranking consistency index of MADM method.
The average ranking consistency index of the i th MADM method is: This index can be used to measure the overall ranking consistency of MADM method.

Consistency Probability
In order to express the consistency of ranking conveniently, this paper divides the value range of consistency index into five grades. The division principle is shown in Table 1.
The consistency probability is expressed as:

Selection and Implementation of MADM Method
In this paper, six typical MADM methods are selected for ranking consistency analysis and evaluation, as shown in Table 2. This paper focuses on two generation strategies, equal weight and linear weight, to study the consistency of MADM ranking under the condition of given weight, and analyze the influence of evaluation attribute weight on ranking consistency. This paper designs and implements six typical MADM methods listed in Table 2. Each randomly generates a decision matrix and attribute weight, which is processed by different MADM, thus obtaining six groups of ranking vectors. The total number of ranking vectors generated in the simulation process is 6 × 10000 = 60000 groups.

Construction of the Ranking Consistency Matrix
Three measures, Spearman, Kendall and RBO, are used to evaluate the consistency index of the ranking vectors, and a ranking consistency matrix is constructed. The dimension of the ranking consistency matrix is (6×6). According to equation (5), the average ranking consistency index of MADM method is further calculated.

Probability Calculation of Ranking Consistency
Given the threshold value , is obtained by statistical analysis from Equation (6). In this paper, the probability cumulative distribution curve is used to express the distribution characteristics of consistency index. The curve shape fully illustrates the performance of each MADM method.
The analysis and evaluation method includes four processes, which are initialization, MADM implementation, ranking and consistency analysis and evaluation. The initialization part mainly realizes the setting of simulation times, attribute number, alternative scheme number, attribute weight and other parameters. The implementation of MADM method mainly realizes six kinds of decisionmaking processes of MADM. The ranking process is to arrange the processing results of MADM method in descending order to form ranking vectors. Consistency analysis realizes the calculation, probability statistics and result evaluation of three consistency indexes.

Example Description
The number of alternatives n=7, the number of attributes m=30, and the elements of the decision matrix obey the uniform distribution in [0,1] interval and are randomly generated. In the example, the simulation times are set to N=10000, and the attribute weights are allocated linearly.

Consistency Index Test
The results of an example show that Spearman, Kendall and RBO can measure the ranking consistency of MADM method, and have the same change rule, as shown in Figure 1.
As can be seen from Figure 1, the ranking consistency index of SAW and PROMETHEE is 1, which means that the ranking results are completely consistent. ELECTRE has poor ranking consistency with any other method.

MADM Ranking Consistency Analysis
In order to measure the consistency of a single MADM method, this paper calculates the average value of ranking consistency according to equation (5). The average consistency of the six MADM methods is shown in Figure 2.
As can be seen from Figure 2, the order of ranking consistency performance from good to bad is SAW, PROMETHEE, GRA, TOPSIS, VIKOR and ELECTRE.

Consistency Probability Evaluation
In view of the same variation rule of consistency index (Figure 1), only Spearman index is used to express consistency in this section. Figure 3 shows the change of Spearman consistency index of SAW and ELECTRE MADM methods. The abscissa of Figure 3 shows 100 randomly generated decision matrix scenarios, and the ordinate shows the change of SAW-ELECTRE ranking consistency index. The average value of the index is 0.345, the maximum value is 0.964, and the minimum value is-0.   The cumulative distribution curves of SAW-TOPSIS, SAW-GRA, SAW-ELECTRE and SAW-VIKOR are shown in Figure 4.
As can be seen from Figure 4, GRA has better ranking consistency and ELECTRE is poor.

Effect of Attribute Quantity on Consistency
The number of attributes m is reduced from 30 to 15, and other conditions remain unchanged. The average ranking consistency index changes as shown in Table 3. As can be seen from Table 3, the reduction of the number of attributes will generally improve the average ranking consistency index.

Influence of Attribute Weights on Consistency
The number of attributes m=30 remains unchanged, the attribute weight changes from linear distribution to equal weight distribution, and other conditions remain unchanged. The index changes are shown in Table 4. As can be seen from Table 4, the average ranking consistency of MADM method can be significantly improved by adopting equal weight allocation strategy for attributes.

Conclusion
In this paper, the ranking consistency of six typical MADM methods is studied by numerical simulation. Among them, SAW and PROMETHEE have the best ranking consistency, GRA takes the second place, TOPSIS is in the middle, ELECTRE and VIKOR have poor performance. In addition, reducing the number of attributes can improve the consistency level, and equal attribute weights can significantly improve the consistency, among which VIKOR has the most obvious performance improvement.
The above analysis and evaluation conclusions have reference value for the selection of MADM methods and attribute optimization. In the next step, we will focus on the ranking consistency of MADM fuzzy extension method.