Methods of coordinating fuzzy expert evaluations in the design of metrological support means

The article explores methods of coordinating expert judgments in the design of technical measuring instruments. A methodology for analyzing fuzzy matrices of pairwise comparisons of the effectiveness of preliminary designs of metrological support means is proposed. It includes defuzzification of pairwise comparison matrices; finding a median pairwise comparison matrix; determining fuzzy mean values for each element, and forming fuzzy weights for the compared means by individual levels of the membership function values. The procedure for finding the final membership function of metrological support tools is formalized to minimize the deviations of the fuzzy paired comparison matrices of expert opinions from the final matrix of these comparisons. Methods for optimizing the final ranking of alternative projects based on fuzzy group evaluations were investigated and developed. A fuzzy modification of the Condorcet principle is developed for the case of fuzzy dominance of alternatives. These methods of expert evaluations serve the purpose of increasing the efficiency of development and creation of metrological support means.


Introduction
When creating metrological equipment, the task of choosing the most acceptable option out of several design solutions is to be solved. At the same time, ensuring the economic efficiency of metrological instrumentation projects does not mean achieving the required levels of quality. In the early stages of designing new equipment, the expert's opinion is an essential factor to consider. By involving several experts, it is possible to increase the reliability and validity of the decisions made. The issues of coordinating the opinions of several experts are quite actively studied by scientists in Russia and abroad in the modern decision-making theory [1]. In such problems, group decision-making methods are usually used [2]. There are two methods for group decision-making: searching for consensus and the group members' opinion aggregation into some final assessments [3]. The widely used Delphi method essentially combines these two approaches in an iterative procedure. It is often proposed to use "classical" methods of multicriteria evaluation of alternatives to aggregate expert opinions (Pareto principle, weighted sum, utility theory, preference functions, etc.) [4]. A. B. Petrovsky proposed to use the methodology of verbal decision analysis and the multiset theory for group decision-making [5]. However, the expert often cannot unambiguously give a clear assessment of the object of examination and sometimes finds it difficult to determine the relationship of preference between the two objects clearly.
Interval assessments of specific characteristics of the objects of expertise are often more natural. Valuable information from the expert can evaluate the degree of his confidence that the assessment takes a particular value. The mechanism of the fuzzy set theory is well suited for such estimates [6]. The use of fuzzy set theory in decision-making has been increasingly popular in recent decades. It is worth noting the works of A. N. Borisov Despite the existence of these and several other works, the problem under consideration has not yet received its comprehensive solution. Therefore, the development of group methods of decision-making in the case of fuzzy weights, fuzzy areas of preference, and fuzzy expert assessments, and the analysis of the possibility of adapting the use of "classical" approaches to group decision-making to fuzzy estimates, are of great interest. Also, the bulk of the available works on the subject do not present studies of the computational efficiency of decision support procedures in a fuzzy information environment, although, in such an environment, significantly more computational resources are required compared to the "classical" approaches [11; 12].

Fuzzy pairwise comparison method
In the fuzzy matrix of pairwise comparisons, the expert sets the membership function for each pair of objects ( ). For classical multiplicative levels of preferences from the set = {1/9, 1/8, 1/ 7 … 1, 2, 3, … 9}, it sets the degree of confidence of the expert in this assessment. The range of values ( ) is the interval [0,1]. At the same time, the property of inverse symmetry is preserved: The expert may not specify an affiliation for all elements of X. The values at the intermediate points are determined by linear approximation. The expert can specify the most possible value and the interval beyond which the membership is zero in the simplest case. This case corresponds to the membership function of the triangular form.
The defuzzification method of paired comparison matrices is the simplest from the point of view of algorithmic implementation. The center of gravity method carries out defuzzification, and then it is permissible to use the usual methods for paired comparison matricesfinding the maximum eigenvalue and the corresponding eigenvector. The resulting rankings can be processed by "classical" methods of matching expert judgments since there are no fuzzy numbers. The main disadvantage of this approach is the loss of important information about the expert's degree of confidence in the assigned assessments.
To solve the problem under consideration, the authors assume that the fuzzy pair comparisons for expert k are presented in the form of a matrix: and as a metric for the distance between the relations for two experts, the Manhattan distance is used: For fuzzy matrices of pairwise comparisons processing, the search for the median matrix of pairwise comparisons as an optimization problem is formulated, in which the sum of the distances from all relations to the mean sets its target function: This statement of the problem allows searching for such a membership function that it is "equidistant" from all other membership functions. At the same time, the restriction on the inverse symmetry property must be maintained. Solving such a problem is quite a resource-intensive procedure. Its computational complexity depends exponentially on the number of segments of the polyline of which each of the membership functions consists. The resulting optimization problem belongs to linear programming and is solved by the classical simplex method. However, this approach does not guarantee the resulting matrix's overall consistency and the absence of non-transitive judgments. Adding a component associated with the paired comparison matrix's consistency index to the objective function makes the optimization problem curvilinear.
The method of finding fuzzy averages for each element of the paired comparison matrix is simple, and the result of its work is relatively easy to interpret. Using the principle of generalization of fuzzy numbers, the matrices are added, and each element is divided by the number of summands. This method is not difficult to implement. The degree of uncertainty of the judgments of all experts in the matrix remains at an acceptable level. However, the most possible value (where ( )=1) may not correspond to the opinion of a significant proportion of experts.
A special algorithm has been developed to generate fuzzy weights for the compared objects (alternatives/criteria) for individual levels of the membership function values. The input of the algorithm is fuzzy matrices of pairwise comparisons. The generalized steps of the algorithm: 1) Cycle over all membership levels m from 0 to 1 with a given small step d (in experiments d=0.01).
2) Determining the maximum value for all elements of all pairwise comparison matrices for which the membership level is m. As a result of this step, "normal" ("clear") paired comparison matrices are obtained.
3) Search for the obtained "normal" matrices of the maximum eigenvalue and the eigenvector corresponding to it. The elements of the eigenvector are the ranks of the evaluated objects with confidence degree m.
4) The median is searched for among the obtained eigenvectors. As a result of this step, one point is obtained on the diagram of the final ranks of objects' membership function. 5) Return to the beginning of the loop until m is equal to 1. The output of the algorithm results in fuzzy object ranks. This method is original. Its novelty lies in the sequential determination of integral ranks by the levels of the membership function. It has polynomial complexity and immediately gets fuzzy ranks of alternatives while preserving information about expert judgments' uncertainty.
The simplest way to aggregate expert opinions is to use a fuzzy weighted sum, where each expert is assigned a fuzzy weight that indicates confidence in the expert's competence. Another way of aggregation is to solve an optimization problem in which such a fuzzy ranking is searched for so that the maximum deviation of its membership function from the membership functions of the alternatives obtained by the experts is minimal. This aggregation method in practice allowed obtaining a ranking that does not contradict the opinion of a significant proportion of experts. The resulting problem in the original formulation is not linear, but entering additional binary and continuous variables is reduced to mixed linear integer programming.

The ranking alternatives method based on a fuzzy modification of the Condorcet principle
To aggregate expert opinions, consider a fuzzy modification of the Condorcet principle for the case of fuzzy dominance of alternatives. In the case of fuzzy ranked alternatives, the dominance ratio is determined by two values. Let a be an estimate of the possibility that alternative A is better or equivalent to B. Fuzzy sets do not exclude the inverse situation, that alternative B is better or equivalent to A, denote the numerical estimate of such a possibility as b. Fuzzy ranks can determine the values of a and b for any alternatives by comparing fuzzy numbers based on the generalization principle.
In the Condorcet principle's classical version, each expert unambiguously votes for the first or second alternative. In this fuzzy case, the estimates' ambiguity is proposed to be determined by setting the hlevels for the rank membership function. If h is less than a, then it is claimed that A is better or equivalent to B at the h-level. Similarly, if h is less than b, then it is claimed that B is better or equivalent to A at the h-level. Thus, at each h-level, the number of expert votes for the dominance of A and B is determined. The decision is made by a majority vote. The characteristic points where the voting results may change are the points from the set of all values of a and b. As a result, the membership function for determining the fuzzy dominance of alternatives is obtained, taking into account the analysis of all experts' opinions. The proposed method allowed formulating the following algorithm for fuzzy ranking: The algorithm input is the fuzzy ranks of the set of alternatives obtained by a group of experts. The output of the algorithm is a fuzzy integral ranking of all alternatives. The software has been developed in Python that implements the proposed methods for processing fuzzy expert judgments. A special FuzzyNumber class has been developed to implement operations on fuzzy numbers. The PySCIPOpt library and the SCIP Optimization Suite optimizer are connected to solve optimization problems.

Conclusion
The developed methods' results allow determining the qualitative ranks of alternatives for a given degree of confidence and making decisions based on the opinion of a group of experts, expressed in fuzzy matrices of paired comparisons. As a result of the conducted research, the procedure for searching for the final membership function of alternatives is formalized, and a fuzzy modification of the Condorcet principle is developed for the case of fuzzy dominance of alternatives.
Based on the agreed qualitative judgments of experts, formalized in terms of the theory of fuzzy sets, the designer of metrological equipment can make optimal decisions that increase metrological support efficiency.