Occupational health and safety: Designing and building with MACBETH a value risk-matrix for evaluating health and safety risks

Risk matrices (RMs) are commonly used to evaluate health and safety risks. Nonetheless, they violate some theoretical principles that compromise their feasibility and use. This study describes how multiple criteria decision analysis methods have been used to improve the design and the deployment of RMs to evaluate health and safety risks at the Occupational Health and Safety Unit (OHSU) of the Regional Health Administration of Lisbon and Tagus Valley. ‘Value risk-matrices’ (VRMs) are built with the MACBETH approach in four modelling steps: a) structuring risk impacts, involving the construction of descriptors of impact that link risk events with health impacts and are informed by scientific evidence; b) generating a value measurement scale of risk impacts, by applying the MACBETH-Choquet procedure; c) building a system for eliciting subjective probabilities that makes use of a numerical probability scale that was constructed with MACBETH qualitative judgments on likelihood; d) and defining a classification colouring scheme for the VRM. A VRM built with OHSU members was implemented in a decision support system which will be used by OHSU members to evaluate health and safety risks and to identify risk mitigation actions.


Introduction
Occupational health and safety (OH&S) is "the science of the anticipation, recognition, evaluation and control of hazards arising in or from the workplace that could impair the health and well-being of workers" [1] (p. viii). Proper management of OH&S risks has been recognized as a social concern topic and as a workers' right by many organizations, including the International Labour Organization, the World Health Organization, the European Commission and national governments, in line with promoting a healthy and productive labour force [2].
A lot of research has been devoted to the development of tools for risk assessment, with risk matrices (RMs) being one of the tools most widely used to evaluate risks in general, and OH&S risks in particular [3][4][5], as they are easy to handle, demand for limited expertise, and are enabled by several popular software packages. Nonetheless, several studies point out that RMs violate theoretical principles that compromise their feasibility and use [6][7][8][9].
The Occupational Health and Safety Unit (OHSU) of the Regional Health Administration of Lisbon and Tagus Valley (RHA LVT) is responsible for evaluating OH&S risks for all the individuals working in primary care centres and in administrative offices of the RHA LVT. Although OHSU team members have been using RMs for OH&S risk assessment [5], they have recognized several difficulties in their use and asked for our help to improve their risk evaluation process.
This study reports how tools from Multiple Criteria and Decision Analysis (MCDA) have been used to design a Value Risk-Matrix (VRM) to be used by the OHSU to evaluate OH&S risks, as well as briefly describes a decision support system (DSS) that was created to enable the use of that VRM. Several components of the development of VRMs make a contribution to literature, for instance, the modelling of risk impact dimensions with the MACBETH-Choquet procedure (the first application of this procedure) and the modelling of probabilities with MACBETH.
This article is organized in five sections. Section 2 briefly introduces RMs, their problems and challenges. Section 3 describes what a VRM is, and the activities required for its design at the OHSU. Section 4 briefly describes how each activity was applied with OHSU members and the DSS that was created to assist OHSU. Section 5 presents concluding remarks.

Literature review
In the European Union several people die per year as a consequence of occupational accidents and diseases, with most of these being avoidable [10]. The first step in preventing them is risk assessment [10], which provides decision makers (DMs) with an improved understanding of risks that could threaten their goals and on the extent to which mitigation actions need to be implemented [11].
Research and regulation of OH&S are a relatively recent phenomenon [12]. Within common frameworks of management of OH&S risks, several guidelines [4], research studies [13][14][15] and international standards [11] advocate the use of RMs. RMs are used and recommended to evaluate and mitigate risks in many health care contexts, such as by the National Patient Safety Agency from the United Kingdom [4]. In this tool, individual sources of risk are typically characterized in terms of risk probability and impact. Table 1 illustrates the RM used by the OHSU of the RHA LVT to evaluate OH&S risks [5], which is similar to many other RMs. That RM includes two 4-level rating scales in each probability and impact component. The risk score is given by crossing both information, i.e. by multiplying its probability and impact. This is complemented by information on risk criticality, as depicted by the traffic light system that colors each combination of probability and impact in the RM (corresponding to the 5 risk categories "very low", "low", "moderate", "high" and "very high") [5]. Table 1. Example of a quantitative RM (adapted from [5]), with five risk categories (from light grey to dark grey) "very low", "low", "moderate", "high" and "very high".

Very low 2. Low 3. High
Besides these problems that have been reported in literature, judgmental dependencies between risk impacts in different dimensions may occur, and therefore modelling should identify and account for these value interdependencies (note that value interdependencies differ from cases of statistical or physical correlations, as they depend on the DM perspectives and values). Situations of preference interdependencies have been found in multiple contexts, and have been recognised as a major challenge in MCDA literature [16], with several studies exploring the use of Choquet integral (CI) operators to model these interdependencies, as these operators demand for weaker conditions in comparison to other models, such as the multilinear one that demands for the respect of weak difference independence conditions [17]. There is thus scope for improving tools to assist the evaluation of OH&S risks, using the same framework as RMs, but overcoming the methodological issues above identified and addressing the challenge of identifying and modelling value interdependencies in risk impacts.

Methodological approach
Our work with the OHSU started by a set of interviews that identified the difficulties felt by members of the unit in the process of evaluating OH&S risks, and their perceived challenges. Summarily, these interviews have shown: difficulties in identification and measurement of risk impacts due to the lack of technical instruments (e.g., for measuring noise levels), as well as linking risk sources with their consequences due lack of access to systematic evidence; great difficulties in assessing probabilities given a lack of a registry and of a summary of evidence in the area; and disagreement with some of the values implicit in the RM in use (e.g., third-degree burns and a death are similarly rated).
In order to correct problems from RMs, we propose using a VRM -as shown in Figure 1, a VRM is similar to the RM as it combines the two risk components, impact and probability, to define a few risk categories (e.g., from grade I to grade V). However, (discrete) risk impacts present in common RMs -as in Table 1 -are replaced by (continuous) multicriteria value scores, (discrete) probabilities by a (continuous) probability scale by adopting specific MCDA tools, such that discontinuities between adjacent categories (as in Table 1) are avoided and the principle of translation is respected. The value and probability scores of a risk determine its position in the VRM and users can easily observe whether that risk is seen as critical to the organization and may require specific managerial attention [9]. Building the VRM to the OHSU followed a socio-technical approach and a sequence of several interconnected activities, as depicted in Figure 2. The social component consisted in the development of these activities within various meetings, workshops and decision conferences with the DMs, which in our case were the professional members of the OHSU with different backgrounds and distinct viewpoints regarding the evaluation of OH&S risks (namely, doctors, nurses, a sanitary engineer and OH&S risk technicians). The social component enabled the development of a requisite VRM (with model requisiteness as defined by [18]) that captured the perspectives of the team members. On the technical side, developing the VRM included: (1) the structuring of risk impacts, (2) the construction of the value scale for risk impacts with MACBETH-Choquet procedure, (3) the construction of a system of rules for defining subjective probabilities (using reference probabilities constructed with the MACBETH approach), (4) the design of risk categories, and (5) building a DSS to enable the use of the VRM for OH&S risk assessment. An explanation on what consisted each of these activities, and how it was developed with the DMs will be provided in the next section. We have used the MACBETH approach to build the VRM as it is recognised as being a simple and transparent MCDA tool in modelling complex multidimensional problems, based only on non-numerical judgments about differences in attractiveness between evaluation elements, avoiding the "difficulty" [17] of expressing value judgments numerically [9]. Although the MACBETH approach has been previously used in real contexts, this is the first time that it was applied the MACBETH-Choquet procedure and that MACBETH was adapted and applied for probability assessment (requiring a change in its formulation and questioning protocol).  Step S.I: Nomenclature OHSU's guidelines for using RMs and all the reports based on it were analyzed. This allowed us to detect incoherencies and inconsistencies, namely on the nomenclature in use, and given the ambiguity in several risk terms, the terms in use were discussed and defined with members of the OHSU. Accordingly, the risk terms defined in Table 2 were adopted.
Step S.II: How to identify and measure risk sources and their consequences? Given the difficulties reported by the DMs on identifying and measuring the expression levels of risk sources and their consequences due to the lack of technical tools (e.g. for measuring noise levels), we made an intensive literature review to build qualitative (or quantitative or pictorial) descriptors of impact that link different risk sources and expression levels with health consequences. Table 3 exemplifies the descriptor for the risk source "noise", which links, for example, the noise level present in a waiting room with quantitative levels, according to literature (for instance, making explicit the exposure limit imposed by legislation, 87 dB [19]). Descriptors were similarly built for other risk sources. Table 2. Nomenclature proposed and validated by the DMs.

Terms Description Event
Situation or phenomenon in which there is uncertainty in its occurrence.

Risk source (or hazard)
Source of potential adverse effects or a situation capable of causing adverse effects in terms of health (injuries or illnesses) [7].

Expression levels of the risk source
A risk source can be expressed in different ways that we called expression levels.

Measure of the expression levels related to a risk source
Identification and awareness/measurement of qualitative or quantitative or pictorial expression levels.

Risk
"A risk is a random event that may possibly occur and, if it did occur, would have a negative impact on the goals of the organization" [20] (p.3). Typically, it is characterized by two components: its probability of occurrence and impacts [10].

Risk typology
Risk nature or risk classification in five types enshrined in the Decree-Law no. 441/91 and in the Law no. 99/2003.

Consequence
Outcome of a risk [11] (bodily injury, functional disorder or disease) affecting the objectives of an organization. Impact Severity [20] or effects of the consequences' occurrence. Step S.III: Identification of impact dimensions This step consisted on identifying the impact dimensions to evaluate risk impacts. Due to the recursive nature of our proposal methodology, the impact dimensions were iterated until the three dimensions -"employee's health" (EH), "work capability" (WC) and "absenteeism" (A) -were identified, as depicted by the value tree in Figure 3. The DMs were concerned about OH&S risk impacts for the health of the employee and for the employers (if the employee returns, and with which capabilities). Step S.IV: Impact scales construction This step consisted in building impact scales for each dimension. For the "employee's health" dimension, the DMs were concerned about the impact on the employee's health. This led to the construction of a continuous impact scale based on the disability adjusted life year measure, which varies from the status quo (corresponding to no impact, i.e., 0 Years of Healthy Life Lost (YHLL)) to the worst level (corresponding to the death of the employee that implies 34 YHLLs). For the "work capability" dimension, a 5-level qualitative impact scale was built by adapting the legal nomenclature [23], namely: "null disability", "recoverable disability", "irrecoverable partial disability with return to work", "irrecoverable partial disability with no return to work" and "irrecoverable disability". Regarding the "absenteeism" dimension, a continuous impact was constructed based on the number of days of absence from work.
Step S.V: How to estimate the consequences' impact in each dimension? Given the DMs difficulty in estimating impacts of consequences due to lack of knowledge, an intensive literature review was done to build tables linking the consequences of different risks to the impact scales. For instance, according to literature, an amputation of a finger was shown to correspond to 3.5 YHLL [24,25], to an irrecoverable partial disability and to 26 days of absence from work [26].

Impact value measurement with the MACBETH-Choquet procedure
Building a value measurement scale for risk impacts requires the conversion of impact into value. This activity was carried out by using the MACBETH-Choquet procedure that allows for modelling interdependencies between impact dimensions. This approach combines MACBETH with the CI, through the use of a MACBETH global matrix filled with judgments of global attractiveness between risk impacts, being the basis for determining the CI parameters (for further details, consult [27]). This value measurement technique was chosen as we found out that there was preferential dependence between some pairs of dimensions (for instance, work capability assessment judgmentally depended upon employee's health, according to the DMs' preferences), and this work allowed for using that procedure in a real context for the first time. The first step for applying the MACBETH-Choquet procedure was to define all the feasible combinations of different impact levels across all the three dimensions -in our example, 36 combinations of impacts were considered. In particular, those combinations vary from the status quo corresponding to no impact (0 YHLL, null disability and 0 days of Absence from Work (AW)) to the worst level corresponding to states equivalent to death (34 YHLL, irrecoverable disability and 18 years of AW).
MACBETH-based elicitation was then used with the DMs to quantify the relative attractiveness of reverting each set of impacts through a qualitative pairwise comparison questioning mode, supported by the Microsoft Office PowerPoint 2007 (MOP 07). This questioning protocol was built to be applied in a sequential and interactive manner (using the ActiveX textbox tool). To illustrate the protocol, the DMs were asked about the attractiveness of fully reversing (or cancelling out) the following combination of impacts: 34 HLL, irrecoverable disability and 18 years of AW, in the MACBETH categorical scale "null" (C 0 ), "very weak" (C 1 ), "weak" (C 2 ), "moderate" (C 3 ), "strong" (C 4 ), "very strong" (C 5 ) or "extreme" (C 6 ) -that are represented in the matrix of Figure 4(a) by the respective indices "1" to "6" for the sake of saving space. The DMs answered that reversing this risk impact was extremely attractive, with this being then recorded in the MACBETH matrix of qualitative judgements presented in Figure 4. The same type of questioning protocol was applied for all the other combinations of risk impacts (as explained in [28]) and the answers were recorded in the M-MACBETH software (Figure 4). The DMs revised some judgments when inconsistencies between answers were detected, and after discussion, a global (numerical) score was validated (as shown in Figure 4(b)).
Regarding the CI parameters, the Shapley and interaction parameters are key concepts for the understanding and analysis of interdependencies between dimensions. According to Grabisch (1997), the Shapley value can be defined as an average value of the contribution of dimension i alone considering different baselines in the other dimensions. With regard to the interaction parameter between 2 dimensions for instance, it can be defined as an average of the added value given by putting together the 2 dimensions together, all the baselines being considered. The interaction parameters I ij range in the interval [-1,1] with: (a) I ij > 0 for a synergistic behaviour between dimensions i and j, (b) I ij < 0 for an antagonistic behaviour between dimensions i and j, and (c) I ij = 0 when there is no interaction between dimensions i and j. The mathematical representations of the Shapley and the interaction parameters are given by (1) and (2), respectively.

Subjective probabilities through a system of rules
Following difficulties felt by the OHSU group in selecting probabilities for risks (historical information on risks was not available), as well as technical and behavioural inconsistencies in associating numerical probabilities to verbal expressions pointed out in literature [29], we have designed a system to help the group to construct subjective probabilities. This system had as underlying assumptions: probabilities elicitation in a verbal formal [30]; probability can be seen as "a personal judgment of the likelihood of a proposition or event rather than an objective fact" [31], with a constructive view of probabilities [17] being useful in the context. Building a system to construct subjective probabilities involved: a selection of a wide range of representative risk events that covers all the likelihood scale, according to literature and to experience from the members of the OHSU; the construction of a numerical probability scale for these representative risk events, using the MACBETH approach adapted for probability assessment; and then a system of rules in which, for a new risk, the OHSU will find the most similar representative event, and then will re-adjust the numerical probability according to pros and cons, i.e. by aspects that can contribute for a higher/lower likelihood (for instance, if employees have attended a course on OH&S risks, the likelihood of some risk events occurring will be lower).
After a set of representative risks was selected, the MACBETH questioning mode was adapted for the probability assessment context, with the OHSU group being asked for verbal judgments about differences of likelihood between risk events. This approach is grounded on (a) the definition of two reference levels, a certain event and an impossible event which correspond to 1 and 0, respectively; (b) a qualitative pairwise comparisons questioning protocol based on questions-type such as "How likely is risk to occur?", or said in another way "which is the difference in likelihood between and an impossible event?" (with being fully specified as a risk event). This probability assessment consisted on a conditional assessment of subjective probabilities, as it required a certain priori knowledge [32] about the risk event (for instance, with "effectiveness of security measures already implemented" and "exposure" being made explicit). Additional questions were asked to distinguish between which events were more probable than others (for further details consult [9]). After discussion with the OHSU members, the consistent MACBETH matrix present in Figure 5 was obtained, and the final numerical scale of subjective probabilities present in Figure 5(b) was validated.
Finally, a generic system with specific rules for the assessment of probability of a new risk was designed: for a new risk, the user of the VRM should find a reference event with each to compare the new risk, and then adjust the probability according to the pros and cons that may increase or decrease probability (that is, specific factors that will contribute for an increase (or decrease) in the probability, such as whether the employee received training to manage the respective risk; details are available in [33]).

Risk classification and value risk-matrix
The previous activities enabled the determination of the impact value score and probability of each risk event. Constructing a VRM to be used for evaluating OH&S risks and for analyzing the impact of risk mitigation measures required the identification of risk categories. After discussion with the OHSU group, the five categories depicted in Table 4 (from grade I to grade V) were associated with ranges of combination of the two risk components (impact value score and probability). Then, the OHSU members were asked to classify a wide range of risk event (with different locations in the VRM) in the five categories. With these answers, we were able to adjust iso-curves that separate adjacent categories with MATLAB, by using a cubic spline interpolant. This step also provided a validation of the VRM, as members of the OHSU had to analyse and discuss the combinations of subjective probability, impact value and risk category for a wide range of risk sources. Based on the impact value scale, the subjective probability scale and on the identification of risk categories, a VRM was designed (see Figure 1). This framework constitutes the basis for the OHSU evaluating OH&S risks and identifying the need for risk mitigation actions.

Decision support tool
Once designed the VRM and determined the impact value scores and the probabilities for a representative set of risks, we started constructing a DSS to facilitate the use of the VRM by the OHSU. This DSS, that we designate as "SAAR SST " (Portuguese acronym for "Sistema de Apoio à Avaliação Riscos em Segurança e Saúde no Trabalho", see Figure 6), works as a Microsoft Excel and Word add-in that contains a personalized code in Visual Basic for Applications. It integrates the functionality of creating a global database of risks and additional information from literature to inform the evaluation of risks. The configuration of the DSS was discussed in detail with the OHSU group so that it contained all the important functionalities and that it was easy to use.

Discussion
Departing from an assessment of the problems felt by the OHSU in evaluating OH&S risks and from the RMs methodological problems identified in the literature, this study describes how MCDA concepts and tools were designed and applied -within an interactive learning process -to build a DSS based on a VRM to evaluate OH&S risks. On the social viewpoint, the OHSU members have found the process for developing the VRM very useful, as it enables and stimulated an identification of problems, a clarification of concepts, and a stimulating learning process. On the technical viewpoint, the VRM corresponds to a flexible and versatile approach that fits in a proper manner the OHSU preferences and overcomes several problems from classical RMs. Therefore, we believe that VRM is a promising methodology that can be applied in many other risk evaluation contexts.
Several aspects are worth of further research. First, VRMs should be developed to other contexts and the VRM for evaluating OH&S risks can be tested with other DMs working in the area. Secondly, further research is needed to prioritize and select mitigation actions in the context of limited resources. We suggest the use of the recent portfolio decision analysis component of the MACBETH DSS, as explained in [9]. Thirdly, it is worth exploring uncertainties associated with the measurement and evaluation of OH&S risks and of mitigation actions.