Fault-tolerant Control of a Cyber-physical System

Cyber-physical systems represent a new emerging field in automatic control. The fault system is a key component, because modern, large scale processes must meet high standards of performance, reliability and safety. Fault propagation in large scale chemical processes can lead to loss of production, energy, raw materials and even environmental hazard. The present paper develops a multi-agent fault-tolerant control architecture using robust fractional order controllers for a (13C) cryogenic separation column cascade. The JADE (Java Agent DEvelopment Framework) platform was used to implement the multi-agent fault tolerant control system while the operational model of the process was implemented in Matlab/SIMULINK environment. MACSimJX (Multiagent Control Using Simulink with Jade Extension) toolbox was used to link the control system and the process model. In order to verify the performance and to prove the feasibility of the proposed control architecture several fault simulation scenarios were performed.


Introduction
Cyber-physical systems represent an interconnection between process computer and physical systems through communication, sensor and actuator technologies. The application domains are rapidly increasing like: medical, automotive, aircraft and even military systems. However, in spite all the advantages introduced by this control approach there is also a key issue: the fault control system [1,2]. Industrial chemical processes, like the ( 13 C) cryogenic isotope separation column cascade are even more difficult to control due to the high complexity and nonlinear dynamics. The research regarding trains of cryogenic isotope separation columns, is scarce, because is a very unique equipment, only few such columns exist in the world. The automation of the isotope separation cascade system is a lot more complicated than in the case of a single separation column making necessary the use of a distributed control system due to many interconnections amongst the columns. Also, distributed control systems use specific real time networks and communication protocols that may introduce variable time delays and even data consistency problems which can lead in extreme cases to the instability of the system making fault control system mandatory. Even if multi-agent systems (MAS) were widely used to study distributed control and collaborative tasks for complex systems there is still the need for fault tolerance in order to increase dependability even in the presence of faults [3,4]. The research community proposed various algorithms and architectures for fault tolerant solutions. In [5] Khalili et all. proposed an adaptive fault-tolerant control (FTC) scheme for a class of nonlinear uncertain multi-agent systems. Bora and Dikenelli introduce a replication-based approach, a centralized self-adaptive fault tolerant approach that exploits a feedback control loop in [6]. The solution proposed in this work is a fault-tolerant distributed control architecture based on fractional order controllers in order to handle the possible system uncertainties exploiting the increased robustness of the fractional order controllers (FOPID) which have two more degrees of freedom [7,8]. This approach was tested successfully by the authors in prior studies in which fractional order controllers were implemented for control of a single isotope separation column [9][10][11]. The only downside of the proposed approach is represented by the increased complexity in FOPID controllers' parameter tuning procedure by using optimization techniques. The fault tolerant architecture concerns mainly the interactions between the subsystems by introducing a supervisor layer that identifies faults and determines the control configuration to keep the closed loop system performances in an acceptable domain. In the present work, fault tolerant multi-agent system FTMAS was developed and implemented in JADE and then it was linked with MATLAB/Simulink, using Multiagent Control Using Simulink with Jade Extension (MACSimJX). The paper is structured into four sections. After the introductory part, section 2 presents the plant characteristics. The third section is focused on the proposed fault tolerant multi-agent system design and implementation. The work ends with concluding remarks presented in section 4.

Plant Characteristics
The ( 13 C) isotope has the most uses in a broad range of industrial fields like medicine, biology, chemical engineering and many more: it can be used as a valuable tracer in biological, chemical and environmental science being very useful in medical diagnostics [10]. The main problem is that the natural abundancy of the ( 13 C) isotope is very low (1.1 at.%) which can be raised by cryogenic distillation of the carbon monoxide. This method is based on the higher isotope effect, [12] and on the isotopes nuclear properties being implemented for the available pilot plant developed at NIRDIMT -Cluj-Napoca, Romania. A single column of approximatively 7m height and having a diameter or about ~16mm was able to raise the ( 13 C) concentration only up to (8)(9)(10) at.%.
The plant considered in this work consists of three ( 13 C) cryogenic isotope separation columns connected in series, in a cascade, Figure 1a using a common condenser (cooled with liquid nitrogen) with condensation spaces for each of the three columns and a common thermal insulation. Each column is an independent complex nuclear process itself being used for the production of ( 13 C) carbon stable isotope. Each column is packed with Heli-Pack stainless steel wire of 1.8x1.8x0.2 mm in order to increase the contact surface area between the two counteracting flows and is placed inside a vacuum jacket because the operating temperature is around -190 o C. The pilot plant is designed in such way that permits the individual operation of the columns as well as operation in series or parallel. However, this technological solution increases the complexity of the control system greatly, the number of variable (monitored or controlled) triples hence a complex control method is needed. To grasp the complexity of the whole process it is worth to mention that start-up procedure of the whole process takes around 3-4 days, and after that the isotope the cascade has to operate for months in order to reach the maximum percentage of ( 13 C). From a fault tolerant control system point of view, the most important objective is to keep the product at the end of the third column as pure as possible while also keeping the whole process in the steady state domain. There are several types of faults that can occur like: plant fault, sensor fault, actuator fault and agent failure. For this particular process in the sensor faults can be included for each column: the temperature sensor in the boiler, the pressure transducers, and the flowmeters. The actuator faults include the flow controllers -important for the proper operation of the cascade, heater present in the boiler of each column, recirculating pumps, vacuum pump for the thermal insulation jacket. In the plant fault category can be included the liquid nitrogen level in the condenser, loss of electrical power and column construction fault. Based on various experimental trials it was concluded that the boiler resistor electrical power supply has a high influence on the isotope separation process. An excessive boiling of the liquid CO may lead to the column flooding compromising in this manner the entire separation process. Hence the present work is focused only on a fraction of the entire control system: the boiler resistor electrical power supply control for all the columns in the cascade.

Fault-tolerant Multi-Agent Fractional order Control System
Based on the extensive work performed by researchers in the field of fault tolerant control (FTC) it can be stated that FTC is a complex combination of the fault detection and isolation scheme (FDI), robust control and reconfigurable control [13]. The FDI scheme is responsible with fault detection and location in the event of a fault/failure of a sensor, actuator or agent so that corrective action can be made to eliminate or minimize the effect on the overall system performance. In this work a non-model based FDI [13] scheme is implemented based on trend analysis monitoring. In order to detect the faults in time, a fault detection function was developed to help predict the behavior of the process; hence possible faults/failure may be detected and avoided. This function calculates a moving average of a signal by taking a sample every 30 seconds for 15 minutes and then calculates the average. The resulted average is then compared with the previous average and if their difference is greater or equal than the threshold it indicates possible fault. This fault detection function is implemented on the following parameter differential pressure, boiler temperature and on the boiler liquid CO level for each column. Once the fault is detected a visual and audible alarm is activated, Figure 1b and the fault tolerant multi-agent control system can take the necessary corrective actions by modifying the necessary set-point for certain flow control loops and putting some equipment in their safe state in order to keep the process in the steady state regime and preserve the obtained ( 13 C) isotope concentration at the output of the third column.

The FT-MAS architecture
The chosen platform to implement the proposed fault tolerant multi-agent fractional order control system is the Java Agent Development Environment (JADE) for two reasons: it is an open source platform and it is developed in accordance with the FIPA (Foundation for Intelligent Physical Agents) standards. Because in industrial practice it is desirable to have distributed control the proposed control architecture is a multi-agent based fault-tolerant fractional order control system which consists of the following intelligent agents: one agent for each column (three autonomous agents) and a supervisor agent. Each column agent is responsible for one column, controlled with a fractional order controller.
Each column agent has the information corresponding to its assigned column, and has the capability to communicate it to the other agents, receiving theirs in return. The supervisor agent is responsible for the exchange of the data between the column cascade system and the column agents and also with monitoring and the control of the set-point levels for each column agent in order to reach the imposed output ( 13 C) isotope concentration. JADE is a distributed platform where software agents run and each instance of a runtime environment is called a container. The main container houses three platform key components: the agent management service (AMS), the directory facilitator (DF) and the agent communication channel (ACC) [13]. The AMS maintains a directory with agent identifiers (AIDs) and agent states and the DF facilitates the discovery of network available agents based on their provided services [13]. So it is necessary to effectively manage a possible main container failure to ensure the platform remain fully operational. This objective can be achieved by main container replication and DF persistence [13]. The mathematical model of the ( 13 C) isotope separation column cascade was implemented in Matlab/SIMULINK, but the S-functions available in Matlab/SIMULINK are unable to handle multiple threads of execution, which represent a critical characteristic of MAS. Hence, MACSimJX, was used to act as a middleware between SIMULINK models and the agents. It presents client-server architecture, separating the MAS from SIMULINK [14] being able to overcome the multiple threads of execution problem bringing MAS closer to the physical model. The proposed fault tolerant multiagent fractional order control architecture is presented in Figure2.

Robust Fractional Order Controller Design
Even if H ∞ control is considered to be one of the most popular robust control methodologies for passive fault tolerant control, the present work tries to improve the robustness of the developed fault tolerant multi agent control system by using a fractional order controller as a behavior method for each column agent. The fractional order controller is practically a generalization of classical PID controller having the following transfer function [7]: where K P , K i , K d are the classical tuning parameters of a PID controller. The complexity of the controller is increased by the introduction of two extra tuning parameters λ and μ, the integration and differentiation orders. The difficulty is in fact represented by the system of nonlinear equations that need to be solved in order to determine the tuning parameter values. The nonlinear equation system is derived from the explicit form of the imposed performance criteria expressed in frequency domain like: magnitude at gain crossover frequency, phase margin, robustness to plant uncertainties, high frequency noise attenuation and sensitivity functions [8]. Previous works [9][10][11] were focused on the development of accurate nonlinear mathematical process model which was used together with experimental data in the validation process of an operational model more fitted for further use in control system design. Several fractional order controllers were also developed in previous works but for only one isotope separation column. Hence the controller parameters are determined using the operational process model developed in [11] and based on the chosen imposed performance criteria: the magnitude at the gain crossover frequency, phase margin and open loop phase derivative in order to ensure a certain closed loop settling time, a certain closed loop overshoot and robustness to process gain variations: In order to determine the controller tuning parameter by using the above described particle swarm optimization method an initial population size of 50 particles was considered. Also the inertia weight considered parameters were m=0.9; n = 1.01; p = 1.1; q = 0.051 and r = 1.01. As a result the following controller parameters were determined: K p =71.51, K d =0.012, K i =16.5, =0.87 and =0.1. The obtained fractional order controllers were implemented using the Oustaloup Recursive Approximation method [9]. The next step after the development of any control system is to assess its performance previously to the practical implementation step. Usually, it is important to test the control system capability to counteract the effects of several disturbances and also to test the reference tracking capabilities. An important source of disturbances is represented by various faults, hence, the developed control architecture, Figure2, was used to prove the capabilities of the proposed fault-tolerant multi-agent fractional order control system to handle various faults through simulation. The considered simulation scenarios were inspired by real operation of the ( 13 C) isotope separation cascade. The first considered fault case scenario is a plant fault at t=3.5[sec] on the first column, Figure3a. The disturbance effect is rejected and only the next in line column feels a small part of the effect. The second considered scenario consists in a simple sensor fault for the first column. The results presented in Figure3b show that the performance of the control system decreases but is still in acceptable limits ensuring the stability of the overall process. The presented simulation results clearly demonstrate the capability of