Towards an intelligent HVAC system automation using Reinforcement Learning

HVAC systems are among the biggest energy consumers in buildings and therefore in the focus of optimal control research. In practice, rule-based control and PID controllers are typically used and implemented at the beginning of the building operation. Since this approach neither guarantees optimal or even good control, optimal control algorithms (which can be predictive and adaptive) are in the focus of research. The problem with most of the approaches is that a model of the system is often needed which comes with high engineering efforts. Further, the required computing power can quickly exceed the capacities, even in modern buildings. Therefore, in this paper we investigate the application of a state-of-the-art Reinforcement Learning (RL) algorithm, as a self-calibrating valve controller for two water-air heat exchangers of a real-world air handling unit. We choose a generic problem formulation to pre-train the algorithm with a simulation of an admixing heater and use it to control an injection heater and a throttle cooler. Our results show that after only 70 hours, the control quality significantly increases. Therefore, it seems evident that with pre-trained RL algorithms, a self-improving HVAC automation can be realized with little hardware requirements and without extensive modelling of the system dynamics.