Deep reinforcement learning and population dynamics for water systems control

"We study the control of water-tank systems with linear and nonlinear coupled dynamics. As a way to alleviate the design difficulties and avoid modeling simplifications, we design centralized and decentralized deep reinforcement learning (RL) strategies to control interconnected linear and nonlinear water-tank systems relevant for industrial process control. For the linear variant, we propose a hierarchical control strategy to solve the optimal drainage problem in open-channel systems by combining an optimization technique known as minimum scaled consensus control (MSCC) with the deep deterministic policy gradient (DDPG) algorithm. On the other case, for the nonlinear dynamics we use actor-critic structures for the DDPG and the proximal policy optimization (PPO) algorithm and propose a variant called the multi-critic architecture, which allows the addition of prior knowledge on dominant input-output couplings of multi-input multi-output systems. The proposed approaches for the linear and nonlinear cases show comparable performance with classical control techniques while being completely model independent. Finally, we study the problem of robust resource allocation with momentum using dynamical systems. We propose a class of time-varying differential equations with momentum that achieve acceleration and preserve most of the asymptotic properties of its time-invariant counterpart. Since time-varying dynamics with momentum in continuous-time usually lack of structural robustness properties, we present a hybrid regularization that induces the property of uniform asymptotic stability in the system. We show this by using the invariance principle for well-posed hybrid dynamical systems, and we establish the existence of strictly positive margins of robustness with respect to arbitrarily small disturbances. We illustrate our results via numerical simulations."--Tomado del Formato de Documento de Grado.

• Autores: Ochoa Tamayo, Daniel Esteban
• Fecha de publicación: 2019
• Institución: Universidad de los Andes
• Repositorio: Séneca: repositorio Uniandes
• Idioma: eng
• OAI Identifier: oai:repositorio.uniandes.edu.co:1992/44007
• Acceso en línea: http://hdl.handle.net/1992/44007
• Palabra clave: Sistemas de control - Aplicaciones industriales - Investigaciones; Canales (Ingeniería hidráulica) - Control - Investigaciones; Aprendizaje por refuerzo (Aprendizaje automático) - Aplicaciones - Investigaciones; Ingeniería
• Rights: openAccess
• License: https://repositorio.uniandes.edu.co/static/pdf/aceptacion_uso_es.pdf