Improving Robustness of Deep Reinforcement Learning Systems

Abstract

Reinforcement Learning (RL), a branch of machine learning, is used to solve problems that cannot be dealt with supervised and unsupervised learning techniques. In RL, the actor interacts with the environment and learns by getting a reward signal in a trial-and-error fashion. Reinforcement learning is scaled to deep reinforcement learning (DRL) to handle huge state or action space-based problems. In DRL, deep learning models are used for approximation, auto feature extraction, and generalisation across unvisited states and unexplored actions. Though deep reinforcement learning models are generalisable, they may perform catastrophic actions due to noise in the environment or the perceived state. Incorporating robustness to DRL models is of great importance as when these models are deployed to the real systems, may cause irrelevant behaviour due to noisy scenario. Hence, it would cause hardware damage with increased cost and decreased reliability. This thesis presents a study on deep reinforcement learning that covers applications of DRL in different industry verticals, the evolution of DRL, insight of designing Markov decision process (MDP) for various problems, usable simulation tools to apply DRL, and a list of challenges with future direction. We have also assembled a sensor-enabled robot to find the problem (dimensionality perturbation) and considered the robustness aspect of DRL for applications such as industrial control, autonomous driving, autonomous flight, and planetary exploration. These applications motivate us to consider the robustness aspect as a wrong decision in a noisy state will incur a huge cost.