Title: Differential Dynamic Programming for Constrained Optimal Control and Reinforcement Learning

Abstract: Constrained optimal control has immense applications and has been studied extensively. On the other hand, reinforcement learning where the agent learns a policy to optimize a pre-defined reward by interacting with the environment has undergone rapid development and found applications in many areas such as robot navigation and manipulation. In this talk, we first present a differential dynamic programming (DDP) framework for solving constrained optimal control with general constraints. We then present some recent development in the inverse optimal control and filtering, followed by a DDP-based framework for inverse reinforcement learning (IRL), aiming to design an appropriate cost function that leads to a desired agent behaviour. An open-loop framework which attempts to minimize the discrepancy between the reproduced trajectory and the demonstrations is first solved. Furthermore, inspired by the DDP based solution, we formulate a closed-loop optimal control problem in the inner level and propose a new loss function that directly captures the feedback nature of the expert data generation. It is shown that the closed-loop method is better than the open-loop one. The application of the proposed control and learning framework in path planning and control of robots will be demonstrated.

Biography: Lihua Xie obtained his BEng and MEng degrees from Nanjing University of Science and Technology in 1983 and 1986, respectively, and PhD degree from the University of Newcastle, Australia, in 1992.  He is currently President’s Chair with the School of Electrical and Electronic Engineering, Nanyang Technological University and Director, Center for Advanced Robotics Technology Innovation (CARTIN) which has received over $30M funding from National Research Foundation of Singapore and industry. He has served as Head of Control and Instrumentation Division and Director of Delta-NTU Corporate Laboratory for Cyber-Physical Systems. His research areas include robust control, multi-agent systems, learning-based control, and unmanned systems. He has published extensively in these areas and has been listed as a highly cited researcher. He is currently an Editor-in-Chief of Unmanned Systems and has served as an Editor of IET Book Series on Control and Associate Editor of IEEE Transactions on Automatic Control, Automatica, IEEE Transactions on Control Systems Technology, IEEE Transactions on Control of Network Systems, etc. He was an IEEE Distinguished Lecturer (2011-2014), a member of Board of Governors of IEEE Control System Society (2016-2018) and the General Chair of the 62nd IEEE Conference on Decision and Control. He is currently Vice-President of IEEE Control System Society. Professor Xie is Fellow of Academy of Engineering Singapore and Fellow of IEEE, IFAC, CAA and AAIA.