Название: Adaptive and Learning-Based Control of Safety-Critical Systems
Автор: Max Cohen, Calin Belta
Издательство: Springer
Серия: Synthesis Lectures on Computer Science
Год: 2023
Страниц: 209
Язык: английский
Формат: pdf (true), epub
Размер: 21.5 MB

This book stems from the growing use of learning-based techniques, such as Reinforcement Learning and adaptive control, in the control of autonomous and safety-critical systems. Safety is critical to many applications, such as autonomous driving, air traffic control, and robotics. As these learning-enabled technologies become more prevalent in the control of autonomous systems, it becomes increasingly important to ensure that such systems are safe. To address these challenges, the authors provide a self-contained treatment of learning-based control techniques with rigorous guarantees of stability and safety. This book contains recent results on provably correct control techniques from specifications that go beyond safety and stability, such as temporal logic formulas. The authors bring together control theory, optimization, Machine Learning, and formal methods and present worked-out examples and extensive simulation examples to complement the mathematical style of presentation. Prerequisites are minimal, and the underlying ideas are accessible to readers with only a brief background in control-theoretic ideas, such as Lyapunov stability theory.

Recent advances in Artificial Intelligence (AI) and Machine Learning (ML) have facilitated the design of control and decision-making policies directly from data. For example, advancements in Reinforcement Learning (RL) have enabled robots to learn high-performance control policies purely from trial-and-error interaction with their environment; advances in deep neural network architectures have allowed for learning control policies directly from raw sensory information; advancements in Bayesian inference have allowed for constructing non-parametric models of complicated dynamical systems with probabilistic accuracy guarantees. The fact that these ML techniques can learn control policies and dynamic models directly from data makes them extremely attractive for use in autonomous systems that must operate in the face of uncertainties. Despite these promises, the performance of such ML approaches is tightly coupled to the data used to train the ML model and may act unexpectedly when exposed to data outside its training distribution. That is, although such ML models are extremely expressive at describing the complex input-output relation of the training data, they are not necessarily adaptive in that input data outside of the range of the training dataset may produce unexpected outputs. This phenomenon makes it challenging to directly deploy such learning-based controllers on safety-critical systems that will inevitably encounter unexpected scenarios that cannot be accounted for using pre-existing data.

The main objective of this book is to present a unified framework for the design of controllers that learn from data online with formal guarantees of correctness. We are primarily concerned with ensuring that such learning-based controllers provide safety guarantees, a property formalized using the framework of set invariance. Our focus in this book is on online learning-based control, or adaptive control, in which learning and control occur simultaneously in the feedback loop. Rather than using a controller trained on an a priori dataset collected offline that is then statically deployed on a system, we are interested in using real-time data to continuously update the control policy online and cope with uncertainties that are challenging to characterize until deployment. In this regard, most of the controllers developed in this book are dynamic feedback controllers in that they depend on the states of an auxiliary dynamical system representing an adaptation algorithm that evolves based upon data observed in real-time. This idea is not new—it has been the cornerstone of the field of adaptive control for decades. From this perspective, the main objective of this book is to extend techniques from the field of adaptive control, which has primarily been concerned with stabilization and tracking problems, to consider more complex control specifications, such as safety, that are becoming increasingly relevant in the realm of robotic and autonomous systems.

Intended Audience:
This book is intended to provide an introduction to learning-based control of safety-critical systems for a wide range of scientists, engineers, and researchers. We have attempted to write this book in a self-contained manner—a solid background in vector calculus, linear algebra, and differential equations should be sufficient to grasp most of the mathematical concepts introduced herein. Prior knowledge of nonlinear systems theory would be useful (e.g., an introductory course that covers the basics of Lyapunov stability) as it serves as the starting point for most of the developments in this book, but is not strictly necessary as we briefly cover the concepts used throughout this book in Chap. 2. Researchers from control theory are shown how established control-theoretic tools, such as Lyapunov functions, can be suitably transposed to address problems richer than stabilization and tracking problems. They are also exposed to Machine Learning and its integration with control-theoretic tools with the goal of dealing with uncertainty. Machine Learning researchers are shown how control-theoretic and formal methods tools can be leveraged to provide guarantees of correctness for learning-based control approaches, such as Reinforcement Learning.

Contents: