Sequential Learning, Optimization and Control for Cyber-Physical Systems

Convergence of dramatic increase in the available data and processing
power, enabled by ubiquitous sensing and computing capabilities is rapidly
changing engineered systems. Cyber-Physical Systems (CPS) refers to
such systems with tightly integrated computational, control and physical
capabilities, like transportation networks, smart cities, autonomous
vehicles, sensor networks, power grids and healthcare systems. However,
designing and implementing CPS involve an array of complex and
challenging tasks: learning and making inference from data, designing
scalable optimization and control methods, and developing decentralized
and adaptive decision making algorithms.
In the first part of the talk, I will discuss an approach for simulation-based
optimization and control of Markov Decision Process (MDP) models, in the
context of CPS. Designing exact optimization and control of such systems
may be intractable due to its complexity. I develop a class of algorithms
called Empirical Dynamic Programming to overcome this difficulty, and
provide provable non-asymptotic performance guarantees.
In the second part of the talk, I will discuss a strategy for sequential
learning and decision making for decentralized CPS. I will first introduce a
multi-player multi-armed bandits framework for modeling this class of
problems. I will then present a sequential learning and decision making
algorithm to solve this problem and show that it achieves optimal
performance. I will also briefly discuss my work on data-driven learning
and control in the context of transportation CPS.
Dileep Kalathil is a postdoctoral scholar in the Department of Electrical
Engineering and Computer Sciences at the University of California,
Berkeley. He received his PhD from University of Southern California
(USC) in 2014 where he won the best PhD Dissertation Prize in the USC
Department of Electrical Engineering. He received an M.Tech from IIT
Madras where he won the award for the best academic performance in the
EE department. His research interests include control theory, sequential
learning, game theory and applied probability.