Sequential Decision Making in Cooperative Multi-Agent Systems with Constraints

PASSCODE:  334593

   The decentralized stochastic control literature has extensively studied sequential decision making under uncertainty for cooperative multi-agent systems. The predominant consideration has been the setting where the agents optimize a single long-term cost. The aim of this dissertation is to investigate the more general situation where the team seeks to minimize one long-term (objective) cost while maintaining multiple other long-term (constraint) costs within prescribed limits. Such a setting arises in several real-world application domains—communication networks, traffic management, energy-grid optimization, e-commerce pricing, environmental monitoring, and so on—in the context of efficient operation while maintaining safe operating margins. To achieve this aim, we study in-depth a general mathematical model of sequential decision-making—a cooperative Multi-Agent Constrained Partially ObservableMarkov Decision Process (MA-C-POMDP).

   We establish first results on strong duality and existence of a saddle-point, and develop connections of the MA-C-POMDP model with existing models to derive a primal-dual approach for both model-based as well as data-driven learning of optimal control. We illustrate the applicability of our results by developing team-based optimal control of multiple video streams over the wireless edge. Finally, we conclude the thesis with some reflection and open questions.

CHAIR: Vijay Subramanian