Transfer learning for contextual multi-armed bandits

Abstract:  Transfer learning, which aims to improve learning performance in a target domain by leveraging knowledge from related source domains, has recently attracted significant attention. Despite its empirical success, our theoretical understanding of transfer learning remains highly limited. In this talk, I will discuss recent progress towards characterizing transfer learning in nonparametric contextual multi-armed bandits, where we assume access to data collected from source bandits before learning the target bandit. Focusing on the covariate shift model, we establish the minimax regret and develop a novel transfer learning algorithm that attains this information-theoretic limit. In view of the general impossibility of adaptation to unknown smoothness, we further develop a data-driven algorithm that achieves near-optimal statistical guarantees (up to a logarithmic factor) while automatically adapting to the unknown parameters over a large collection of parameter spaces under proper conditions. Our results offer a theoretical quantification of the contribution of data from source domains to the learning process within the target domain in the context of nonparametric contextual multi-armed bandits. This is based on the joint work with Tony Cai (Penn) and Hongzhe Li (Penn). Paper: https://arxiv.org/abs/2211.12612.

Bio:  Changxiao Cai is an Assistant Professor in the Department of Industrial and Operations Engineering at the University of Michigan. Prior to this, he was a postdoctoral researcher at the University of Pennsylvania. He obtained his Ph.D. in Electrical Engineering from Princeton University in 2021, and his B.E. in Electronic Engineering from Tsinghua University in 2016. His research interests lie broadly in the intersection of machine learning, statistics, and optimization. He is interested in the theoretical and algorithmic aspects of modern data science problems, with an emphasis on the optimal interplay between statistical accuracy/efficiency and computational complexity.