Hyper-parameter Tuning for ML Models: A Monte-Carlo Tree Search (MCTS) Approach

Abstract 

We study the application of online learning techniques in the context of hyper-parameter tuning, which is of growing importance in general machine learning. Modern neural networks have several tunable parameters, where training for even one such parameter configuration can take several hours to days. We first cast hyper-parameter tuning as optimizing a multi-fidelity black-box function (which is noise-less) and propose a multi-fidelity tree search algorithm for the same. We then present extensions of our model and algorithm, so that they can function even in the presence of noise. We show that our tree-search based algorithms can outperform state of the art hyper-parameter tuning algorithms on several benchmark data-sets.

Biography