Colloquium - Boyu Wang (DSAS)

Speaker: Boyu Wang (Western Univiersity - Brain and Mind Institute)

Title: Gap Minimization for Knowledge Sharing and Transfer

Abstract: Learning from multiple related tasks by knowledge sharing and transfer has become increasingly relevant over the last two decades. In order to successfully transfer information from one task to another, it is critical to understand the similarities and differences between the domains. In this work, we present the notion of performance gap, a simple, intuitive, and novel measure of the distance between different learning tasks. Orthogonal to the existing measures which are used as tools to bound the difference of expected risks between tasks (e.g., $\mathcal{H}$-divergence, discrepancy distance), the performance gap can be viewed as a data and algorithm dependent regularizer, which controls the model complexity and leads to finer guarantees. More importantly, it also provides new insight and motivates new principles for designing new strategies for knowledge sharing and transfer, which is complementary to existing theories and algorithms. We take the instance weighting approach to transfer learning as an example and propose a set of general, principled rules for designing new instance weighting schemes. These rules lead to gapBoost, a novel and principled boosting algorithm that explicitly minimizes the performance gap between source and target domains for transfer learning. Our experimental evaluation on benchmark data sets shows that gapBoost significantly outperforms existing boosting-based transfer learning algorithms.