Word embeddings-based transfer learning for boosted relational dependency networks

Abstract

Machine learning algorithms have proven to be a great asset in different applications. However, traditional machine learning methods assume data is independent identically distributed (i.i.d.) and despises the relational structure of the data, which contains crucial information about how objects participate in relationships and events. Statistical machine learning models are a concise representation of probabilistic dependencies among the attributes of an object. Statistical Relational Learning (SRL) extends statistical learning to represent and learn from data with several objects and their relations. SRL models do not suppose data to be i.i.d. but, as traditional machine learning models, also assume training and testing data are sampled from the same distribution. Transfer learning has emerged as an essential technique to handle scenarios where such an assumption does not hold, as it relies on leveraging the knowledge acquired in one or more learning tasks as a starting point to solve a new task. When employing transfer learning to SRL, the primary challenge is to transfer the learned structure, mapping the vocabulary from a source domain to a different target domain. In this dissertation, we propose \mbox{TransBoostler}, which uses pre-trained word embeddings to guide the mapping as the name of a predicate usually has a semantic connotation that can be mapped to a vector space model. After transferring, TransBoostler employs theory revision to adapt the mapped model to the target data. In the experimental results, TransBoostler has successfully transferred trees from a source to a different target domain. It performs equal or better than previous works and requires less training time for most of the investigated scenarios.