Julien Perez, Fei Liu
NIPS, Barcelona, Spain, 05 - 10 December 2016
Machine reading using differentiable reasoning models has recently shown remarkable progress. In this context, End-to-End trainable Memory Networks (MemN2N) have demonstrated promising performance on simple natural language based reasoning tasks such as factual reasoning and basic deduction. However, other tasks, namely multi-fact questionanswering, positional reasoning or dialog related tasks, remain challenging particularly due to the necessity of more complex interactions between the memory and controller modules composing this family of models. In this paper, we introduce a novel End-to-End memory access regulation mechanism inspired by the current progress on the connection short-cutting principle in the field of computer vision. Concretely, we develop a Gated End-to-End trainable Memory Network architecture (GMemN2N). From a machine learning perspective, this new capability is learned in an end-to-end fashion without the use of any additional supervision signal which is, as far as our knowledge goes, the first model of this kind. Our experiments show significant improvements on the most challenging tasks in the 20 bAbI, without the use of any domain knowledge. Then, we show improvement on the Dialog bAbI tasks including the real human-bot conversion-based DSTC-2 dataset. On these two datasets, we achieve the new state of the art.
Also Available on arXiv
Report number: