Loading...
Augmenting Differentiable Neural Computer with Read Network and Key-Value Memory
Author(s)
Yadav, Alok
Date Issued
January 1, 2021
Type
Conference Paper
Abstract
A Differential Neural Computer (DNC) is a type of neural network architecture that can make use of external memory. A DNC model can represent complex data sequences and reason about them. However, training a complex DNC with large memory matrices was slow, hence this study focused on improving a DNC model to learn faster and perform bAbi question-Answering tasks more accurately. The attempted improvements were to use key-value pairs for memory locations instead of arrays and to obtain a read vector from the memory matrix with a neural network. Evaluation of the improved DNC models on the bAbi dataset showed that their compute time was 13% shorter and their error rate was 6.6% lower. To conclude, the attempted improvements were successful, and the training speed of the key-value memory model also became shorter.
Citation
Icsec 2021 25th International Computer Science and Engineering Conference, 262-266, 2021