Treffer: Graph Convolutional Encoders for Syntax-aware AMR Parsing

Graph Convolutional Networks (GCNs), a natural architecture for modeling graph-structured data, have recently entered the playing field of NLP as sentence encoders over dependency structure. Contemporary setups of semantic role labeling (SRL), neural machine translation (NMT), and event extraction have demonstrated the superiority of GCNs to CNN and RNN encoders, which expect inherently grid-like or sequential inputs. In this thesis, we explore GCN encoders in a fully neural paradigm of AMR parsing, taking Cai and Lam's (2020) state-of-the-art parser as the framework. We hypothesize that GCN encoders are especially well suited for this problem, following the intuition that syntactic structure strongly informs graph-based semantic structure and can be viewed as an intermediate step towards obtaining it from sequential input. Unlike in previous setups, our GCN encoder has to compete with the extremely successful Transformer baseline (the parser's default encoder), and performs only modestly worse while 1) having an order of magnitude fewer parameters, 2) incorporating explicit syntactic information, and 3) not relying on positional encoding. Our extensive experiments around GCN and Transformer (as well as BiLSTM and GAT) encoder configurations shed light on some of the settings that contribute to the successes of the respective architectures. We confirm that the "syntactic GCN" is the best-performing GCN layer, make empirical observations about Transformers and GCNs based on comparative results and dependency tree statistics, and draw parallels between the Transformer and GCN models in terms of their ability to learn relational structure.