The figure show: Deep Learning Architecture
LSTM (Long-Short Term Memory) neural networks are the standard architecture for many tasks where sequential processing is required such as text classification, language modelling, and analysis of time-series data. The major problem is that such neural networks are slow when compared with CNN (Convolutional Neural Networks). CNNs are mainly used when simultaneous (parallel) processing is required such as image-object detection and automatic video annotation.
Quasi-RNN is an architecture which merges these two architectures thus resulting in parallel processing of sequential data. To our knowledge, this is the first time that a Quasi-RNN was used for the task of dependency parsing.
The next Linguistics Circle seminar is entitled 'Maltese Dependency Parsing using Deep Learning Techniques'. The speaker is Mr Andrei Zammit.
The seminar will be held on Friday 2 November from 12.00 to 14:00 in Room 105 Old Humanities Building (OH105).
Applications such as information retrieval and sentiment analysis depend on natural language processing tools. Dependency parsing is one of the tasks performed in NLP that analyses the grammatical structure of a sentence by determining the relationships between the words in a sentence. Whilst there are several parsers for many European languages, Maltese remains a low-resourced language and currently there is no parser for Maltese.
This work investigates parsing of Maltese by using novel Deep Learning and bootstrapping techniques from multilingual sources, with the aim of contributing to the increase in computational resources for Maltese and also to dependency parsing. Results show an Unlabelled Attachment Score of 90% and Labelled Attachment Score of 86% when using a Quasi-Recurrent Neural Network (QRNN) with a bootstrapped data source of Maltese and other Romance languages. To our knowledge, this is the first time that a QRNN is applied to the task of dependency parsing. Thus, we report on the applicability of this technique for the task of dependency parsing in general.
