README.md 1.51 KB
Newer Older
Wang, Andy's avatar
Wang, Andy committed
1
Preprocessing contains the loading and preprocessing of the data which is then saved as pickle files - the files containing the vocabularised sentences have already been uploaded to this repository. The files with the pre-trained embeddings were too large to upload and have to be created from Preprocessing.
Wang, Andy's avatar
Wang, Andy committed
2

Wang, Andy's avatar
Wang, Andy committed
3
FFNN contains the Feed Forward Network described in Section 4.1 of the report.
Wang, Andy's avatar
Wang, Andy committed
4

Wang, Andy's avatar
Wang, Andy committed
5 6 7 8 9 10 11 12 13
The following models refer to section 4.2 of the report.

GRU_REG uses pretrained embeddings, and splits the data into two batching groups. Each group is then padded with 0 vectors to have the same size. We feed only the last hidden state into the regression network.
GRU_REG_PACKED also uses pretrained embeddings, however we use the pack_padded_sequence functionality for padding. In this file we explored average pooling, max pooling and an attention layer.

The following models refer to section 4.3 of the report.

GRU_REG_EMB uses the same batching methodology as GRU_REG. We learn embeddings for each language seperately.
GRU_REG_EMB_SHARED tries to learn one embedding space for both languages.
Vosylius, Vitalis's avatar
Vosylius, Vitalis committed
14 15
GRU_REG_EMB_PACKED uses one embedding space and the pack_padded_sequence functionality for padding.

Park, Se's avatar
Park, Se committed
16 17 18
XLM_FFNN contains the model described in 4.4 Section of the report as well as some necessary preprocessing in order to use pre-trained XLM model as described in Section 3.

BERT_LSTM_FFNN contains the model described in 4.5 Section of the report as well as some necessary preprocessing in order to use pre-trained BERT model as described in Section 3.