diff --git a/README.md b/README.md index 6eeae683d7bfd9fdc760a8c0c6ea1b2ceb229c62..39b691d5ff812f5758125dbef977468a20e6bf48 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,32 @@ - back translation (ella) - feature space synonym replacemnet (emily) +# hyperparameter +simple transformers.ai +- learning rate +- train_batch_size +- num_train_epochs +- early stopping (only req for longer epochs) +- optimizer on (adam w and ada factor) +- scheduler +- logging_steps +- model (each try one model) + + +# for augmentation +- percentage of word embeddings replaced in BERT (em) + - how much percentage of all sentences + +- synonym (azhara) + - percentage of words replacing + +- back translation () + - which languages, and amount of languages + + +- evaluate + + # one other newer model than roberta