Skip to content
Snippets Groups Projects
Commit 0243544e authored by Azhara's avatar Azhara
Browse files

plan

parent 5618ba37
No related branches found
No related tags found
No related merge requests found
...@@ -10,15 +10,20 @@ ...@@ -10,15 +10,20 @@
- feature space synonym replacemnet (emily) - feature space synonym replacemnet (emily)
# hyperparameter # hyperparameter
simple transformers.ai azhara
- learning rate - learning rate [0.0001, 0.0002, 0.0005, 0.001, 0.002, 0.005, 0.01]
- train_batch_size - optimizer on [AdamW, Adafactor]
- num_train_epochs
ella
- early stopping (only req for longer epochs) - early stopping (only req for longer epochs)
- optimizer on (adam w and ada factor) - num_train_epochs [1, 5, 10, 15, 20]
- scheduler
- logging_steps emily
- model (each try one model) - train_batch_size [8, 16, 32, 64, 128]
- scheduler ["linear_schedule_with_warmup", "polynomial_decay_schedule_with_warmup", "constant_schedule_with_warmup"]
# creative stuff
- model ["facebook/bart-large-cnn", "distilroberta-base", "bert-base-cased"]
# for augmentation # for augmentation
...@@ -28,7 +33,7 @@ simple transformers.ai ...@@ -28,7 +33,7 @@ simple transformers.ai
- synonym (azhara) - synonym (azhara)
- percentage of words replacing - percentage of words replacing
- back translation () - back translation (ella)
- which languages, and amount of languages - which languages, and amount of languages
......
This diff is collapsed.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment