Is this support for hyperparameters tuning?

[zy455761544]

I am looking for a multi-task learning model and found this repo is a good start. Thanks for your contribution! I am wondering if there is any easy way to implement hyperparameters tuning in your codes?

[saransh-mehta]

Hi @zy455761544
You can provide values for following hyper-parameters as arguments to the training script - train.py

epochs
train_batch_size
eval_batch_size
learning_rate
epsilon
grad_accumulation_step
num_of_warmup_steps
grad_clip_value

To know about all the arguments present you can try $ train.py -h from your command line.
In case, you're looking for any specific hyper-parameters, you can let me know.
Thanks!

[zy455761544]

Thank you. I plan to experiment on auto hyper-parameters tuning, using library such as gridserachCV (sklearn) instead of changing those hyper-parameters manually. Maybe I need to modify partial codes inside the train.py

[saransh-mehta]

For auto hyper-parameters tuning, you can find the optimizer/scheduler/ related codes in models/model.py under multiTaskModel class. Libraries like Optuna might also be useful for hyper-parameters tuning.
If you get good results with adding auto-tuning into the codes and wish to contribute, you can raise a pull request and we can add it as a feature too.