I'm attempting to reproduce the ADL downstream results reported in the paper using your provided cleaned data data/adl_30hz_clean and the UKB-100k SSL checkpoint at model_check_point/mtl_best.mdl. However, both my Finetune-after-Conv and Finetune-all-layers performances are consistently below the reported performance, with gaps within or around 10%. I suspect this is due to downstream hyperparameters mismatches. Could you please share the exact fine-tuning hyperparameters and any configuration details (e.g., was weighted_loss_fn=true used; did you turn on augmentation=true, etc.) for downstream tasks?
Setting augmentation=false and including two learning rate choices, I fine-tuned on a single GPU node and obtained the following results (means ± SD over 7 LOSO folds).
|
FT after Conv |
|
FT all layers |
|
|
F1 |
Kappa |
F1 |
Kappa |
| Reported |
0.645 ± 0.107 |
0.654 ± 0.123 |
0.829 ± 0.101 |
0.849 ± 0.113 |
| Reproduced (lr=1e-4) |
0.591 ± 0.091 |
0.536 ± 0.126 |
0.786 ± 0.203 |
0.755 ± 0.231 |
| Reproduced (lr=1e-3) |
0.583 ± 0.115 |
0.585 ± 0.156 |
0.684 ± 0.279 |
0.676 ± 0.305 |
I'm attempting to reproduce the ADL downstream results reported in the paper using your provided cleaned data
data/adl_30hz_cleanand the UKB-100k SSL checkpoint atmodel_check_point/mtl_best.mdl. However, both my Finetune-after-Conv and Finetune-all-layers performances are consistently below the reported performance, with gaps within or around 10%. I suspect this is due to downstream hyperparameters mismatches. Could you please share the exact fine-tuning hyperparameters and any configuration details (e.g., wasweighted_loss_fn=trueused; did you turn onaugmentation=true, etc.) for downstream tasks?Setting
augmentation=falseand including two learning rate choices, I fine-tuned on a single GPU node and obtained the following results (means ± SD over 7 LOSO folds).