| CMD: -n 4 -g 4 -o falcon-40b-ft-alpaca-dolly-dutch/2e-4lr+512tbs+5ep+32a+4r-v2 -t 80 -p falcon-40b-ft-alpaca-dolly-dutch -e --learning_rate 2e-4 --model_name_or_path tiiuae/falcon-40b --per_device_train_batch_size 8 --per_device_eval_batch_size 8 --gradient_accumulation_steps 4 --save_total_limit 3 --lora_alpha 32 --lora_r 4 -i Hyperparameters were taken from LoRA paper. See Table 11. | |
| INFO: Hyperparameters were taken from LoRA paper. See Table 11. | |