mrm8488

/

t5-small-finetuned-squadv2

like 1

I am trying to use the canonical run_seq2seq_qa to finetune T5-small on SquadV2, but so far I was not able to reach to the f1 accuracy that you have achieved (eval_f1 ~ 66). Do you have any suggestions how to modify the script for a better finetuning?

python3 transformer-sparsity/examples/pytorch/question-answering/run_seq2seq_qa.py \
            --model_name_or_path t5-small \
            --dataset_name squad_v2 \
            --context_column context \
            --question_column question \
            --answer_column answers \
            --do_train \
            --do_eval \
            --per_device_train_batch_size 16 \
            --per_device_eval_batch_size 8 \
            --learning_rate 3e-5 \
            --num_train_epochs 10 \
            --max_seq_length 384 \
            --doc_stride 128 \
            --load_best_model_at_end \
            --eval_steps ${eval_steps} \
            --save_steps ${eval_steps} \
            --evaluation_strategy steps \
            --logging_steps ${eval_steps} \
            --version_2_with_negative \
            --logging_strategy steps \
            --save_total_limit 4 \
            --greater_is_better true \
            --metric_for_best_model f1 \
            --label_names "start_positions", "end_positions" \
            --predict_with_generate \
            --overwrite_output_dir \
            --output_dir ${ckpt_path} 2>&1 | tee ~/${ckpt_path}/finetune_run_$(date +"%Y_%m_%d_%I_%M_%p").log