bert-30

This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 11.4865

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss
11.2225	0.18	5	12.3054
10.9594	0.36	10	12.2315
11.0948	0.54	15	12.1603
10.8444	0.71	20	12.0935
10.7294	0.89	25	12.0294
10.5432	1.07	30	11.9697
10.71	1.25	35	11.9133
10.5919	1.43	40	11.8601
10.5393	1.61	45	11.8109
10.562	1.79	50	11.7650
10.6558	1.96	55	11.7222
10.4995	2.14	60	11.6835
10.3046	2.32	65	11.6488
10.2809	2.5	70	11.6169
10.6538	2.68	75	11.5887
10.2451	2.86	80	11.5635
10.2671	3.04	85	11.5420
10.7256	3.21	90	11.5243
10.2996	3.39	95	11.5094
10.1658	3.57	100	11.4983
10.4609	3.75	105	11.4906
10.3872	3.93	110	11.4865

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1