bert-24.1

This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 11.5138

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
11.3192	0.09	5	12.3266
11.4418	0.18	10	12.2766
11.029	0.27	15	12.2278
11.1589	0.36	20	12.1813
11.1385	0.45	25	12.1361
11.1645	0.55	30	12.0921
10.4417	0.64	35	12.0500
11.0789	0.73	40	12.0100
10.6311	0.82	45	11.9712
10.5261	0.91	50	11.9340
10.2874	1.0	55	11.8991
10.5003	1.09	60	11.8652
10.6206	1.18	65	11.8330
10.8413	1.27	70	11.8025
10.3731	1.36	75	11.7735
10.8143	1.45	80	11.7455
10.5414	1.55	85	11.7199
10.4919	1.64	90	11.6950
10.3187	1.73	95	11.6721
10.5598	1.82	100	11.6508
10.1028	1.91	105	11.6310
10.4634	2.0	110	11.6125
10.3986	2.09	115	11.5958
10.2164	2.18	120	11.5810
10.3932	2.27	125	11.5674
10.5229	2.36	130	11.5549
10.1181	2.45	135	11.5444
10.5176	2.55	140	11.5354
10.0784	2.64	145	11.5279
10.599	2.73	150	11.5223
10.3577	2.82	155	11.5180
10.3107	2.91	160	11.5150
10.5243	3.0	165	11.5138

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1