Hugging Face's logo

Snowkcon
/
bge_large_ja_llama3_70

Safetensors
bert
Generated from Trainer
Model card Files Community
Snowkcon commited on
Commit
86ed5f7
•
1 Parent(s): 01a4965

Model save

Browse files
Files changed (9) hide show
  1. README.md +126 -0
  2. final/config.json +41 -0
  3. final/model.safetensors +3 -0
  4. final/special_tokens_map.json +37 -0
  5. final/tokenizer.json +0 -0
  6. final/tokenizer_config.json +57 -0
  7. final/training_args.bin +3 -0
  8. final/vocab.txt +0 -0
  9. model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,126 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ metrics:
5
+ - precision
6
+ - recall
7
+ - accuracy
8
+ model-index:
9
+ - name: bge_large_ja_llama3_70
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # bge_large_ja_llama3_70
17
+
18
+ This model was trained from scratch on an unknown dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 1.0015
21
+ - Precision: 0.4921
22
+ - Recall: 0.3553
23
+ - F1 Macro: 0.3419
24
+ - Accuracy: 0.4442
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 0.0003
44
+ - train_batch_size: 256
45
+ - eval_batch_size: 128
46
+ - seed: 0
47
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
+ - lr_scheduler_type: linear
49
+ - num_epochs: 10
50
+
51
+ ### Training results
52
+
53
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 Macro | Accuracy |
54
+ |:-------------:|:------:|:-----:|:---------------:|:---------:|:------:|:--------:|:--------:|
55
+ | No log | 0 | 0 | 11.2074 | 0.0112 | 0.1667 | 0.0210 | 0.0673 |
56
+ | 1.1011 | 0.1575 | 1000 | 1.1103 | 0.4472 | 0.3212 | 0.3124 | 0.4230 |
57
+ | 1.0928 | 0.3150 | 2000 | 1.0853 | 0.4367 | 0.3324 | 0.3258 | 0.4117 |
58
+ | 1.0651 | 0.4724 | 3000 | 1.0764 | 0.4678 | 0.3383 | 0.3363 | 0.4380 |
59
+ | 1.0585 | 0.6299 | 4000 | 1.0546 | 0.4620 | 0.3380 | 0.3232 | 0.4267 |
60
+ | 1.0513 | 0.7874 | 5000 | 1.0633 | 0.4558 | 0.3414 | 0.3283 | 0.4138 |
61
+ | 1.0411 | 0.9449 | 6000 | 1.0415 | 0.4700 | 0.3430 | 0.3296 | 0.4334 |
62
+ | 1.0391 | 1.1024 | 7000 | 1.0445 | 0.4852 | 0.3415 | 0.3220 | 0.4227 |
63
+ | 1.0373 | 1.2598 | 8000 | 1.0378 | 0.4815 | 0.3483 | 0.3342 | 0.4330 |
64
+ | 1.039 | 1.4173 | 9000 | 1.0394 | 0.4762 | 0.3432 | 0.3273 | 0.4265 |
65
+ | 1.0408 | 1.5748 | 10000 | 1.0313 | 0.4992 | 0.3416 | 0.3253 | 0.4375 |
66
+ | 1.0274 | 1.7323 | 11000 | 1.0287 | 0.4959 | 0.3429 | 0.3265 | 0.4350 |
67
+ | 1.0296 | 1.8898 | 12000 | 1.0346 | 0.4822 | 0.3450 | 0.3278 | 0.4257 |
68
+ | 1.0404 | 2.0472 | 13000 | 1.0310 | 0.4844 | 0.3456 | 0.3349 | 0.4433 |
69
+ | 1.0194 | 2.2047 | 14000 | 1.0234 | 0.4828 | 0.3487 | 0.3325 | 0.4331 |
70
+ | 1.0088 | 2.3622 | 15000 | 1.0236 | 0.4813 | 0.3464 | 0.3315 | 0.4393 |
71
+ | 1.0136 | 2.5197 | 16000 | 1.0215 | 0.4986 | 0.3432 | 0.3247 | 0.4400 |
72
+ | 1.046 | 2.6772 | 17000 | 1.0194 | 0.4953 | 0.3455 | 0.3306 | 0.4412 |
73
+ | 1.0133 | 2.8346 | 18000 | 1.0202 | 0.4843 | 0.3488 | 0.3342 | 0.4428 |
74
+ | 1.0096 | 2.9921 | 19000 | 1.0189 | 0.4915 | 0.3478 | 0.3319 | 0.4338 |
75
+ | 0.9948 | 3.1496 | 20000 | 1.0146 | 0.4964 | 0.3487 | 0.3292 | 0.4358 |
76
+ | 1.0041 | 3.3071 | 21000 | 1.0174 | 0.4640 | 0.3510 | 0.3390 | 0.4335 |
77
+ | 1.0039 | 3.4646 | 22000 | 1.0211 | 0.4621 | 0.3493 | 0.3347 | 0.4304 |
78
+ | 1.0286 | 3.6220 | 23000 | 1.0127 | 0.5012 | 0.3484 | 0.3312 | 0.4398 |
79
+ | 1.0068 | 3.7795 | 24000 | 1.0183 | 0.5036 | 0.3451 | 0.3298 | 0.4475 |
80
+ | 1.0082 | 3.9370 | 25000 | 1.0128 | 0.4801 | 0.3513 | 0.3361 | 0.4377 |
81
+ | 1.0013 | 4.0945 | 26000 | 1.0219 | 0.4976 | 0.3470 | 0.3375 | 0.4465 |
82
+ | 1.0045 | 4.2520 | 27000 | 1.0123 | 0.5015 | 0.3493 | 0.3360 | 0.4447 |
83
+ | 1.0051 | 4.4094 | 28000 | 1.0128 | 0.5018 | 0.3488 | 0.3346 | 0.4453 |
84
+ | 1.0176 | 4.5669 | 29000 | 1.0135 | 0.4759 | 0.3520 | 0.3349 | 0.4357 |
85
+ | 1.0002 | 4.7244 | 30000 | 1.0109 | 0.4927 | 0.3484 | 0.3329 | 0.4439 |
86
+ | 0.9972 | 4.8819 | 31000 | 1.0143 | 0.4823 | 0.3517 | 0.3382 | 0.4337 |
87
+ | 0.9907 | 5.0394 | 32000 | 1.0096 | 0.4955 | 0.3507 | 0.3371 | 0.4433 |
88
+ | 0.9546 | 5.1969 | 33000 | 1.0099 | 0.4847 | 0.3586 | 0.3497 | 0.4420 |
89
+ | 0.9973 | 5.3543 | 34000 | 1.0100 | 0.4911 | 0.3494 | 0.3327 | 0.4426 |
90
+ | 0.9939 | 5.5118 | 35000 | 1.0267 | 0.4651 | 0.3511 | 0.3322 | 0.4220 |
91
+ | 0.9915 | 5.6693 | 36000 | 1.0078 | 0.4861 | 0.3553 | 0.3464 | 0.4452 |
92
+ | 1.0101 | 5.8268 | 37000 | 1.0070 | 0.4952 | 0.3552 | 0.3441 | 0.4441 |
93
+ | 0.9869 | 5.9843 | 38000 | 1.0076 | 0.4970 | 0.3547 | 0.3434 | 0.4447 |
94
+ | 0.9797 | 6.1417 | 39000 | 1.0063 | 0.4946 | 0.3520 | 0.3351 | 0.4430 |
95
+ | 0.9783 | 6.2992 | 40000 | 1.0114 | 0.4984 | 0.3542 | 0.3445 | 0.4484 |
96
+ | 1.0314 | 6.4567 | 41000 | 1.0059 | 0.4927 | 0.3521 | 0.3369 | 0.4414 |
97
+ | 0.9764 | 6.6142 | 42000 | 1.0049 | 0.4976 | 0.3520 | 0.3364 | 0.4438 |
98
+ | 0.9762 | 6.7717 | 43000 | 1.0056 | 0.4935 | 0.3539 | 0.3425 | 0.4456 |
99
+ | 1.0073 | 6.9291 | 44000 | 1.0053 | 0.4774 | 0.3546 | 0.3419 | 0.4395 |
100
+ | 0.9764 | 7.0866 | 45000 | 1.0054 | 0.4871 | 0.3547 | 0.3401 | 0.4398 |
101
+ | 0.9795 | 7.2441 | 46000 | 1.0066 | 0.4928 | 0.3562 | 0.3458 | 0.4456 |
102
+ | 0.9707 | 7.4016 | 47000 | 1.0039 | 0.4905 | 0.3544 | 0.3418 | 0.4434 |
103
+ | 0.9681 | 7.5591 | 48000 | 1.0042 | 0.4873 | 0.3532 | 0.3381 | 0.4441 |
104
+ | 0.982 | 7.7165 | 49000 | 1.0035 | 0.4870 | 0.3539 | 0.3374 | 0.4412 |
105
+ | 0.9967 | 7.8740 | 50000 | 1.0040 | 0.4762 | 0.3565 | 0.3464 | 0.4437 |
106
+ | 0.9871 | 8.0315 | 51000 | 1.0075 | 0.5050 | 0.3523 | 0.3406 | 0.4480 |
107
+ | 0.9654 | 8.1890 | 52000 | 1.0038 | 0.4956 | 0.3518 | 0.3358 | 0.4439 |
108
+ | 0.9897 | 8.3465 | 53000 | 1.0035 | 0.4970 | 0.3512 | 0.3366 | 0.4440 |
109
+ | 0.9958 | 8.5039 | 54000 | 1.0069 | 0.4961 | 0.3536 | 0.3415 | 0.4478 |
110
+ | 0.9969 | 8.6614 | 55000 | 1.0033 | 0.4897 | 0.3542 | 0.3413 | 0.4456 |
111
+ | 0.9899 | 8.8189 | 56000 | 1.0023 | 0.4847 | 0.3542 | 0.3409 | 0.4426 |
112
+ | 0.9766 | 8.9764 | 57000 | 1.0051 | 0.4963 | 0.3546 | 0.3438 | 0.4481 |
113
+ | 0.9827 | 9.1339 | 58000 | 1.0031 | 0.4867 | 0.3557 | 0.3425 | 0.4404 |
114
+ | 0.9878 | 9.2913 | 59000 | 1.0029 | 0.4958 | 0.3536 | 0.3411 | 0.4460 |
115
+ | 0.966 | 9.4488 | 60000 | 1.0020 | 0.4943 | 0.3547 | 0.3409 | 0.4456 |
116
+ | 0.9769 | 9.6063 | 61000 | 1.0022 | 0.4913 | 0.3555 | 0.3435 | 0.4449 |
117
+ | 0.9808 | 9.7638 | 62000 | 1.0019 | 0.4926 | 0.3553 | 0.3406 | 0.4424 |
118
+ | 0.9934 | 9.9213 | 63000 | 1.0015 | 0.4921 | 0.3553 | 0.3419 | 0.4442 |
119
+
120
+
121
+ ### Framework versions
122
+
123
+ - Transformers 4.43.3
124
+ - Pytorch 2.4.0+cu121
125
+ - Datasets 2.20.0
126
+ - Tokenizers 0.19.1
final/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/mnt/workspace/models/bge-large-zh-v1.5",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": 0.0,
9
+ "directionality": "bidi",
10
+ "eos_token_id": 2,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.0,
13
+ "hidden_size": 1024,
14
+ "id2label": {
15
+ "0": "LABEL_0"
16
+ },
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 4096,
19
+ "label2id": {
20
+ "LABEL_0": 0
21
+ },
22
+ "layer_norm_eps": 1e-12,
23
+ "max_position_embeddings": 512,
24
+ "model_type": "bert",
25
+ "num_attention_heads": 16,
26
+ "num_hidden_layers": 24,
27
+ "output_past": true,
28
+ "pad_token_id": 0,
29
+ "pooler_fc_size": 768,
30
+ "pooler_num_attention_heads": 12,
31
+ "pooler_num_fc_layers": 3,
32
+ "pooler_size_per_head": 128,
33
+ "pooler_type": "first_token_transform",
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "regression",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.43.3",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 21128
41
+ }
final/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e23d6a8fa29530abba9638a35007b3bb28f5275b6c82fdd09475bb59e5c72a8e
3
+ size 1302140796
final/special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
final/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
final/tokenizer_config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "never_split": null,
51
+ "pad_token": "[PAD]",
52
+ "sep_token": "[SEP]",
53
+ "strip_accents": null,
54
+ "tokenize_chinese_chars": true,
55
+ "tokenizer_class": "BertTokenizer",
56
+ "unk_token": "[UNK]"
57
+ }
final/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e437824af0ebd2aadda0150e9a2840604969dd7d2e63eb7c814d3a637172e14
3
+ size 5240
final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1c33088ac671809a10777e24b2257e0a4fdfed307b895765654eee389315d565
3
  size 1302140796
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e23d6a8fa29530abba9638a35007b3bb28f5275b6c82fdd09475bb59e5c72a8e
3
  size 1302140796