bobox commited on
Commit
3aba23d
1 Parent(s): 751c5a3

trained on the initial 100k + 100k

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,523 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: []
3
+ library_name: sentence-transformers
4
+ tags:
5
+ - sentence-transformers
6
+ - sentence-similarity
7
+ - feature-extraction
8
+ - generated_from_trainer
9
+ - dataset_size:300000
10
+ - loss:DenoisingAutoEncoderLoss
11
+ base_model: intfloat/e5-base-unsupervised
12
+ datasets: []
13
+ metrics:
14
+ - pearson_cosine
15
+ - spearman_cosine
16
+ - pearson_manhattan
17
+ - spearman_manhattan
18
+ - pearson_euclidean
19
+ - spearman_euclidean
20
+ - pearson_dot
21
+ - spearman_dot
22
+ - pearson_max
23
+ - spearman_max
24
+ widget:
25
+ - source_sentence: One mole of a substance of substance such atoms or). The is known
26
+ or Avogadro's constant
27
+ sentences:
28
+ - how effective are birth control pills and pulling out?
29
+ - can pvc be phthalate free?
30
+ - One mole of a substance is equal to 6.022 × 10²³ units of that substance (such
31
+ as atoms, molecules, or ions). The number 6.022 × 10²³ is known as Avogadro's
32
+ number or Avogadro's constant.
33
+ - source_sentence: is the difference between disability broadly defined a or to be
34
+ significantly impaired relative to the standard an individual group . To the term
35
+ disabled still just more, this or function
36
+ sentences:
37
+ - 'how to open pkf format? On a Windows PC, right-click the file, click "Properties",
38
+ then look under “Type of File.” On a Mac computer, right-click the file, click
39
+ “More Info,” then look under “Kind”. Tip: If it''s the PKF file extension, it
40
+ probably falls under the Audio Files type, so any program used for Audio Files
41
+ should open your PKF file.'
42
+ - When someone dreams you died, it means that whatever you mean to that person's
43
+ psychological state of mind 'has ended' or 'is absent'. ... People dream of dead
44
+ people because they miss something about them that was very strong emotionally
45
+ present when they were there, yet is missing in their daily-life now.
46
+ - what is the difference between disability and disabled? A disability is broadly
47
+ defined as a condition or function judged to be significantly impaired relative
48
+ to the usual standard of an individual or group. ... To most people today the
49
+ term "disabled" still means just that, and, more broadly, means "unable to perform"
50
+ this or that physical or mental function.
51
+ - source_sentence: how you contagious when
52
+ sentences:
53
+ - how long are you contagious when you have rsv?
54
+ - With WiFi on your camera you establish a wireless connection between your camera
55
+ and your phone, tablet, computer, or printer. It's also possible to connect two
56
+ cameras with each other via WiFi. The camera has its own WiFi network that transmits
57
+ signals.
58
+ - So, what does it mean when a guy looks you up and down? It will often mean that
59
+ he is checking you out especially if he only does it to you and he shows other
60
+ signs of attraction when around you. It can also be that he is initially observing
61
+ to see if you're a threat or that he is observing your outfit.
62
+ - source_sentence: you light east while is you can the of the . understanding The
63
+ on left is basically fajr time black you
64
+ sentences:
65
+ - A future - contract to buy (or sell) something in the future. An option - right
66
+ BUT NOT the obligation to buy (or sell) something in the future. A swap - two
67
+ parties exchanging something at agreed points in time. This could be an exchange
68
+ of currencies, of returns on assets, of different interest rate returns, etc..
69
+ - can i connect my iphone to my windows laptop? You can sync an iPhone with a Windows
70
+ 10 computer wirelessly (over your local WiFi network) or via the Lightning cable.
71
+ ... Open iTunes in Windows 10. Plug your iPhone (or iPad or iPod) into the computer
72
+ using a Lightning cable (or older 30-pin connector). Click on Device in iTunes
73
+ and choose your iPhone.
74
+ - 'Yes, Fajr is when you can see the light in the east while Sunrise is when you
75
+ can see the disk of the sun. For those who have a trouble understanding: The blue
76
+ area on the left is basically fajr time. The black area is when you can eat.'
77
+ - source_sentence: should eat diarrhea should solid as soon able you're bottle your
78
+ have, try to them as . at home until 48 last spreading others.
79
+ sentences:
80
+ - which countries were not affected by world war 2? There were eight countries that
81
+ declared neutrality; Portugal, Switzerland, Spain, Sweden, The Vatican, Andorra,
82
+ Ireland and Liechtenstein. However, all of these countries were still involved
83
+ in small ways.
84
+ - how to copy multiple cells in excel and paste?
85
+ - how long should you wait to eat after having diarrhea? You should eat solid food
86
+ as soon as you feel able to. If you're breastfeeding or bottle feeding your baby
87
+ and they have diarrhoea, you should try to feed them as normal. Stay at home until
88
+ at least 48 hours after the last episode of diarrhoea to prevent spreading any
89
+ infection to others.
90
+ pipeline_tag: sentence-similarity
91
+ model-index:
92
+ - name: SentenceTransformer based on intfloat/e5-base-unsupervised
93
+ results:
94
+ - task:
95
+ type: semantic-similarity
96
+ name: Semantic Similarity
97
+ dataset:
98
+ name: sts test
99
+ type: sts-test
100
+ metrics:
101
+ - type: pearson_cosine
102
+ value: 0.7707098586060571
103
+ name: Pearson Cosine
104
+ - type: spearman_cosine
105
+ value: 0.7583632499035035
106
+ name: Spearman Cosine
107
+ - type: pearson_manhattan
108
+ value: 0.7590199401674214
109
+ name: Pearson Manhattan
110
+ - type: spearman_manhattan
111
+ value: 0.747524480818435
112
+ name: Spearman Manhattan
113
+ - type: pearson_euclidean
114
+ value: 0.760482148803808
115
+ name: Pearson Euclidean
116
+ - type: spearman_euclidean
117
+ value: 0.7488744991502696
118
+ name: Spearman Euclidean
119
+ - type: pearson_dot
120
+ value: 0.5774036226110284
121
+ name: Pearson Dot
122
+ - type: spearman_dot
123
+ value: 0.5600384269062831
124
+ name: Spearman Dot
125
+ - type: pearson_max
126
+ value: 0.7707098586060571
127
+ name: Pearson Max
128
+ - type: spearman_max
129
+ value: 0.7583632499035035
130
+ name: Spearman Max
131
+ ---
132
+
133
+ # SentenceTransformer based on intfloat/e5-base-unsupervised
134
+
135
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/e5-base-unsupervised](https://huggingface.co/intfloat/e5-base-unsupervised). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
136
+
137
+ ## Model Details
138
+
139
+ ### Model Description
140
+ - **Model Type:** Sentence Transformer
141
+ - **Base model:** [intfloat/e5-base-unsupervised](https://huggingface.co/intfloat/e5-base-unsupervised) <!-- at revision 6003a5b7ce770b0549203e41115b9fc683f16dad -->
142
+ - **Maximum Sequence Length:** 512 tokens
143
+ - **Output Dimensionality:** 768 tokens
144
+ - **Similarity Function:** Cosine Similarity
145
+ <!-- - **Training Dataset:** Unknown -->
146
+ <!-- - **Language:** Unknown -->
147
+ <!-- - **License:** Unknown -->
148
+
149
+ ### Model Sources
150
+
151
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
152
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
153
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
154
+
155
+ ### Full Model Architecture
156
+
157
+ ```
158
+ SentenceTransformer(
159
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
160
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
161
+ )
162
+ ```
163
+
164
+ ## Usage
165
+
166
+ ### Direct Usage (Sentence Transformers)
167
+
168
+ First install the Sentence Transformers library:
169
+
170
+ ```bash
171
+ pip install -U sentence-transformers
172
+ ```
173
+
174
+ Then you can load this model and run inference.
175
+ ```python
176
+ from sentence_transformers import SentenceTransformer
177
+
178
+ # Download from the 🤗 Hub
179
+ model = SentenceTransformer("bobox/E5-base-unsupervised-TSDAE")
180
+ # Run inference
181
+ sentences = [
182
+ "should eat diarrhea should solid as soon able you're bottle your have, try to them as . at home until 48 last spreading others.",
183
+ "how long should you wait to eat after having diarrhea? You should eat solid food as soon as you feel able to. If you're breastfeeding or bottle feeding your baby and they have diarrhoea, you should try to feed them as normal. Stay at home until at least 48 hours after the last episode of diarrhoea to prevent spreading any infection to others.",
184
+ 'how to copy multiple cells in excel and paste?',
185
+ ]
186
+ embeddings = model.encode(sentences)
187
+ print(embeddings.shape)
188
+ # [3, 768]
189
+
190
+ # Get the similarity scores for the embeddings
191
+ similarities = model.similarity(embeddings, embeddings)
192
+ print(similarities.shape)
193
+ # [3, 3]
194
+ ```
195
+
196
+ <!--
197
+ ### Direct Usage (Transformers)
198
+
199
+ <details><summary>Click to see the direct usage in Transformers</summary>
200
+
201
+ </details>
202
+ -->
203
+
204
+ <!--
205
+ ### Downstream Usage (Sentence Transformers)
206
+
207
+ You can finetune this model on your own dataset.
208
+
209
+ <details><summary>Click to expand</summary>
210
+
211
+ </details>
212
+ -->
213
+
214
+ <!--
215
+ ### Out-of-Scope Use
216
+
217
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
218
+ -->
219
+
220
+ ## Evaluation
221
+
222
+ ### Metrics
223
+
224
+ #### Semantic Similarity
225
+ * Dataset: `sts-test`
226
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
227
+
228
+ | Metric | Value |
229
+ |:--------------------|:-----------|
230
+ | pearson_cosine | 0.7707 |
231
+ | **spearman_cosine** | **0.7584** |
232
+ | pearson_manhattan | 0.759 |
233
+ | spearman_manhattan | 0.7475 |
234
+ | pearson_euclidean | 0.7605 |
235
+ | spearman_euclidean | 0.7489 |
236
+ | pearson_dot | 0.5774 |
237
+ | spearman_dot | 0.56 |
238
+ | pearson_max | 0.7707 |
239
+ | spearman_max | 0.7584 |
240
+
241
+ <!--
242
+ ## Bias, Risks and Limitations
243
+
244
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
245
+ -->
246
+
247
+ <!--
248
+ ### Recommendations
249
+
250
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
251
+ -->
252
+
253
+ ## Training Details
254
+
255
+ ### Training Dataset
256
+
257
+ #### Unnamed Dataset
258
+
259
+
260
+ * Size: 300,000 training samples
261
+ * Columns: <code>sentence_0</code> and <code>sentence_1</code>
262
+ * Approximate statistics based on the first 1000 samples:
263
+ | | sentence_0 | sentence_1 |
264
+ |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
265
+ | type | string | string |
266
+ | details | <ul><li>min: 3 tokens</li><li>mean: 20.46 tokens</li><li>max: 69 tokens</li></ul> | <ul><li>min: 8 tokens</li><li>mean: 47.85 tokens</li><li>max: 132 tokens</li></ul> |
267
+ * Samples:
268
+ | sentence_0 | sentence_1 |
269
+ |:-------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
270
+ | <code>matter An unit of retains all subatomic neutrons Hydrogen (one one neutrons</code> | <code>are particles of matter atoms? An atom is the smallest unit of matter that retains all of the chemical properties of an element. ... Most atoms contain all three of these types of subatomic particles—protons, electrons, and neutrons. Hydrogen (H) is an exception because it typically has one proton and one electron, but no neutrons.</code> |
271
+ | <code>equals how</code> | <code>5 ml equals how many ounces?</code> |
272
+ | <code>"A Country Boy School is poor is forced to its boy to school following official, ignoring mean a jail</code> | <code>"A Country Boy Quits School" by Lao Hsiang is an endearing social satire. It is about a poor Chinese family which is forced to send its boy to school following an official proclamation, ignoring which would mean a jail term.</code> |
273
+ * Loss: [<code>DenoisingAutoEncoderLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#denoisingautoencoderloss)
274
+
275
+ ### Training Hyperparameters
276
+ #### Non-Default Hyperparameters
277
+
278
+ - `eval_strategy`: steps
279
+ - `per_device_train_batch_size`: 14
280
+ - `per_device_eval_batch_size`: 14
281
+ - `num_train_epochs`: 1
282
+ - `multi_dataset_batch_sampler`: round_robin
283
+
284
+ #### All Hyperparameters
285
+ <details><summary>Click to expand</summary>
286
+
287
+ - `overwrite_output_dir`: False
288
+ - `do_predict`: False
289
+ - `eval_strategy`: steps
290
+ - `prediction_loss_only`: True
291
+ - `per_device_train_batch_size`: 14
292
+ - `per_device_eval_batch_size`: 14
293
+ - `per_gpu_train_batch_size`: None
294
+ - `per_gpu_eval_batch_size`: None
295
+ - `gradient_accumulation_steps`: 1
296
+ - `eval_accumulation_steps`: None
297
+ - `learning_rate`: 5e-05
298
+ - `weight_decay`: 0.0
299
+ - `adam_beta1`: 0.9
300
+ - `adam_beta2`: 0.999
301
+ - `adam_epsilon`: 1e-08
302
+ - `max_grad_norm`: 1
303
+ - `num_train_epochs`: 1
304
+ - `max_steps`: -1
305
+ - `lr_scheduler_type`: linear
306
+ - `lr_scheduler_kwargs`: {}
307
+ - `warmup_ratio`: 0.0
308
+ - `warmup_steps`: 0
309
+ - `log_level`: passive
310
+ - `log_level_replica`: warning
311
+ - `log_on_each_node`: True
312
+ - `logging_nan_inf_filter`: True
313
+ - `save_safetensors`: True
314
+ - `save_on_each_node`: False
315
+ - `save_only_model`: False
316
+ - `restore_callback_states_from_checkpoint`: False
317
+ - `no_cuda`: False
318
+ - `use_cpu`: False
319
+ - `use_mps_device`: False
320
+ - `seed`: 42
321
+ - `data_seed`: None
322
+ - `jit_mode_eval`: False
323
+ - `use_ipex`: False
324
+ - `bf16`: False
325
+ - `fp16`: False
326
+ - `fp16_opt_level`: O1
327
+ - `half_precision_backend`: auto
328
+ - `bf16_full_eval`: False
329
+ - `fp16_full_eval`: False
330
+ - `tf32`: None
331
+ - `local_rank`: 0
332
+ - `ddp_backend`: None
333
+ - `tpu_num_cores`: None
334
+ - `tpu_metrics_debug`: False
335
+ - `debug`: []
336
+ - `dataloader_drop_last`: False
337
+ - `dataloader_num_workers`: 0
338
+ - `dataloader_prefetch_factor`: None
339
+ - `past_index`: -1
340
+ - `disable_tqdm`: False
341
+ - `remove_unused_columns`: True
342
+ - `label_names`: None
343
+ - `load_best_model_at_end`: False
344
+ - `ignore_data_skip`: False
345
+ - `fsdp`: []
346
+ - `fsdp_min_num_params`: 0
347
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
348
+ - `fsdp_transformer_layer_cls_to_wrap`: None
349
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
350
+ - `deepspeed`: None
351
+ - `label_smoothing_factor`: 0.0
352
+ - `optim`: adamw_torch
353
+ - `optim_args`: None
354
+ - `adafactor`: False
355
+ - `group_by_length`: False
356
+ - `length_column_name`: length
357
+ - `ddp_find_unused_parameters`: None
358
+ - `ddp_bucket_cap_mb`: None
359
+ - `ddp_broadcast_buffers`: False
360
+ - `dataloader_pin_memory`: True
361
+ - `dataloader_persistent_workers`: False
362
+ - `skip_memory_metrics`: True
363
+ - `use_legacy_prediction_loop`: False
364
+ - `push_to_hub`: False
365
+ - `resume_from_checkpoint`: None
366
+ - `hub_model_id`: None
367
+ - `hub_strategy`: every_save
368
+ - `hub_private_repo`: False
369
+ - `hub_always_push`: False
370
+ - `gradient_checkpointing`: False
371
+ - `gradient_checkpointing_kwargs`: None
372
+ - `include_inputs_for_metrics`: False
373
+ - `eval_do_concat_batches`: True
374
+ - `fp16_backend`: auto
375
+ - `push_to_hub_model_id`: None
376
+ - `push_to_hub_organization`: None
377
+ - `mp_parameters`:
378
+ - `auto_find_batch_size`: False
379
+ - `full_determinism`: False
380
+ - `torchdynamo`: None
381
+ - `ray_scope`: last
382
+ - `ddp_timeout`: 1800
383
+ - `torch_compile`: False
384
+ - `torch_compile_backend`: None
385
+ - `torch_compile_mode`: None
386
+ - `dispatch_batches`: None
387
+ - `split_batches`: None
388
+ - `include_tokens_per_second`: False
389
+ - `include_num_input_tokens_seen`: False
390
+ - `neftune_noise_alpha`: None
391
+ - `optim_target_modules`: None
392
+ - `batch_eval_metrics`: False
393
+ - `batch_sampler`: batch_sampler
394
+ - `multi_dataset_batch_sampler`: round_robin
395
+
396
+ </details>
397
+
398
+ ### Training Logs
399
+ | Epoch | Step | Training Loss | sts-test_spearman_cosine |
400
+ |:------:|:-----:|:-------------:|:------------------------:|
401
+ | 0 | 0 | - | 0.7211 |
402
+ | 0.0233 | 500 | 6.3144 | - |
403
+ | 0.0467 | 1000 | 5.3949 | - |
404
+ | 0.0500 | 1072 | - | 0.6820 |
405
+ | 0.0700 | 1500 | 5.0531 | - |
406
+ | 0.0933 | 2000 | 4.8547 | - |
407
+ | 0.1001 | 2144 | - | 0.7126 |
408
+ | 0.1167 | 2500 | 4.7058 | - |
409
+ | 0.1400 | 3000 | 4.5771 | - |
410
+ | 0.1501 | 3216 | - | 0.7290 |
411
+ | 0.1633 | 3500 | 4.4591 | - |
412
+ | 0.1867 | 4000 | 4.3502 | - |
413
+ | 0.2001 | 4288 | - | 0.7351 |
414
+ | 0.2100 | 4500 | 4.3071 | - |
415
+ | 0.2333 | 5000 | 4.2042 | - |
416
+ | 0.2501 | 5360 | - | 0.7464 |
417
+ | 0.2567 | 5500 | 4.1657 | - |
418
+ | 0.2800 | 6000 | 4.1111 | - |
419
+ | 0.3002 | 6432 | - | 0.7492 |
420
+ | 0.3033 | 6500 | 4.045 | - |
421
+ | 0.3267 | 7000 | 4.017 | - |
422
+ | 0.3500 | 7500 | 3.9651 | - |
423
+ | 0.3502 | 7504 | - | 0.7554 |
424
+ | 0.3733 | 8000 | 3.9199 | - |
425
+ | 0.3967 | 8500 | 3.8691 | - |
426
+ | 0.4002 | 8576 | - | 0.7517 |
427
+ | 0.4200 | 9000 | 3.8563 | - |
428
+ | 0.4433 | 9500 | 3.815 | - |
429
+ | 0.4502 | 9648 | - | 0.7540 |
430
+ | 0.4667 | 10000 | 3.7892 | - |
431
+ | 0.4900 | 10500 | 3.7543 | - |
432
+ | 0.5003 | 10720 | - | 0.7585 |
433
+ | 0.5133 | 11000 | 3.7391 | - |
434
+ | 0.5367 | 11500 | 3.7442 | - |
435
+ | 0.5503 | 11792 | - | 0.7587 |
436
+ | 0.5600 | 12000 | 3.7187 | - |
437
+ | 0.5833 | 12500 | 3.6855 | - |
438
+ | 0.6003 | 12864 | - | 0.7572 |
439
+ | 0.6067 | 13000 | 3.6751 | - |
440
+ | 0.6300 | 13500 | 3.6373 | - |
441
+ | 0.6503 | 13936 | - | 0.7574 |
442
+ | 0.6533 | 14000 | 3.6292 | - |
443
+ | 0.6767 | 14500 | 3.6277 | - |
444
+ | 0.7000 | 15000 | 3.6084 | - |
445
+ | 0.7004 | 15008 | - | 0.7575 |
446
+ | 0.7233 | 15500 | 3.6103 | - |
447
+ | 0.7467 | 16000 | 3.5953 | - |
448
+ | 0.7504 | 16080 | - | 0.7576 |
449
+ | 0.7700 | 16500 | 3.6232 | - |
450
+ | 0.7933 | 17000 | 3.5741 | - |
451
+ | 0.8004 | 17152 | - | 0.7583 |
452
+ | 0.8167 | 17500 | 3.5639 | - |
453
+ | 0.8400 | 18000 | 3.5667 | - |
454
+ | 0.8504 | 18224 | - | 0.7589 |
455
+ | 0.8633 | 18500 | 3.5598 | - |
456
+ | 0.8866 | 19000 | 3.5636 | - |
457
+ | 0.9005 | 19296 | - | 0.7584 |
458
+ | 0.9100 | 19500 | 3.5536 | - |
459
+ | 0.9333 | 20000 | 3.5529 | - |
460
+ | 0.9505 | 20368 | - | 0.7584 |
461
+ | 0.9566 | 20500 | 3.5485 | - |
462
+ | 0.9800 | 21000 | 3.5503 | - |
463
+ | 1.0 | 21429 | - | 0.7584 |
464
+
465
+
466
+ ### Framework Versions
467
+ - Python: 3.10.13
468
+ - Sentence Transformers: 3.0.1
469
+ - Transformers: 4.41.2
470
+ - PyTorch: 2.1.2
471
+ - Accelerate: 0.31.0
472
+ - Datasets: 2.19.2
473
+ - Tokenizers: 0.19.1
474
+
475
+ ## Citation
476
+
477
+ ### BibTeX
478
+
479
+ #### Sentence Transformers
480
+ ```bibtex
481
+ @inproceedings{reimers-2019-sentence-bert,
482
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
483
+ author = "Reimers, Nils and Gurevych, Iryna",
484
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
485
+ month = "11",
486
+ year = "2019",
487
+ publisher = "Association for Computational Linguistics",
488
+ url = "https://arxiv.org/abs/1908.10084",
489
+ }
490
+ ```
491
+
492
+ #### DenoisingAutoEncoderLoss
493
+ ```bibtex
494
+ @inproceedings{wang-2021-TSDAE,
495
+ title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
496
+ author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna",
497
+ booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
498
+ month = nov,
499
+ year = "2021",
500
+ address = "Punta Cana, Dominican Republic",
501
+ publisher = "Association for Computational Linguistics",
502
+ pages = "671--688",
503
+ url = "https://arxiv.org/abs/2104.06979",
504
+ }
505
+ ```
506
+
507
+ <!--
508
+ ## Glossary
509
+
510
+ *Clearly define terms in order to be accessible across audiences.*
511
+ -->
512
+
513
+ <!--
514
+ ## Model Card Authors
515
+
516
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
517
+ -->
518
+
519
+ <!--
520
+ ## Model Card Contact
521
+
522
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
523
+ -->
config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "intfloat/e5-base-unsupervised",
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-12,
15
+ "max_position_embeddings": 512,
16
+ "model_type": "bert",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 0,
20
+ "position_embedding_type": "absolute",
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.41.2",
23
+ "type_vocab_size": 2,
24
+ "use_cache": true,
25
+ "vocab_size": 30522
26
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.0.1",
4
+ "transformers": "4.41.2",
5
+ "pytorch": "2.1.2"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ }
14
+ ]
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f14c5f8cf1bed6f2929d39ef70bd0b7a433bc3d5a16c456119bb5b4d62771f25
3
+ size 437996134
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "mask_token": "[MASK]",
48
+ "model_max_length": 512,
49
+ "pad_token": "[PAD]",
50
+ "sep_token": "[SEP]",
51
+ "strip_accents": null,
52
+ "tokenize_chinese_chars": true,
53
+ "tokenizer_class": "BertTokenizer",
54
+ "unk_token": "[UNK]"
55
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff